glusterfs.git/xlators/performance, branch v3.3.1qa3

performance/io-cache: use pthread_mutex_trylock to hold mutex in statedumps

2012-08-16T14:17:34+00:00

Do not use pthread_mutex_lock and gf_log functions while dumping information
to statedump, to avoid deadlocks.

Change-Id: I6569366856fc2bc0fefb49c8379e2e4337717ce4
BUG: 843787
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/3799
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/write-behind: use pthread_mutex_trylock to hold mutex in statedumps

2012-08-16T05:51:59+00:00

Change-Id: I24c83b1b5e83ef3e38a019043c7fbca13b19ff43
BUG: 841543
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/3815
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/io-threads: Provide option to turn off least-priority

2012-08-12T07:09:06+00:00

RCA:
In cases when self-heal is in progress, self-heal fops are starved
because of least-priority. This affects other fops with conflicting
inode, entry locks with self-heal.

Fix:
This patch provides configuring enable/disable of least-priority.

Additional changes:
Moved RCHECKSUM fop to low instead of least because it will still
affect the performance of other fops if RCHECKSUM is in LEAST
priority.

Tests:
Tested that the enabling/disabling of fops is working fine.
Tested that RCHECKSUM fop priority is assigned LOW when
least-priority is disabled.

BUG: 843704
Signed-off-by: Pranith Kumar K 
Change-Id: I892f99d6d0a3e0ae6c0a280f82e2203af0c346f6
Reviewed-on: http://review.gluster.com/3751
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/read-ahead: use pthread_mutex_trylock to hold mutex in statedumps

2012-08-12T06:58:31+00:00

Change-Id: I4de64915a9c6a46e126ef4a5b987e49de558f827
BUG: 843796
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/3801
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/quick-read: use pthread_mutex_trylock to hold mutex in statedumps

2012-08-12T06:57:02+00:00

Do not use pthread_mutex_lock and gf_log functions while dumping information
to statedump, to avoid deadlocks.

Change-Id: Ic77d96bc52f2a2a32629c0ae20bba797317e0a81
BUG: 843789
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/3800
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

mount/fuse: Use state->lk_lock to print lock information on EAGAIN.

2012-05-25T11:25:20+00:00

Change-Id: I24a4a0b1c8dc0b8e08b380a5bc8efc111ccdb2c3
BUG: 808400
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.com/3438
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/quick-read: fix race-conditions in qr_unlink.

2012-05-23T13:46:07+00:00

The list of fds on which open needs to be done as part of unlink,
was being modified at different places using different locks.
This resulted in a race-condition where open was marked as in-transit,
but fdctx was removed from the list of fds on which open was being
sent even before open was done. Because of this, open_in_transit would
be set forever (as an open was never actually sent, there would be no
open_cbk called and hence we could not reset the variable), blocking
all the future fd based fops on this fd.

Change-Id: Ie84a55bee578869a9a060a094ba28480e7643ae8
BUG: 819490
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.com/3372
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/md-cache: use mdc_fsetattr_cbk as the callback if setattr is on a fd

2012-05-23T09:04:23+00:00

Change-Id: I114a16055540e0cd3317b83b329600251ffe03c3
BUG: 823886
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/3404
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

license: dual license under GPLV2 and LGPLV3+

2012-05-11T05:06:16+00:00

Note that the license was not changed in any of the following:
	.../argp-standalone/...
        .../booster/...
        .../cli/...
        .../contrib/...
        .../extras/...
        .../glusterfsd/...
        .../glusterfs-hadoop/...
        .../mod_clusterfs/...
        .../scheduler/...
        .../swift/...

The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server

Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.

If you find any errors or ommissions please don't hesitate to let me know.

The complete list of files with the license change is:
	libglusterfs/src/byte-order.h
	libglusterfs/src/call-stub.c
	libglusterfs/src/call-stub.h
	libglusterfs/src/checksum.c
	libglusterfs/src/checksum.h
	libglusterfs/src/circ-buff.c
	libglusterfs/src/circ-buff.h
	libglusterfs/src/common-utils.c
	libglusterfs/src/common-utils.h
	libglusterfs/src/compat-errno.c
	libglusterfs/src/compat-errno.h
	libglusterfs/src/compat.c
	libglusterfs/src/compat.h
	libglusterfs/src/daemon.c
	libglusterfs/src/daemon.h
	libglusterfs/src/defaults.c
	libglusterfs/src/defaults.h
	libglusterfs/src/dict.c
	libglusterfs/src/dict.h
	libglusterfs/src/event-history.c
	libglusterfs/src/event-history.h
	libglusterfs/src/event.c
	libglusterfs/src/event.h
	libglusterfs/src/fd-lk.c
	libglusterfs/src/fd-lk.h
	libglusterfs/src/fd.c
	libglusterfs/src/fd.h
	libglusterfs/src/gf-dirent.c
	libglusterfs/src/gf-dirent.h
	libglusterfs/src/globals.c
	libglusterfs/src/globals.h
	libglusterfs/src/glusterfs.h
	libglusterfs/src/graph-print.c
	libglusterfs/src/graph-utils.h
	libglusterfs/src/graph.c
	libglusterfs/src/hashfn.c
	libglusterfs/src/hashfn.h
	libglusterfs/src/iatt.h
	libglusterfs/src/inode.c
	libglusterfs/src/inode.h
	libglusterfs/src/iobuf.c
	libglusterfs/src/iobuf.h
	libglusterfs/src/latency.c
	libglusterfs/src/latency.h
	libglusterfs/src/list.h
	libglusterfs/src/lkowner.h
	libglusterfs/src/locking.h
	libglusterfs/src/logging.c
	libglusterfs/src/logging.h
	libglusterfs/src/mem-pool.c
	libglusterfs/src/mem-pool.h
	libglusterfs/src/mem-types.h
	libglusterfs/src/options.c
	libglusterfs/src/options.h
	libglusterfs/src/rbthash.c
	libglusterfs/src/rbthash.h
	libglusterfs/src/run.c
	libglusterfs/src/run.h
	libglusterfs/src/scheduler.c
	libglusterfs/src/scheduler.h
	libglusterfs/src/stack.c
	libglusterfs/src/stack.h
	libglusterfs/src/statedump.c
	libglusterfs/src/statedump.h
	libglusterfs/src/syncop.c
	libglusterfs/src/syncop.h
	libglusterfs/src/syscall.c
	libglusterfs/src/syscall.h
	libglusterfs/src/timer.c
	libglusterfs/src/timer.h
	libglusterfs/src/trie.c
	libglusterfs/src/trie.h
	libglusterfs/src/xlator.c
	libglusterfs/src/xlator.h
	libglusterfsclient/src/libglusterfsclient-dentry.c
	libglusterfsclient/src/libglusterfsclient-internals.h
	libglusterfsclient/src/libglusterfsclient.c
	libglusterfsclient/src/libglusterfsclient.h
	rpc/rpc-lib/src/auth-glusterfs.c
	rpc/rpc-lib/src/auth-null.c
	rpc/rpc-lib/src/auth-unix.c
	rpc/rpc-lib/src/protocol-common.h
	rpc/rpc-lib/src/rpc-clnt.c
	rpc/rpc-lib/src/rpc-clnt.h
	rpc/rpc-lib/src/rpc-transport.c
	rpc/rpc-lib/src/rpc-transport.h
	rpc/rpc-lib/src/rpcsvc-auth.c
	rpc/rpc-lib/src/rpcsvc-common.h
	rpc/rpc-lib/src/rpcsvc.c
	rpc/rpc-lib/src/rpcsvc.h
	rpc/rpc-lib/src/xdr-common.h
	rpc/rpc-lib/src/xdr-rpc.c
	rpc/rpc-lib/src/xdr-rpc.h
	rpc/rpc-lib/src/xdr-rpcclnt.c
	rpc/rpc-lib/src/xdr-rpcclnt.h
	rpc/rpc-transport/rdma/src/name.c
	rpc/rpc-transport/rdma/src/name.h
	rpc/rpc-transport/rdma/src/rdma.c
	rpc/rpc-transport/rdma/src/rdma.h
	rpc/rpc-transport/socket/src/name.c
	rpc/rpc-transport/socket/src/name.h
	rpc/rpc-transport/socket/src/socket.c
	rpc/rpc-transport/socket/src/socket.h
	xlators/cluster/afr/src/afr-common.c
	xlators/cluster/afr/src/afr-dir-read.c
	xlators/cluster/afr/src/afr-dir-read.h
	xlators/cluster/afr/src/afr-dir-write.c
	xlators/cluster/afr/src/afr-dir-write.h
	xlators/cluster/afr/src/afr-inode-read.c
	xlators/cluster/afr/src/afr-inode-read.h
	xlators/cluster/afr/src/afr-inode-write.c
	xlators/cluster/afr/src/afr-inode-write.h
	xlators/cluster/afr/src/afr-lk-common.c
	xlators/cluster/afr/src/afr-mem-types.h
	xlators/cluster/afr/src/afr-open.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.h
	xlators/cluster/afr/src/afr-self-heal-common.c
	xlators/cluster/afr/src/afr-self-heal-common.h
	xlators/cluster/afr/src/afr-self-heal-data.c
	xlators/cluster/afr/src/afr-self-heal-entry.c
	xlators/cluster/afr/src/afr-self-heal-metadata.c
	xlators/cluster/afr/src/afr-self-heal.h
	xlators/cluster/afr/src/afr-self-heald.c
	xlators/cluster/afr/src/afr-self-heald.h
	xlators/cluster/afr/src/afr-transaction.c
	xlators/cluster/afr/src/afr-transaction.h
	xlators/cluster/afr/src/afr.c
	xlators/cluster/afr/src/afr.h
	xlators/cluster/afr/src/pump.c
	xlators/cluster/afr/src/pump.h
	xlators/cluster/dht/src/dht-common.c
	xlators/cluster/dht/src/dht-common.h
	xlators/cluster/dht/src/dht-diskusage.c
	xlators/cluster/dht/src/dht-hashfn.c
	xlators/cluster/dht/src/dht-helper.c
	xlators/cluster/dht/src/dht-inode-read.c
	xlators/cluster/dht/src/dht-inode-write.c
	xlators/cluster/dht/src/dht-layout.c
	xlators/cluster/dht/src/dht-linkfile.c
	xlators/cluster/dht/src/dht-mem-types.h
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/dht-rename.c
	xlators/cluster/dht/src/dht-selfheal.c
	xlators/cluster/dht/src/dht.c
	xlators/cluster/dht/src/nufa.c
	xlators/cluster/dht/src/switch.c
	xlators/cluster/stripe/src/stripe-helpers.c
	xlators/cluster/stripe/src/stripe-mem-types.h
	xlators/cluster/stripe/src/stripe.c
	xlators/cluster/stripe/src/stripe.h
	xlators/features/index/src/index-mem-types.h ¹
	xlators/features/index/src/index.c ¹
	xlators/features/index/src/index.h ¹
	xlators/performance/io-cache/src/io-cache.c
	xlators/performance/io-cache/src/io-cache.h
	xlators/performance/io-cache/src/ioc-inode.c
	xlators/performance/io-cache/src/ioc-mem-types.h
	xlators/performance/io-cache/src/page.c
	xlators/performance/io-threads/src/io-threads.c
	xlators/performance/io-threads/src/io-threads.h
	xlators/performance/io-threads/src/iot-mem-types.h
	xlators/performance/md-cache/src/md-cache-mem-types.h
	xlators/performance/md-cache/src/md-cache.c
	xlators/performance/quick-read/src/quick-read-mem-types.h
	xlators/performance/quick-read/src/quick-read.c
	xlators/performance/quick-read/src/quick-read.h
	xlators/performance/read-ahead/src/page.c
	xlators/performance/read-ahead/src/read-ahead-mem-types.h
	xlators/performance/read-ahead/src/read-ahead.c
	xlators/performance/read-ahead/src/read-ahead.h
	xlators/performance/symlink-cache/src/symlink-cache.c
	xlators/performance/write-behind/src/write-behind-mem-types.h
	xlators/performance/write-behind/src/write-behind.c
	xlators/protocol/auth/addr/src/addr.c ¹
	xlators/protocol/auth/login/src/login.c ¹
	xlators/protocol/client/src/client-callback.c
	xlators/protocol/client/src/client-handshake.c
	xlators/protocol/client/src/client-helpers.c
	xlators/protocol/client/src/client-lk.c
	xlators/protocol/client/src/client-mem-types.h
	xlators/protocol/client/src/client.c
	xlators/protocol/client/src/client.h
	xlators/protocol/client/src/client3_1-fops.c

¹ Copyright only, license reverted to original

Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

quick-read, md-cache: selectively cache xattr data to conserve memory

2012-05-08T22:34:01+00:00

The md-cache translator can cache items for a long period of time and
thus is sensitive to memory usage associated with the items it
references. This implements two changes to help conserve memory:

- quick-read - Migrate the file content data to a local dict and
	delete from the xdata dict that passes through the stack.
- md-cache - Create a local dict to selectively store the xattr
	data md-cache is interested in. This includes a slight
	optimization to not allocate an empty dict in the case
	where we have not received any xattr's of interest.

I've tested both changes independently and together by running a
readdirp test against several compiled source trees (~340k files).
The base test results in a 7.7GB RSS on the client. The quick-read
modification cuts RSS down to 4.1GB, which is still large due
to md-cache unintelligently caching a large number of empty
dictionaries. The combined modification cuts RSS down to 462MB
on the same workload.

NOTE: Theoretically the md-cache change should supercede the
quick-read change, but practically I save an extra 150MB or so with
both. I already had the change and consider it an improvement that
quick-read clean up after itself.

BUG: 812876
Change-Id: Id59734d12dd6476b0e32480939e633448adb6884
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.com/3268
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy