glusterfs.git/xlators/cluster/afr/src/afr-common.c, branch v3.3.2qa4

cluster/afr: detect in-progress creation in lookup and return ENOENT

2013-06-19T05:49:01+00:00

        Port of http://review.gluster.org/4625

if any subvol returned ENOENT while parent entrylk lock was held,
yield and return ENOENT for the entire lookup.

This is how the issue happens:

Multiple clients A, B and C are attempting 'mkdir -p /mnt/a/b/c'

1 Client A is in the middle of mkdir(/a). It has acquired lock.
  It has performed mkdir(/a) on one subvol, and second one is still
  in progress
2 Client B performs a lookup, sees directory /a on one,
  ENOENT on the other, succeeds lookup.
3 Client B performs lookup on /a/b on both subvols, both return ENOENT
  (one subvol because /a/b does not exist, another because /a
  itself does not exist)
4 Client B proceeds to mkdir /a/b. It obtains entrylk on inode=/a with
  basename=b on one subvol, but fails on other subvol as /a is yet to
  be created by Client A.
5 Client A finishes mkdir of /a on other subvol
6 Client C also attempts to create /a/b, lookup returns ENOENT on
  both subvols.
7 Client C tries to obtain entrylk on on inode=/a with basename=b,
  obtains on one subvol (where B had failed), and waits for B to unlock
  on other subvol.
8 Client B finishes mkdir() on one subvol with GFID-1 and completes
  transaction and unlocks
9 Client C gets the lock on the second subvol, At this stage second
  subvol already has /a/b created from Client B, but Client C does not
  check that in the middle of mkdir transaction
10 Client C attempts mkdir /a/b on both subvols. It succeeds on
   ONLY ONE (where Client B could not get lock because of
   missing parent /a dir) with GFID-2, and gets EEXIST from ONE subvol.
This way we have /a/b in GFID mismatch. One subvol got GFID-1 because
Client B performed transaction on only one subvol (because entrylk()
could not be obtained on second subvol because of missing parent dir --
caused by premature/speculative succeeding of lookup() on /a when locks
are detected). Other subvol gets GFID-2 from Client C because while
it was waiting for entrylk() on both subvols, Client B was in the
middle of creating mkdir() on only one subvol, and Client C does not
"expect" this when it is between lock() and pre-op()/op() phase of the
transaction.

Change-Id: I40107d4638ffdcb7b1ff4748c8e5ea92e62697e8
BUG: 860210
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5173
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

afr: Avoid excessive logging in self-heal.

2012-08-17T09:22:02+00:00

- (Excessive) Logging has been very useful as 'bread-crumbs' in
  many a root-cause analyses. This patch aims at avoiding logging when
  the information could be reconstructed using the xattrs, statedump,
  and/or "volume heal" CLI commands.

Change-Id: I8f646cbee44e98495ea6963f9dfcae95375c8900
BUG: 844804
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.com/3827
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Avoid setting split-brain outside inode locks

2012-08-13T04:51:39+00:00

RCA:
The bug is observed because the decision to mark
a file in split-brain is taken outside appropriate locks.
Lookup gathers xattrs outside any lock. The xattrs being
in split-brain in lookup should only be taken as a hint.
Appropriate inodelks should be taken before confirming
a split-brain. Self-heal confirms this at the moment.

Fix:
Self-heals are launched to inspect xattrs when the
data/metadata self-heal options are turned on.
Decision to set/reset split-brain flag is taken inside
appropriate locks.

Known Issue After fix:
If data/metadata self-heal is turned off, inspecting of
xattrs could not be performed so split-brain behavior
does not work correctly if the self-heal options are turned off.
This bug is handled only in upstream.

Change-Id: I59a43d5ce7bf9ca35bff54a51bf4cfa55d717a9e
BUG: 833727
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3691
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Don't reset split-brain when data-self-heal is off

2012-05-29T05:29:59+00:00

Change-Id: Ie3b2c19e0bee85f64d4c2522beaf7d08e46b8a9e
BUG: 804606
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3466
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Assign gfid path if path is NULL in lookup

2012-05-19T12:21:42+00:00

BUG: 820355
Change-Id: I68bea2b8408e741af5fff9e6588ddb87421dbda4
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3374
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Return EIO if read-child < 0 in inode-read fops

2012-05-18T06:02:24+00:00

BUG: 800884
Change-Id: Ia3382d169e96308eaf48fbd717a72d5266e40677
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3351
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

license: dual license under GPLV2 and LGPLV3+

2012-05-11T05:06:16+00:00

Note that the license was not changed in any of the following:
	.../argp-standalone/...
        .../booster/...
        .../cli/...
        .../contrib/...
        .../extras/...
        .../glusterfsd/...
        .../glusterfs-hadoop/...
        .../mod_clusterfs/...
        .../scheduler/...
        .../swift/...

The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server

Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.

If you find any errors or ommissions please don't hesitate to let me know.

The complete list of files with the license change is:
	libglusterfs/src/byte-order.h
	libglusterfs/src/call-stub.c
	libglusterfs/src/call-stub.h
	libglusterfs/src/checksum.c
	libglusterfs/src/checksum.h
	libglusterfs/src/circ-buff.c
	libglusterfs/src/circ-buff.h
	libglusterfs/src/common-utils.c
	libglusterfs/src/common-utils.h
	libglusterfs/src/compat-errno.c
	libglusterfs/src/compat-errno.h
	libglusterfs/src/compat.c
	libglusterfs/src/compat.h
	libglusterfs/src/daemon.c
	libglusterfs/src/daemon.h
	libglusterfs/src/defaults.c
	libglusterfs/src/defaults.h
	libglusterfs/src/dict.c
	libglusterfs/src/dict.h
	libglusterfs/src/event-history.c
	libglusterfs/src/event-history.h
	libglusterfs/src/event.c
	libglusterfs/src/event.h
	libglusterfs/src/fd-lk.c
	libglusterfs/src/fd-lk.h
	libglusterfs/src/fd.c
	libglusterfs/src/fd.h
	libglusterfs/src/gf-dirent.c
	libglusterfs/src/gf-dirent.h
	libglusterfs/src/globals.c
	libglusterfs/src/globals.h
	libglusterfs/src/glusterfs.h
	libglusterfs/src/graph-print.c
	libglusterfs/src/graph-utils.h
	libglusterfs/src/graph.c
	libglusterfs/src/hashfn.c
	libglusterfs/src/hashfn.h
	libglusterfs/src/iatt.h
	libglusterfs/src/inode.c
	libglusterfs/src/inode.h
	libglusterfs/src/iobuf.c
	libglusterfs/src/iobuf.h
	libglusterfs/src/latency.c
	libglusterfs/src/latency.h
	libglusterfs/src/list.h
	libglusterfs/src/lkowner.h
	libglusterfs/src/locking.h
	libglusterfs/src/logging.c
	libglusterfs/src/logging.h
	libglusterfs/src/mem-pool.c
	libglusterfs/src/mem-pool.h
	libglusterfs/src/mem-types.h
	libglusterfs/src/options.c
	libglusterfs/src/options.h
	libglusterfs/src/rbthash.c
	libglusterfs/src/rbthash.h
	libglusterfs/src/run.c
	libglusterfs/src/run.h
	libglusterfs/src/scheduler.c
	libglusterfs/src/scheduler.h
	libglusterfs/src/stack.c
	libglusterfs/src/stack.h
	libglusterfs/src/statedump.c
	libglusterfs/src/statedump.h
	libglusterfs/src/syncop.c
	libglusterfs/src/syncop.h
	libglusterfs/src/syscall.c
	libglusterfs/src/syscall.h
	libglusterfs/src/timer.c
	libglusterfs/src/timer.h
	libglusterfs/src/trie.c
	libglusterfs/src/trie.h
	libglusterfs/src/xlator.c
	libglusterfs/src/xlator.h
	libglusterfsclient/src/libglusterfsclient-dentry.c
	libglusterfsclient/src/libglusterfsclient-internals.h
	libglusterfsclient/src/libglusterfsclient.c
	libglusterfsclient/src/libglusterfsclient.h
	rpc/rpc-lib/src/auth-glusterfs.c
	rpc/rpc-lib/src/auth-null.c
	rpc/rpc-lib/src/auth-unix.c
	rpc/rpc-lib/src/protocol-common.h
	rpc/rpc-lib/src/rpc-clnt.c
	rpc/rpc-lib/src/rpc-clnt.h
	rpc/rpc-lib/src/rpc-transport.c
	rpc/rpc-lib/src/rpc-transport.h
	rpc/rpc-lib/src/rpcsvc-auth.c
	rpc/rpc-lib/src/rpcsvc-common.h
	rpc/rpc-lib/src/rpcsvc.c
	rpc/rpc-lib/src/rpcsvc.h
	rpc/rpc-lib/src/xdr-common.h
	rpc/rpc-lib/src/xdr-rpc.c
	rpc/rpc-lib/src/xdr-rpc.h
	rpc/rpc-lib/src/xdr-rpcclnt.c
	rpc/rpc-lib/src/xdr-rpcclnt.h
	rpc/rpc-transport/rdma/src/name.c
	rpc/rpc-transport/rdma/src/name.h
	rpc/rpc-transport/rdma/src/rdma.c
	rpc/rpc-transport/rdma/src/rdma.h
	rpc/rpc-transport/socket/src/name.c
	rpc/rpc-transport/socket/src/name.h
	rpc/rpc-transport/socket/src/socket.c
	rpc/rpc-transport/socket/src/socket.h
	xlators/cluster/afr/src/afr-common.c
	xlators/cluster/afr/src/afr-dir-read.c
	xlators/cluster/afr/src/afr-dir-read.h
	xlators/cluster/afr/src/afr-dir-write.c
	xlators/cluster/afr/src/afr-dir-write.h
	xlators/cluster/afr/src/afr-inode-read.c
	xlators/cluster/afr/src/afr-inode-read.h
	xlators/cluster/afr/src/afr-inode-write.c
	xlators/cluster/afr/src/afr-inode-write.h
	xlators/cluster/afr/src/afr-lk-common.c
	xlators/cluster/afr/src/afr-mem-types.h
	xlators/cluster/afr/src/afr-open.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.h
	xlators/cluster/afr/src/afr-self-heal-common.c
	xlators/cluster/afr/src/afr-self-heal-common.h
	xlators/cluster/afr/src/afr-self-heal-data.c
	xlators/cluster/afr/src/afr-self-heal-entry.c
	xlators/cluster/afr/src/afr-self-heal-metadata.c
	xlators/cluster/afr/src/afr-self-heal.h
	xlators/cluster/afr/src/afr-self-heald.c
	xlators/cluster/afr/src/afr-self-heald.h
	xlators/cluster/afr/src/afr-transaction.c
	xlators/cluster/afr/src/afr-transaction.h
	xlators/cluster/afr/src/afr.c
	xlators/cluster/afr/src/afr.h
	xlators/cluster/afr/src/pump.c
	xlators/cluster/afr/src/pump.h
	xlators/cluster/dht/src/dht-common.c
	xlators/cluster/dht/src/dht-common.h
	xlators/cluster/dht/src/dht-diskusage.c
	xlators/cluster/dht/src/dht-hashfn.c
	xlators/cluster/dht/src/dht-helper.c
	xlators/cluster/dht/src/dht-inode-read.c
	xlators/cluster/dht/src/dht-inode-write.c
	xlators/cluster/dht/src/dht-layout.c
	xlators/cluster/dht/src/dht-linkfile.c
	xlators/cluster/dht/src/dht-mem-types.h
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/dht-rename.c
	xlators/cluster/dht/src/dht-selfheal.c
	xlators/cluster/dht/src/dht.c
	xlators/cluster/dht/src/nufa.c
	xlators/cluster/dht/src/switch.c
	xlators/cluster/stripe/src/stripe-helpers.c
	xlators/cluster/stripe/src/stripe-mem-types.h
	xlators/cluster/stripe/src/stripe.c
	xlators/cluster/stripe/src/stripe.h
	xlators/features/index/src/index-mem-types.h ¹
	xlators/features/index/src/index.c ¹
	xlators/features/index/src/index.h ¹
	xlators/performance/io-cache/src/io-cache.c
	xlators/performance/io-cache/src/io-cache.h
	xlators/performance/io-cache/src/ioc-inode.c
	xlators/performance/io-cache/src/ioc-mem-types.h
	xlators/performance/io-cache/src/page.c
	xlators/performance/io-threads/src/io-threads.c
	xlators/performance/io-threads/src/io-threads.h
	xlators/performance/io-threads/src/iot-mem-types.h
	xlators/performance/md-cache/src/md-cache-mem-types.h
	xlators/performance/md-cache/src/md-cache.c
	xlators/performance/quick-read/src/quick-read-mem-types.h
	xlators/performance/quick-read/src/quick-read.c
	xlators/performance/quick-read/src/quick-read.h
	xlators/performance/read-ahead/src/page.c
	xlators/performance/read-ahead/src/read-ahead-mem-types.h
	xlators/performance/read-ahead/src/read-ahead.c
	xlators/performance/read-ahead/src/read-ahead.h
	xlators/performance/symlink-cache/src/symlink-cache.c
	xlators/performance/write-behind/src/write-behind-mem-types.h
	xlators/performance/write-behind/src/write-behind.c
	xlators/protocol/auth/addr/src/addr.c ¹
	xlators/protocol/auth/login/src/login.c ¹
	xlators/protocol/client/src/client-callback.c
	xlators/protocol/client/src/client-handshake.c
	xlators/protocol/client/src/client-helpers.c
	xlators/protocol/client/src/client-lk.c
	xlators/protocol/client/src/client-mem-types.h
	xlators/protocol/client/src/client.c
	xlators/protocol/client/src/client.h
	xlators/protocol/client/src/client3_1-fops.c

¹ Copyright only, license reverted to original

Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Self-heald: Dump the event history completely

2012-05-07T07:51:40+00:00

Change-Id: Icf08ef1752795276f88c343d1d74af104095c6cb
BUG: 796579
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3276
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Handle transient parent-entry xactions in lookup

2012-04-19T07:25:17+00:00

This patch addresses the case when the lookup on an entry is performed
while it is being renamed. The lookup can possibly return 2 different
gfids when lookup on one subvol reached before rename and on other
after rename. In such cases the conflicting entry self-heal is
triggered to resolve the issue, but if there are lot entry transactions
going on the parent directory of the entry then the non-blocking
locks could fail resulting in EIO. To avoid this, lookup queries
locks xlator if there are any parent-entrylk on entry's basename.
If afr finds that there are such locks and gfids are differing then
it chooses the file with latest ctime as the iatt of the entry.
This solution is not foolproof, but it decreases the probability of
hitting the EIO. The correct solution is to take blocking locks on
the parent-entry to find out the correct source. Taking blocking
locks in lookup is not good. One stale entry lock can hang the whole
filesystem. So we chose to go with this for now.

Change-Id: Ibebb6c3074f56f80a96893b6bf5b77941e30d400
BUG: 765551
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3179
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: Perform gfid-less lookup in afr

2012-04-12T22:26:48+00:00

Change-Id: I78d9f0563e25047f392675ae32db38d2c94f6651
BUG: 795355
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/3129
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati