glusterfs.git/xlators/cluster/afr, branch v3.9dev

afr/index: changes for granular entry self-heal

2016-04-30T13:44:07+00:00

Implements new indices type ENTRY_CHANGES where other
xlators can add/delete names.

Change-Id: I01c5568997085e11d22ba36a4376c70b78fb3827
BUG: 1269461
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/12482
Tested-by: Krutika Dhananjay 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: Entry self-heal performance enhancements

2016-04-30T01:21:56+00:00

Change-Id: I52da41dff5619492b656c2217f4716a6cdadebe0
BUG: 1269461
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/12442
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

dht/afr/client/posix: Fail mkdir without gfid-req

2016-04-29T13:57:39+00:00

Do not allow directory creations without gfids as
after the directories are created, operations
on them fail anyway. So it is better to fail mkdir.

BUG: 1317361
Change-Id: I8f8e3b38bbded1960b7215bac0432500f7e78038
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13690
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/afr: Do not fsync when durability is off

2016-04-27T15:06:35+00:00

BUG: 1329501
Change-Id: Id402c20f2fa19b22bc402295e03e7a0ea96b0c40
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14048
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

afr: propagate child up event after timeout

2016-04-27T07:35:19+00:00

Problem: During mount, afr waits for response from all its children before
notifying the parent xlator. In a 1x2 replica volume , if one of the nodes is
down, the mount will hang for more than a minute until child down is received
from the client xlator for that node.

Fix:
When parent up is received by afr, start a 10 second timer. In the timer call
back, if we receive a successful child up from atleast one brick, propagate the
event to the parent xlator.

Change-Id: I31e57c8802c1a03a4a5d581ee4ab82f3a9c8799d
BUG: 1054694
Signed-off-by: Ravishankar N 
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11113
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

cluster/afr: Fix inode-leak in data self-heal

2016-04-25T06:17:43+00:00

Thanks to Olia-Kremmyda for finding the bug on github review,
https://github.com/gluster/glusterfs/commit/b8106d1127f034ffa88b5dd322c23a10e023b9b6

Change-Id: Ib8640ed0c331a635971d5d12052f0959c24f76a2
BUG: 1329773
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14052
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Krutika Dhananjay

cluster/afr: Fix spurious entries in heal info

2016-04-20T11:51:20+00:00

Problem:
Locking schemes in afr-v1 were locking the directory/file completely during
self-heal. Newer schemes of locking don't require Full directory, file locking.
But afr-v2 still has compatibility code to work-well with older clients, where
in entry-self-heal it takes a lock on a special 256 character name which can't
be created on the fs. Similarly for data self-heal there used to be a lock on
(LLONG_MAX-2, 1). Old locking scheme requires heal info to take sh-domain locks
before examining heal-state.  If it doesn't take sh-domain locks, then there is
a possibility of heal-info hanging till self-heal completes because of
compatibility locks.  But the problem with heal-info taking sh-domain locks is
that if two heal-info or shd, heal-info try to inspect heal state in parallel
using trylocks on sh-domain, there is a possibility that both of them assuming
a heal is in progress. This was leading to spurious entries being shown in
heal-info.

Fix:
As long as there is afr-v1 way of locking, we can't fix this problem with
simple solutions.  If we know that the cluster is running newer versions of
locking schemes, in those cases we can give accurate information in heal-info.
So introduce a new option called 'locking-scheme' which if it is 'granular'
will give correct information in heal-info. Not only that, Extra network hops
for taking compatibility locks, sh-domain locks in heal info will not be
necessary anymore. Thus it improves performance.

BUG: 1322850
Change-Id: Ia563c5f096b5922009ff0ec1c42d969d55d827a3
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13873
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Anuradha Talur 
Reviewed-by: Krutika Dhananjay

cluster/afr: Fix partial heals in 3-way replication

2016-04-15T09:51:06+00:00

Problem:
When there are 2 sources and one sink and if two self-heal daemons
try to acquire locks at the same time, there is a chance that it
gets a lock on one source and sink leading partial to heal. This will
need one more heal from the remaining source to sink for the complete
self-heal. This is not optimal.

Fix:
Upgrade non-blocking locks to blocking lock on all the subvolumes, if
the number of locks acquired is majority and there were eagains.

BUG: 1318751
Change-Id: Iae10b8d3402756c4164b98cc49876056ff7a61e5
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13766
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N

cluster/afr: Fix witness counting code in src/sink detection

2016-04-12T06:27:58+00:00

Problem:
In afr-v1 pre-op, xattrop increments self xattr first then it increments the
value on rest. In post-op, xattr value is decreased first on rest and at last
it gets decremented on self. So for a possible operation to be witnessed i.e.
a fop is seen by the brick it is important to have at least 1 pending op
because without completing pre-op fop won't come. The other possibility is when
fop completes but at the time of post-op after decrementing pending counts on
others just before decrementing its own pending count, the brick dies.

Fix:
Fix witness detection code in afr_self_heal_find_direction()

BUG: 1322253
Change-Id: Ia7e76482c0a46e775e269bb96ec1b9490a3ac18f
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13811
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N

arbiter: write performance improvement

2016-04-11T11:32:37+00:00

Problem: The throughput for a 'dd' workload was much less for arbiter
configuration when compared to normal replica-3 volume. There were 2
issues:

i)arbiter_writev was using the request dict as response dict while
unwinding, leading to incorect GLUSTERFS_WRITE_IS_APPEND and
GLUSTERFS_OPEN_FD_COUNT values (=4), leading to immediate post-ops
because is_afr_delayed_changelog_post_op_needed() failed due to
afr_are_multiple_fds_opened() check.

ii) The arbiter code in afr was setting local->transaction.{start and len} =0
to take full file locks. What this meant was even for simultaenous but
non-overlapping writevs, afr_transaction_eager_lock_init() was not
happening because afr_locals_overlap() always stays true. Consequently
is_afr_delayed_changelog_post_op_needed() failed due to
local->delayed_post_op not being set.

Fix:
i) Send appropriate response dict values in arbiter_writev.
ii) Modify flock params instead of local->transaction.{start and len} to
take full file locks in the transaction.

Also changed _fill_writev_xdata() in posix to fill rsp_xdata for
whatever key is requested for.

Change-Id: I1c5fc5e98aba49ade540bb441a022e65b753432a
BUG: 1324004
Signed-off-by: Ravishankar N 
Reported-by: Robert Rauch 
Reported-by: Russel Purinton 
Reviewed-on: http://review.gluster.org/13906
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System