cluster/afr: Fix witness counting code in src/sink detection

Problem:
In afr-v1 pre-op, xattrop increments self xattr first then it increments the
value on rest. In post-op, xattr value is decreased first on rest and at last
it gets decremented on self. So for a possible operation to be witnessed i.e.
a fop is seen by the brick it is important to have at least 1 pending op
because without completing pre-op fop won't come. The other possibility is when
fop completes but at the time of post-op after decrementing pending counts on
others just before decrementing its own pending count, the brick dies.

Fix:
Fix witness detection code in afr_self_heal_find_direction()

BUG: 1322253
Change-Id: Ia7e76482c0a46e775e269bb96ec1b9490a3ac18f
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13811
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>

1 files changed, 19 insertions, 7 deletions

diff --git a/xlators/cluster/afr/src/afr-self-heal-common.c b/xlators/cluster/afr/src/afr-self-heal-common.c
index 3f4e9703546..be4cc75910e 100644
--- a/xlators/cluster/afr/src/afr-self-heal-common.c
+++ b/xlators/cluster/afr/src/afr-self-heal-common.c
@@ -696,15 +696,27 @@ afr_selfheal_find_direction (call_frame_t *frame, xlator_t *this,
                 }
         }
 
-        /* In afr-v1 if a file is self-accused but didn't have any pending
-         * operations on others then it is similar to 'dirty' in afr-v2.
-         * Consider such cases as witness.
-         */
-        for (i = 0; i < priv->child_count; i++) {
-                if (self_accused[i] && !pending[i])
-                        witness[i] += matrix[i][i];
+        /* One more class of witness similar to dirty in v2 is where no pending
+         * exists but we have self-accusing markers. This can happen in afr-v1
+         * if the brick crashes just after doing xattrop on self but
+         * before xattrop on the other xattrs on the brick in pre-op. */
+        if (AFR_COUNT (pending, priv->child_count) == 0) {
+                for (i = 0; i < priv->child_count; i++) {
+                        if (self_accused[i])
+                                witness[i] += matrix[i][i];
+                }
+        } else {
+                /* In afr-v1 if a file is self-accused and has pending
+                 * operations on others then it is similar to 'dirty' in afr-v2.
+                 * Consider such cases as witness.
+                 */
+                for (i = 0; i < priv->child_count; i++) {
+                        if (self_accused[i] && pending[i])
+                                witness[i] += matrix[i][i];
+                }
         }
 
+
         /* count the number of dirty fops witnessed */
         for (i = 0; i < priv->child_count; i++)
                 witness[i] += dirty[i];