glusterd: ignore importing volume which is undergoing a delete operation

Problem explanation:

Assuming in a 3 nodes cluster, if N1 originates a delete operation and
while N1's commit phase completes, either glusterd service of N2 or N3
gets disconnected from N1 (before completing the commit phase), N1 will
attempt to end up importing the volume which is in-flight for a delete
in other nodes as a fresh resulting into an incorrect configuration
state.

Fix:

Mark a volume as stage deleted once a volume delete operation passes
it's staging phase and reset this flag during unlock phase. Now during
this intermediate phase if the same volume gets imported to other peers,
it shouldn't considered to be recreated.

An automated .t is quite tough to implement with the current infra.

Test Case:

1. Keep creating and deleting volumes in a loop on a 3 node cluster
2. Simulate n/w failure between the peers (ifdown followed by ifup)
3. Check if output of 'gluster v list | wc -l' is same across all 3
nodes during 1 & 2.

Change-Id: Ifdd5dc39699120258d7fdd42fe2deb9de25c6246
Fixes: bz#1605077
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>

1 files changed, 27 insertions, 3 deletions

diff --git a/xlators/mgmt/glusterd/src/glusterd-utils.c b/xlators/mgmt/glusterd/src/glusterd-utils.c
index 5afeebac78b..362d3b6adc4 100644
--- a/xlators/mgmt/glusterd/src/glusterd-utils.c
+++ b/xlators/mgmt/glusterd/src/glusterd-utils.c
@@ -1718,7 +1718,7 @@ glusterd_volinfo_find_by_volume_id (uuid_t volume_id, glusterd_volinfo_t **volin
 }
 
 int32_t
-glusterd_volinfo_find (char *volname, glusterd_volinfo_t **volinfo)
+glusterd_volinfo_find (const char *volname, glusterd_volinfo_t **volinfo)
 {
         glusterd_volinfo_t      *tmp_volinfo = NULL;
         int32_t                 ret = -1;
@@ -3016,6 +3016,11 @@ glusterd_add_volume_to_dict (glusterd_volinfo_t *volinfo,
         if (ret)
                 goto out;
 
+        snprintf (key, sizeof (key), "%s%d.stage_deleted", prefix, count);
+        ret = dict_set_uint32 (dict, key, (uint32_t)volinfo->stage_deleted);
+        if (ret)
+                goto out;
+
          /* tiering related variables */
 
         snprintf (key, sizeof (key), "%s%d.cold_brick_count", prefix, count);
@@ -3390,6 +3395,7 @@ glusterd_compare_friend_volume (dict_t *peer_data, int32_t count,
         uint32_t                cksum = 0;
         uint32_t                quota_cksum = 0;
         uint32_t                quota_version = 0;
+        uint32_t                stage_deleted = 0;
         int32_t                 version = 0;
         xlator_t                *this = NULL;
 
@@ -3405,9 +3411,15 @@ glusterd_compare_friend_volume (dict_t *peer_data, int32_t count,
                 goto out;
 
         ret = glusterd_volinfo_find (volname, &volinfo);
-
         if (ret) {
-                *status = GLUSTERD_VOL_COMP_UPDATE_REQ;
+                snprintf (key, sizeof (key), "volume%d.stage_deleted", count);
+                ret = dict_get_uint32 (peer_data, key, &stage_deleted);
+                /* stage_deleted = 1 means the volume is still in the process of
+                 * deleting a volume, so we shouldn't be trying to create a
+                 * fresh volume here which would lead to a stale entry
+                 */
+                if (stage_deleted == 0)
+                        *status = GLUSTERD_VOL_COMP_UPDATE_REQ;
                 ret = 0;
                 goto out;
         }
@@ -3949,6 +3961,7 @@ glusterd_import_volinfo (dict_t *peer_data, int count,
         char               *rebalance_id_str = NULL;
         int                op_version        = 0;
         int                client_op_version = 0;
+        uint32_t           stage_deleted     = 0;
 
         GF_ASSERT (peer_data);
         GF_ASSERT (volinfo);
@@ -3961,6 +3974,17 @@ glusterd_import_volinfo (dict_t *peer_data, int count,
                 goto out;
         }
 
+        snprintf (key, sizeof (key), "%s%d.stage_deleted", prefix, count);
+        ret = dict_get_uint32 (peer_data, key, &stage_deleted);
+        /* stage_deleted = 1 means the volume is still in the process of
+         * deleting a volume, so we shouldn't be trying to create a
+         * fresh volume here which would lead to a stale entry
+        */
+        if (stage_deleted) {
+                ret = 0;
+                goto out;
+        }
+
         ret = glusterd_volinfo_new (&new_volinfo);
         if (ret)
                 goto out;