glusterd: Brick process fails to come up with brickmux on

Issue:
1- In a cluster of 3 Nodes N1, N2, N3. Create 3 volumes vol1,
vol2, vol3 with 3 bricks (one from each node)
2- Set cluster.brick-multiplex on
3- Start all 3 volumes
4- Check if all bricks on a node are running on same port
5- Kill N1
6- Set performance.readdir-ahead for volumes vol1, vol2, vol3
7- Bring N1 up and check volume status
8- All bricks processes not running on N1.

Root Cause -
Since, There is a diff in volfile versions in N1 as compared
to N2 and N3 therefore glusterd_import_friend_volume() is called.
glusterd_import_friend_volume() copies the new_volinfo and deletes
old_volinfo and then calls glusterd_start_bricks().
glusterd_start_bricks() looks for the volfiles and sends an rpc
request to glusterfs_handle_attach(). Now, since the volinfo
has been deleted by glusterd_delete_stale_volume()
from priv->volumes list before glusterd_start_bricks() and
glusterd_create_volfiles_and_notify_services() and
glusterd_list_add_order is called after glusterd_start_bricks(),
therefore the attach RPC req gets an empty volfile path
and that causes the brick to crash.

Fix- Call glusterd_list_add_order() and
glusterd_create_volfiles_and_notify_services before
glusterd_start_bricks() cal is made in glusterd_import_friend_volume

Change-Id: Idfe0e8710f7eb77ca3ddfa1cabeb45b2987f41aa
Fixes: bz#1773856
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>

1 files changed, 8 insertions, 6 deletions

diff --git a/xlators/mgmt/glusterd/src/glusterd-utils.c b/xlators/mgmt/glusterd/src/glusterd-utils.c
index 57ff41d0595..c947e6c926b 100644
--- a/xlators/mgmt/glusterd/src/glusterd-utils.c
+++ b/xlators/mgmt/glusterd/src/glusterd-utils.c
@@ -4964,6 +4964,13 @@ glusterd_import_friend_volume(dict_t *peer_data, int count)
         goto out;
     }
 
+    ret = glusterd_create_volfiles(new_volinfo);
+    if (ret)
+        goto out;
+
+    glusterd_list_add_order(&new_volinfo->vol_list, &priv->volumes,
+                            glusterd_compare_volume_name);
+
     if (glusterd_is_volume_started(new_volinfo)) {
         (void)glusterd_start_bricks(new_volinfo);
         if (glusterd_is_snapd_enabled(new_volinfo)) {
@@ -4978,19 +4985,14 @@ glusterd_import_friend_volume(dict_t *peer_data, int count)
         }
     }
 
-    ret = glusterd_create_volfiles_and_notify_services(new_volinfo);
-    if (ret)
-        goto out;
-
     ret = glusterd_import_quota_conf(peer_data, count, new_volinfo, "volume");
     if (ret) {
         gf_event(EVENT_IMPORT_QUOTA_CONF_FAILED, "volume=%s",
                  new_volinfo->volname);
         goto out;
     }
-    glusterd_list_add_order(&new_volinfo->vol_list, &priv->volumes,
-                            glusterd_compare_volume_name);
 
+    ret = glusterd_fetchspec_notify(this);
 out:
     gf_msg_debug("glusterd", 0, "Returning with ret: %d", ret);
     return ret;