gluster: Sometimes Brick process is crashed at the time of stopping brick

Problem: Sometimes brick process is getting crashed at the time
         of stop brick while brick mux is enabled.

Solution: Brick process was getting crashed because of rpc connection
          was not cleaning properly while brick mux is enabled.In this patch
          after sending GF_EVENT_CLEANUP notification to xlator(server)
          waits for all rpc client connection destroy for specific xlator.Once rpc
          connections are destroyed in server_rpc_notify for all associated client
          for that brick then call xlator_mem_cleanup for for brick xlator as well as
          all child xlators.To avoid races at the time of cleanup introduce
          two new flags at each xlator cleanup_starting, call_cleanup.

BUG: 1544090
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>

Note: Run all test-cases in separate build (https://review.gluster.org/#/c/19700/)
      with same patch after enable brick mux forcefully, all test cases are
      passed.

Change-Id: Ic4ab9c128df282d146cf1135640281fcb31997bf
updates: bz#1544090

1 files changed, 3 insertions, 3 deletions

diff --git a/libglusterfs/src/graph.c b/libglusterfs/src/graph.c
index d36cf7b3da5..2d560b7f265 100644
--- a/libglusterfs/src/graph.c
+++ b/libglusterfs/src/graph.c
@@ -841,7 +841,7 @@ is_graph_topology_equal (glusterfs_graph_t *graph1, glusterfs_graph_t *graph2)
                 trav2 = trav2->children->xlator;
                 for (ltrav = trav1->children; ltrav; ltrav = ltrav->next) {
                         trav1 = ltrav->xlator;
-                        if (strcmp (trav1->name, trav2->name) == 0) {
+                        if (!trav1->cleanup_starting && !strcmp (trav1->name, trav2->name)) {
                                 break;
                         }
                 }
@@ -1088,7 +1088,7 @@ glusterfs_graph_reconfigure (glusterfs_graph_t *oldgraph,
         new_xl = FIRST_CHILD (new_xl);
 
         for (trav = old_xl->children; trav; trav = trav->next) {
-                if (strcmp (trav->xlator->name, new_xl->name) == 0) {
+                if (!trav->xlator->cleanup_starting && !strcmp (trav->xlator->name, new_xl->name)) {
                         return xlator_tree_reconfigure (trav->xlator, new_xl);
                 }
         }
@@ -1237,7 +1237,7 @@ glusterfs_graph_attach (glusterfs_graph_t *orig_graph, char *path,
                 xl->volfile_id[strlen(xl->volfile_id)-4] = '\0';
         }
 
-        /* TBD: memory leaks everywhere */
+        /* TODO memory leaks everywhere need to free graph in case of error */
         if (glusterfs_graph_prepare (graph, this->ctx, xl->name)) {
                 gf_log (this->name, GF_LOG_WARNING,
                         "failed to prepare graph for xlator %s", xl->name);