geo-rep: Fix for "Transport End Point not connected" issue

problem: Geo-rep gsyncd process mounts the master and slave volume
         on master nodes and slave nodes respectively and starts
         the sync. But it doesn't wait for the mount to be in ready
         state to accept I/O. The gluster mount is considered to be
         ready when all the distribute sub-volumes is up. If the all
         the distribute subvolumes are not up, it can cause ENOTCONN
         error, when lookup on file comes and file is on the subvol
         that is down.

solution: Added a Virtual Xattr "dht.subvol.status" which returns "1"
          if all subvols are up and "0" if all subvols are not up.
          Geo-rep then uses this virtual xattr after a fresh mount, to
          check whether all subvols are up or not and then starts the
          I/O.

fixes: bz#1664335
Change-Id: If3ad01d728b1372da7c08ccbe75a45bdc1ab2a91
Signed-off-by: Harpreet Kaur <hlalwani@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>

1 files changed, 11 insertions, 0 deletions

diff --git a/geo-replication/syncdaemon/resource.py b/geo-replication/syncdaemon/resource.py
index ae5600d1d9a..c0ed408cf49 100644
--- a/geo-replication/syncdaemon/resource.py
+++ b/geo-replication/syncdaemon/resource.py
@@ -37,6 +37,7 @@ from syncdutils import NoStimeAvailable, PartialHistoryAvailable
 from syncdutils import ChangelogException, ChangelogHistoryNotAvailable
 from syncdutils import get_changelog_log_level, get_rsync_version
 from syncdutils import GX_GFID_CANONICAL_LEN
+from syncdutils import gf_mount_ready
 from gsyncdstatus import GeorepStatus
 from syncdutils import lf, Popen, sup
 from syncdutils import Xattr, matching_disk_gfid, get_gfid_from_mnt
@@ -950,6 +951,16 @@ class Mounter(object):
                 logging.exception('mount cleanup failure:')
                 rv = 200
             os._exit(rv)
+
+        #Polling the dht.subvol.status value.
+        RETRIES = 10
+        while not gf_mount_ready():
+            if RETRIES < 0:
+                logging.error('Subvols are not up')
+                break
+            RETRIES -= 1
+            time.sleep(0.2)
+
         logging.debug('auxiliary glusterfs mount prepared')