summaryrefslogtreecommitdiffstats
path: root/geo-replication/syncdaemon
Commit message (Collapse)AuthorAgeFilesLines
* geo-rep: Do not restart workers when log-rsync-performance config changeAravinda VK2017-01-051-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Geo-rep restarts workers when any of the configurations changed. We don't need to restart workers if tunables like log-rsync-performance is modified. With this patch, Geo-rep workers will get new "log-rsync-performance" config automatically without restart. > Reviewed-on: http://review.gluster.org/15816 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1402727 Change-Id: I40ec253892ea7e70c727fa5d3c540a11e891897b Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit a268e2865c21ec8d2b4fed26715e986cfcc66fad) Reviewed-on: http://review.gluster.org/16070 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix Last synced status column issue during Hybrid CrawlAravinda VK2016-12-021-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | During Hybrid crawl, Geo-rep maintains stime xattr in subdirectories along with the Brick root. This is done to skip directories if Geo-rep crashes before Hybrid crawl completes. Update Last synced status only when stime xattr updated in brick root. Status output will mislead if it shows sub directory stime as last synced time. > Reviewed-on: http://review.gluster.org/15869 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1399470 Change-Id: I5b73aee7ae4a1c1e2d1001d1f55559b9f9efd6e6 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 1876454d2e7950f25d1e5bb8e2c07ab27d521498) Reviewed-on: http://review.gluster.org/15962 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Handle ENOENT during unlinkAravinda VK2016-12-021-2/+4
| | | | | | | | | | | | | | | | | | | | | Do not raise traceback if a file/dir not exists during unlink or rmdir > Reviewed-on: http://review.gluster.org/15868 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> BUG: 1399092 Change-Id: Idd43ca1fa6ae6056c3cd493f0e2f151880a3968c Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit ecd6da0a754f21909dbbd8189228f5a27a15df3e) Reviewed-on: http://review.gluster.org/15940 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Upgrade conf file only if it is session configAravinda VK2016-10-251-1/+4
| | | | | | | | | | | | | | | | | | | | | Ignore config upgrade if it is template config file present in /var/lib/glusterd/geo-replication/gsyncd_template.conf > Reviewed-on: http://review.gluster.org/15669 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> BUG: 1388150 Change-Id: I2cbba3103b6801c16ff57f778a90b9a0bb2467cf Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 1506c7a98d8d3b31e68d0f214ab331f28ffa9fb5) Reviewed-on: http://review.gluster.org/15715 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Logging improvementsAravinda VK2016-10-254-29/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | - Redundant log messages removed. - Worker and connected slave node details added in "starting worker" log - Added log for Monitor state change - Added log for Worker status change(Initializing/Active/Passive/Faulty) - Added log for Crawl status Change - Added log for config set and reset - Added log for checkpoint set, reset and completion > Reviewed-on: http://review.gluster.org/15684 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1387990 Change-Id: Icc7173ff3c93de4b862bdb1a61760db7eaf14271 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit cdc30ed8eacb6772e0dabb863ef51cef794d60dd) Reviewed-on: http://review.gluster.org/15710 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep/eventsapi: Additional EventsAravinda VK2016-10-195-8/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added following events EVENT_GEOREP_ACTIVE { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_ACTIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_PASSIVE { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_PASSIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_CHECKPOINT_COMPLETED { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_ACTIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH, "checkpoint_time": CHECKPOINT_TIME, "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME } } > Reviewed-on: http://review.gluster.org/15630 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> > Tested-by: Kotresh HR <khiremat@redhat.com> BUG: 1386178 Change-Id: I90716175868c59dd65c8d202e73e0ede90347b6a Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 3830b48b6a46854d6597a36b6f2089ac1e486eb5) Reviewed-on: http://review.gluster.org/15672 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* eventsapi/geo-rep: Geo-rep will not work without eventsapi rpmsAravinda VK2016-09-292-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If glusterfs-events rpm is not installed, Geo-replication will fail since it imports eventtypes. Any call to gsyncd will fail with Import error. Glusterd start fails since it runs `gsyncd.py --version` Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 29, in <module> from syncdutils import FreeObject, norm, grabpidfile, finalize File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 28, in <module> from events import eventtypes ImportError: No module named events > Reviewed-on: http://review.gluster.org/15539 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1380252 Change-Id: I1a9bc086c3d52449ec7296cb2f9ceb16cd41a8a4 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 567dee257e092401cdf0a62d4b89e13c39a0a5aa) Reviewed-on: http://review.gluster.org/15594 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Fix Geo-rep status if monitor.pid file not existsAravinda VK2016-09-141-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If monitor.pid file not exists, gsyncd fails with following traceback Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 201, in main main_i() File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 681, in main_i brick_status.print_status(checkpoint_time=checkpoint_time) File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 343, in print_status for key, value in self.get_status(checkpoint_time).items(): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 262, in get_status with open(self.monitor_pid_file, "r+") as f: IOError: [Errno 2] No such file or directory: '/var/lib/glusterd/ geo-replication/master_node_slave/monitor.pid' If Georep status command this worker's status will not be displayed since not returning expected status output. > Reviewed-on: http://review.gluster.org/15416 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374630 Change-Id: I600a2f5d9617f993d635b9bc6e393108500db5f9 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit c7118a92f52a2fa33ab69f3e3ef1bdabfee847cf) Reviewed-on: http://review.gluster.org/15446 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Defunct tar process after syncAravinda VK2016-09-141-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | After every sync iteration with tarssh mode leaves defunct tar process. Added wait for tar process to prevent this issue. > Reviewed-on: http://review.gluster.org/15426 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1375543 Change-Id: I9953239ef601cc1970c814b00074b45eb00f481e Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 6b30e9bf5a612e105eb7ded0a89ef25fd8530ba5) Reviewed-on: http://review.gluster.org/15488 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Fix ESTALE/EINVAL issue during set_{xtime,stime}Aravinda VK2016-09-141-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Setfattr may get ESTALE/EINVAL if a file is being unlinked. To prevent worker crashing, added retry for these error messages. On second retry it will get ENOENT and that error is handled by ignoring. > Reviewed-on: http://review.gluster.org/15404 > Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> BUG: 1374626 Change-Id: Ic660fa13208366d57c8d3d492bbef611475e45b7 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 9db2826cd8dbcc0fb58f5434c21d00ca5c15c491) Reviewed-on: http://review.gluster.org/15445 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Use configured log_level for libgfchangelog logsAravinda VK2016-09-144-2/+21
| | | | | | | | | | | | | | | | | | | | | | | libgfchangelog was not respecting the log_level configured in Geo-replication. With this patch Libgfchangelog log level can be configured using `config changelog_log_level TRACE`. Default Changelog log level is INFO > Reviewed-on: http://review.gluster.org/15078 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374608 Change-Id: Ida714931129f6a1331b9d0815da77efcb2b898e3 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit f41ec4fcfaa9ca976fddbe9e91aabf840c20111f) Reviewed-on: http://review.gluster.org/15444 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Fix logging sync failuresAravinda VK2016-09-141-1/+8
| | | | | | | | | | | | | | | | | | | | | | If Rsync/Tar subprocess dies, while logging error Geo-rep fails with EBADF while accessing error file. Also worker dies while accessing elines before it is set. > Reviewed-on: http://review.gluster.org/15379 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374597 Change-Id: I9cfce116e8aafa4a98654f5190d40a455af8ec95 Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit c0f877c0374d97e0bee17aac4850d7655a35e61b) Reviewed-on: http://review.gluster.org/15441 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Handle EISDIR error during UnlinkAravinda VK2016-09-141-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During Rename, If Source and Target has same inode then Geo-rep unlinks source. But if source is a directory then this will fail with below traceback Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 113, in worker res = getattr(self.obj, rmeth)(*in_data[2:]) File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 772, in entry_ops os.unlink(entry) OSError: [Errno 21] Is a directory: '.gfid/12711ebf-7fdc-4f4b-9850-2d75581eb 452/New folder' With this patch, if EISDIR, rmdir is tried. Logs error in Slave log in case of ENOTEMPTY. > Reviewed-on: http://review.gluster.org/15132 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> BUG: 1374581 Change-Id: I099af4192adac5125c0a23988ceb6506f91e987f Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 8a6236e4cfc7bf86e881e5c770a19040a6060ad0) Change-Id: Ic3fcf3fa3c049c0d820db86b93e04f6e239deb45 Reviewed-on: http://review.gluster.org/15435 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
* geo-rep: Fix History post processAravinda VK2016-09-085-320/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch removes changelogsdb part of post processing since not got much performance advantage as expected. Entry stime and other logging improvements retained. > Reviewed-on: http://review.gluster.org/15371 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kotresh HR <khiremat@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> BUG: 1374153 Change-Id: Ib99d23f09d96c14bc28225b47d9134260f5551bf Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 5de500cd0116796ff797099c60d33258bd48ce3c) Change-Id: If4ac94de557dd79a1e7c43e01ea8a1d6566569f4 Reviewed-on: http://review.gluster.org/15424 Tested-by: Aravinda VK <avishwan@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: add geo-rep events for server side changesSaravanakumar Arumugam2016-08-315-25/+58
| | | | | | | | | | | | | | | | | Event Type defined in #15351 to avoid merge conflicts Add geo-rep events applicable to changes in geo-rep session in the server side. Change-Id: Ia66574d2abccad7fce6a96667efbc7c6c8903fc6 BUG: 1370445 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/15328 Tested-by: Aravinda VK <avishwan@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Post process Data and Meta ChangelogsAravinda VK2016-08-267-186/+516
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this patch, Data and Meta GFIDs are post processed. If Changelog has UNLINK entry then remove from Data and Meta GFIDs list(If stat on GFID is ENOENT in Master). While processing Changelogs, - Collect all the data and meta operations in a temporary database - Delete all Data and Meta GFIDs which are already unlinked as per Changelogs (unlink only if stat on GFID is ENOENT) - Process all Entry operations as usual - Process data and meta operations in batch(Fetch from Db in batch) - Data sync is again batched based on number of changelogs(Default 1day changelogs). Once the sync is complete, Update last Changelog's time as last_synced time as usual. Additionally maintain entry_stime on Brick root, ignore Entry ops if changelog suffix time is less than entry_stime. If data stime is more than entry_stime, this can happen only when passive worker updates stime by itself by getting mount point stime. Use entry_stime = data_stime in this case. New configurations: max-rsync-retries - Default Value is 10 max-data-changelogs-in-batch - Max number of changelogs to be considered in a batch for syncing. Default value is 5760(4 changelogs per min * 60 min * 24 hours) max-history-changelogs-in-batch - Max number of history changelogs to be processed at once. Default value 86400(4 changelogs per min * 60 min * 24 hours * 15 days) BUG: 1364420 Change-Id: I7b665895bf4806035c2a8573d361257cbadbea17 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15110 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* features/libgfchangelog: Log failure in gf_histroy_changelogKotresh HR2016-08-093-2/+10
| | | | | | | | | | | | | | | | | Add error logs if gf_history_changelog fails. If requested changelog range is not available, log the error and exit instead of continuing the loop and exiting in readdir without logging. Also fixed the duplicate MSGID number in 'changelog-lib-messages.h' Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd BUG: 1362151 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15064 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Handle Config parser errorsAravinda VK2016-07-122-10/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | Python ConfigParser lib has two methods, readfp and read, it should be used as follows. readfp(open("defaults.conf")) read("custom.conf") ConfigParser.read(path) ignores any file errors, which is intentional since errors are handled while loading default config. Geo-rep uses only one config file(Session config in Master side and Template config on Slave side) so we should use readfp to avoid skipping OS errors. config.read is retained in case of `--config-set-rx` where glusterd creates new template config file. BUG: 1349273 Change-Id: I15a14d3743facd7b8c7af0edc70fdefaa43efd04 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14777 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* geo-rep: fix wrong argument to find_librarySaravanakumar Arumugam2016-07-071-1/+1
| | | | | | | | | | | | | | | | Argument passed to find_library function is wrong. Pass proper argument to find_library. Note: It still works as default handle is libc. Change-Id: I4657a3ef9f5249e3eec4a20582856b58168126ce BUG: 1352423 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/14852 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* georep: add reset-sync-time option for session deleteMilind Changire2016-06-283-3/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | Set the stime xattr at all the brick roots to (0,0) if the argument reset-sync-time has been provided on the command-line. To avoid testing against directory specific stime, the remote stime is assumed to be minus_infinity, if the root directory stime is set to (0,0), before the directory scan begins. This triggers a full volume resync to slave in the case of a geo-rep session recreation with the same master-slave volume pair. Command synopsis: gluster volume geo-replication <MASTERVOL> <SLAVE>::<SLAVEVOL> delete \ [reset-sync-time] Update gluster cli man page to include new sub-command reset-sync-time. Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948 BUG: 1311926 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/14051 Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Safely handle if unliked GFID not present in data listAravinda VK2016-06-191-1/+2
| | | | | | | | | | | | | | | If unlinked GFID is not present in data list to be synced then Geo-rep worker was crashing with KeyError. Handled KeyError with this patch. BUG: 1345744 Change-Id: I5a1c9ca4473e32606df2e5c7e26c95faf55d44c0 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14706 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Error message cleanupAravinda VK2016-06-151-10/+13
| | | | | | | | | | | | | | | | If ssh returns 127 that means the remote gsyncd path is wrong or push-pem failed during create. Existing error message was pointing old documentation. Change-Id: Ifbbb4a604fc0ae0fd5cb2746df6363bf28cde1e9 BUG: 1343943 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14673 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Handle stime/xtime set failuresAravinda VK2016-06-131-6/+10
| | | | | | | | | | | | | | | | | | While setting stime/xtime, if the file or directory is already deleted then Geo-rep will crash with ENOENT. With this patch, Geo-rep will ignores ENOENT since stime/xtime can't be applied on a deleted file/directory. Change-Id: I2d90569e51565f81ae53fcb23323e4f47c9e9672 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1339471 Reviewed-on: http://review.gluster.org/14529 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* glusterd/geo-rep: upgrade path when slave vol uuid involvedSaravanakumar Arumugam2016-06-031-0/+3
| | | | | | | | | | | | | | | | | | slave volume uuid is involved in identifying a geo-replication session. This patch addresses upgrade path, where existing geo-rep session is gracefully upgraded to involve slave volume uuid. Change-Id: Ib7ff5109b161592f24fc86fc7e93a407655fab86 BUG: 1337473 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14425 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix volume stop with geo-rep sessionAravinda VK2016-06-031-0/+12
| | | | | | | | | | | | | | | | | | | The slave volume uuid is appended to slave_url which is saved in 'gsync_slaves' dictionary with the commit "http://review.gluster.org/13111". But the gsyncd expects slave_url with out slave volume uuid otherwise it fails which in turn fails volume stop with geo-rep. This patch fixes the same. Change-Id: I036f4fce43115ff848daebfe1b2c58765cffab5a BUG: 1342420 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14636 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: update peers section in gsyncd confSaravanakumar Arumugam2016-05-311-3/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Once Slave volume uuid is involved as part of a geo-rep session, it is possible to create the same geo-rep session with different (slave)host. But, it reflects default values for geo-rep configuration values originally configured for old geo-rep session. Reason is, slave host is used while saving config options in gsyncd.conf. With new slave host, it is not possible to retrieve those config values. Solution: Remove slave host related information from gsyncd.conf and have only master volume and slave volume as part of peers section. Also, during upgrade from old geo-rep session, update peers section to reflect only master volume and slave volume. Change-Id: I7debf35a09a28d030b706b0c3e5d82c9b0467d0e BUG: 1340853 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/14558 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Handle Worker kill gracefully if worker already diedAravinda VK2016-05-301-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If Agent dies for any reason, monitor tries to kill Worker also. But if worker is also died then kill command raises error ESRCH: No such process. [2016-05-23 16:49:33.903965] I [monitor(monitor):326:monitor] Monitor: Changelog Agent died, Aborting Worker(/bricks/brick0/master_brick0) [2016-05-23 16:49:33.904535] E [syncdutils(monitor):276:log_raise_exception] <top>: FAIL: Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 306 in twrap tf(*aa) File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 393, in wmon slave_host, master) File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 327, in monitor os.kill(cpid, signal.SIGKILL) OSError: [Errno 3] No such process With this patch, monitor will gracefully handle if worker is already died. Change-Id: I3ae5f816a3a197343b64540cf46f5453167fb660 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1339472 Reviewed-on: http://review.gluster.org/14512 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd/geo-rep: slave volume uuid to identify a geo-rep sessionSaravanakumar Arumugam2016-05-122-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Currently, it is possible to create multiple geo-rep session from the Master host to Slave host(s), where Slave host(s) belonging to the same volume. For example: Consider Master Host M1 having volume tv1 and Slave volume tv2, which spans across two Slave hosts S1 and S2. Currently, it is possible to create geo-rep session from M1(tv1) to S1(tv2) as well as from M1(tv1) to S2(tv2). When the Slave Host is alone modified, it is identified as a new geo-rep session (as slave host and slave volume together are identifying Slave side). Also, it is possible to create both root and non-root geo-rep session between same Master volume and Slave volume. This should also be avoided. Solution: This multiple geo-rep session creation must be avoided and in order to avoid, use Slave volume uuid to identify a Slave. This way, we can identify whether a session is already created for the same Slave volume and avoid creating again (using different host). When the session creation is forced in the above scenario, rename the existing geo-rep session directory with new Slave Host mentioned. Change-Id: I9239759cbc0d15dad63c48b8cf62950bb687c7c8 BUG: 1294813 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13111 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Fix Entry Creation issue with non root UID/GIDAravinda VK2016-03-091-0/+20
| | | | | | | | | | | | | | | | | | | | | | | During entry_ops RENAME Geo-rep sends stat info along with the recorded info from Changelog. In Slave side if Source file exists Geo-rep renames to Target file by calling os.rename. If source file does not exists, it tries to create Target file directly using available stat info from Master. If UID and GID are different in Master for that file then stat info will have different UID/GID during Create. Geo-rep gets EACCES when it tries to create a new entry using gfid-access with different UID/GID. With this patch, Entry creation with different UID/GID is split into two operations. Create Entry with UID:0 and GID:0 and then set UID/GID. Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01 BUG: 1313303 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13542 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: hard-link rename issues on changelog replayMilind Changire2016-03-091-4/+24
| | | | | | | | | | | | | | | | | | | | | Problem: LINK + RENAME changelog when replayed after worker restart causes stale hard-links to persist since VFS returns success for RENAME if hard-links point to same inode. Solution: Worker detects RENAME being issued on hard-links to the same inode and unlinks the source file-name. Conditionally rename by verifying that the source gfid matches with the on-disk gfid on the slave. Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2 BUG: 1296174 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/13189 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix CPU utilization 100% with PopenAravinda VK2016-03-081-0/+4
| | | | | | | | | | | | | | | | | | | Errorstore is maintained by Geo-rep to collect errors from the child processes opened using Popen. If Popen.communicate is used then it closes stderr. When stderr is not available errorstore.tailer() will enter into infinite loop without gap. With this patch, sleep time added when stderr of Child process is already closed. Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94 BUG: 1315601 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13637 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* georep: avoid creating multiple entries with same gfidMilind Changire2016-02-291-4/+12
| | | | | | | | | | | | | | | | | | | | | | Problem: CREATE + RENAME changelogs replayed by geo-replication cause stale old-name entries with same gfid on slave nodes. A gfid is a unique key in the file-system and should not be assigned to multiple entries. Solution: Create entry on slave only if lstat(gfid) at aux-mount fails. This applies to files as well as directories. Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919 BUG: 1296206 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/13316 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Handling Rsync/Tar errors efficientlyAravinda VK2016-02-262-87/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | Geo-rep processes Changelogs in Batch, if one file in batch fails with rsync error that Changelog file is reprocessed multiple times. After MAX_RETRY, it logs all the GFIDs from that batch as Skipped. This patch addresses following issues, 1. When Rsync/Tar fails do not parse Changelog again for retry 2. When Rsync/Tar fails do not replay Entry operations, only retry rsync/tar for those GFIDs 3. Log Error in Rsync/Tar only in the last Retry 4. Do not log Skipped GFIDs since Rsync/Tar errors are logged for only failed files. 5. Changed Entry failures as Error instead of Warning BUG: 1287723 Change-Id: Ie134ce2572693056ab9b9008cd8aa5b5d87f7975 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12856 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Handle ERANGE error during listxattrAravinda VK2016-02-261-7/+21
| | | | | | | | | | | | | | | | | | | | | | | | llistxattr in Geo-rep is two syscall instead of one SIZE = llistxattr(PATH, &BUF, 0); BUF = create_buf(SIZE); _ = llistxattr(PATH, &BUF, SIZE); So if any new xattrs added just after first call by any other worker, second syscall will fail with ERANGE error. Now Geo-rep sends BUF with large size(256*100) and gets value with only one syscall. Raises OSError if fails with ERANGE error even after sending large BUF. Change-Id: I8ade4bbe9a0a8ea908ed9dedcd3f2ff4c6fe6f51 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1294588 Reviewed-on: http://review.gluster.org/13106 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Handle hardlink in Tiering based volumeSaravanakumar Arumugam2016-02-261-10/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Hardlinks are synced as Sticky bit files to Slave in a Tiering based volume. In a Tiering based volume, cold tier is hashed subvolume and geo-rep captures all namespace operations in cold tier. While syncing a file and its corresponding hardlink, it is recorded as MKNOD in cold tier(for both) and We end up creating two different files in Slave. Solution: If MKNOD with Sticky bit set is present, record it as LINK. This way it will create a HARDLINK if source file exists (on slave), else it will create a new file. This way, Slave can create Hardlink file itself (instead of creating a new file) in case of hardlink. Change-Id: Ic50dc6e64df9ed01799c30539a33daace0abe6d4 BUG: 1301032 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13281 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* geo-rep: Fix getting subvol countKotresh HR2015-12-221-1/+4
| | | | | | | | | | | | | | | | Tiering doesn't support disperse volume as hot tier, hence xml output doesn't give 'hotdisperseCount'. Remove the usage of 'hotdisperseCount' in geo-rep and return 0 instead. Change-Id: I736e29257de085a25e38eb02959caad3465ebcda BUG: 1292084 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13062 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vivek Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix getting subvol numberKotresh HR2015-12-212-17/+47
| | | | | | | | | | | | | | | | | | Fix getting subvol number if the volume type is tier. If the volume type was tier, the subvol number was calculated incorrectly and hence few of workers didn't become ACTIVE resulting in files not being replicated from corresponding brick. This patch addresses the same. Change-Id: Ic10ad7f09a0fa91b4bf2aa361dea3bd48be74853 BUG: 1292084 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12994 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Symlink Rename issueAravinda VK2015-12-172-3/+15
| | | | | | | | | | | | | | | | | | | | | If ENTRY creation failed for symlink in Slave and symlink renamed in Master. If Source not exists to Rename in Slave Geo-rep interprets as Create of Target file. Geo-rep sends blob of regular file to create symlink instead of sending blob of symlink. With this patch, Geo-rep identifies symlink and sends respective blob. BUG: 1289859 Change-Id: If9351974d1945141a1d3abb838b7d0de7591e48e Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12917 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Tested-by: Milind Changire <mchangir@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* Add support for sparse files to tarssh methodAlex Markelov2015-12-111-1/+1
| | | | | | | | | | | | | | | | Without '--sparse' option tar will not properly archive sparse file and geo-replication will result in non-sparse file on the remote end. Here is more on how I arrived at this http://markelov.org/wiki/index.php/GlusterFS_3.6.1_on_CentOS_6.5:_geo-replication_and_sparse_files_problem Change-Id: I8d671964a1b48bbb916e4a064571221bf3631494 BUG: 1276839 Signed-off-by: Alex Markelov <alex@markelov.org> Reviewed-on: http://review.gluster.org/12476 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: use cold tier bricks for namespace operationsSaravanakumar Arumugam2015-12-034-17/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: symlinks are not getting synced to slave in a Tiering based volume. Solution: Now, symlinks are created directly in cold tier bricks( in the backend). Earlier, cold tier was avoided for namespace operations and only hot tier was used while processing changelogs. Now, cold tier is HASH subvolume in a Tiering volume. So, carry out namespace operation only in cold tier subvolume and avoid hot tier subvolume to avoid any races. Earlier, XSYNC was used(and changeloghistory avoided) during initial sync in order to avoid race while processing historychangelog in Hot tier. This is no longer required as there is no race from Hot tier. Also, avoid both live and history changelog ENTRY operations from Hot tier to avoid any race with cold tier. Change-Id: Ia8fbb7ae037f5b6cb683f36c0df5c3fc2894636e BUG: 1287519 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12844 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: geo-rep to handle CAPS based HostnameSaravanakumar Arumugam2015-12-021-1/+1
| | | | | | | | | | | | | | | | | | | | | Problem: geo-replication session creation fails when Hostname is having CAPS in it. Issue is with the regex pattern which handles only small lettered Hostname. Fix: Fix the regex pattern to handle CAPS based hostname as well. Change-Id: I5c99c102e9706acc2b1fab1e6bf158e68beed373 BUG: 1265522 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/12216 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: fd close and fcntl issueAravinda VK2015-12-011-15/+19
| | | | | | | | | | | | | | | | | | | | When any of the open fd of a file is closed on which fcntl lock is taken even though another fd of the same file is open on which lock is taken, all fcntl locks will be released. This causes both replica workers to be ACTIVE sometimes. This patche fixes that issue. Change-Id: I1e203ab0e29442275338276deb56d09e5679329c BUG: 1285488 Original-Author: Aravinda VK <avishwan@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12752 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Milind Changire <mchangir@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Fix syntax errors in GsyncdErrorAravinda VK2015-11-231-3/+2
| | | | | | | | | | | | %s was not replaced by actual values in GsyncdError BUG: 1279921 Change-Id: I3c0a10f07383ca72844a46f930b4aa3d3c29f568 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12566 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix Geo-rep logging to log datetime in GMT/UTCAravinda VK2015-11-231-0/+1
| | | | | | | | | | | | | Geo-rep is logging in Local time, all other Gluster logs are in GMT/UTC. It is very difficult to co-relate Geo-rep logs with other Gluster logs. BUG: 1282331 Change-Id: Ieae8bda7e4788e587cf4595e21e0e772c210cfbb Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12583 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Fix FD leak from Active Geo-rep workerAravinda VK2015-11-182-0/+16
| | | | | | | | | | | | | | | | | | Active worker tries to acquire lock in each iteration. On every successfull lock acqusition it was not closing previously opened lock fd. To see the leak, get the PID of worker, ps -ax | grep feedback-fd watch ls /proc/$pid/fd BUG: 1225566 Change-Id: Ic476c24c306e7ab372c5560fbb80ef39f4fb31af Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/12332 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: New Config option for ssh_portAravinda VK2015-11-172-3/+8
| | | | | | | | | | | | | | | | | | | | | | | If different port used for SSH instead of 22, Geo-replication was failing to establish SSH connection. ssh_port option can be added using config:ssh_command and config:ssh_command_tar, but user has to remember complete ssh command used with parameter to add/modify ssh port. This patch adds new config option for ssh_port, gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \ config ssh_port 52022 Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1276028 Reviewed-on: http://review.gluster.org/12444 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: Kill Geo-rep Worker when Agent process diesAravinda VK2015-11-093-14/+44
| | | | | | | | | | | | | | | | | | When Changelog agent process dies, Geo-replication fails to detect and worker will run without respective Changelog agent. Status shows Active/Passive without any progress. With this patch, Worker process gets killed whenever Changelog agent dies. Change-Id: I30b4cc77f924f7e8174b8bfe415ac17f0b3851b4 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1277076 Reviewed-on: http://review.gluster.org/12485 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix syncing chown in xsync crawlKotresh HR2015-11-081-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE Problem: The sequence of entry creation and chown in master is recorded as creation of entry with resulted user:group in xsync changelog. During sync, entry creation is always split into two ops, MKNOD and SETATTR. Hence the issue is not being hit otherwise it would have failed with EPERM if parent is owned by different user. But with shard translator being enabled on slave, doing entry creation with MKNOD and SETATTR is not allowed, SETATTR fails as it accesses inode structure which is not linked. Solution: The sequence of entry creation and chown in master should be recorded as MKNOD and SETATTR separately always and do entry creation with single op in gfid-access xlator. The gfid-access patch will be sent separately. Change-Id: I93e554bf9342397a7660503f5128e9709f8a0cd8 BUG: 1265148 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12205 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Update last_synced_time in XSyncAravinda VK2015-11-041-14/+9
| | | | | | | | | | | | | During XSync crawl, last_synced time in status file was not updated. This patch fixes the issue by updating status file when stime xattr is updated after Xsync or Changelog Crawl. Change-Id: I4dc3a2d4c3d8378a939da0868caf1aef4f789599 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1247536 Reviewed-on: http://review.gluster.org/11771 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* geo-rep: Handle FXATTROP and XATTROPKotresh HR2015-10-291-1/+2
| | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE If it is FXATTROP or XATTROP in changelog, add the gfid to rsync queue. Change-Id: If68d38d7ed00b70a4618cfcc8e75df3fbadbf724 BUG: 1265148 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12226 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
generated by cgit (git 2.34.1) at 2024-06-11 09:53:21 +0000