summaryrefslogtreecommitdiffstats
path: root/geo-replication
Commit message (Collapse)AuthorAgeFilesLines
* geo-rep: Fix syncing of self healed hardlinksKotresh HR2017-08-111-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: In a distribute replicate volume, if the hardlinks are created when a subvolume is down, it gets healed from other subvolume when it comes up. If this subvolume becomes ACTIVE in geo-rep there are chances that those hardlinks won't be synced to slave. Cause: AFR can't detect hardlinks during self heal. It just create those files using mknod and the same is recorded in changelog. Geo-rep processes these mknod and ignores it as it finds gfid already on slave. Solution: Geo-rep should process the mknod as link if the gfid already exists on slave. > Change-Id: I2f721b462b38a74c60e1df261662db4b99b32057 > BUG: 1475308 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/17880 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit d685e4238fafba8f58bf01174c79cb5ca35203e5) Change-Id: I2f721b462b38a74c60e1df261662db4b99b32057 BUG: 1476212 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17907 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix entry failure because parent dir doesn't existKotresh HR2017-07-071-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In a distributed volume on master, it can so happen that the RMDIR followed by MKDIR is recorded in changelog on a particular subvolume with same gfid and pargfid/bname but not on all subvolumes as below. E 61c67a2e-07f2-45a9-95cf-d8f16a5e9c36 RMDIR \ 9cc51be8-91c3-4ef4-8ae3-17596fcfed40%2Ffedora2 E 61c67a2e-07f2-45a9-95cf-d8f16a5e9c36 MKDIR 16877 0 0 \ 9cc51be8-91c3-4ef4-8ae3-17596fcfed40%2Ffedora2 While processing this changelog, geo-rep thinks RMDIR is successful and does recursive rmdir on slave. But in the master the directory still exists. This could lead to data discrepancy between master and slave. Cause: RMDIR-MKDIR pair gets recorded so in changelog when the directory removal is successful on cached subvolume and failed in one of hashed subvol for some reason (may be down). In this case, the directory is re-created on cached subvol which gets recorded as MKDIR again in changelog. Solution: So while processing RMDIR geo-replication should stat on master with gfid and should not delete it if it's present. > Change-Id: If5da1d6462eb4d9ebe2e88b3a70cc454411a133e > BUG: 1467718 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/17695 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit b25bf64f3a3520a96ad557daa4903c0ceba96d72) Change-Id: If5da1d6462eb4d9ebe2e88b3a70cc454411a133e BUG: 1468198 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17714 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
* geo-rep: Retry on EBUSYKotresh HR2017-04-272-10/+11
| | | | | | | | | | | | | | | | | | | | | | | Do not crash on EBUSY error. Add EBUSY retry errno list. Crash only if the error persists even after max retries. > BUG: 1434018 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/16924 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Aravinda VK <avishwan@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Change-Id: Ia067ccc6547731f28f2a315d400705e616cbf662 BUG: 1441927 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17049 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix EBUSY tracebackKotresh HR2017-04-272-2/+2
| | | | | | | | | | | | | | | | | | | | | | EBUSY was added to retry list of errno_wrap without importing. Fixing the same. > BUG: 1434018 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/17011 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Aravinda VK <avishwan@redhat.com> Change-Id: Ide81a9ccc9b948a96265b6890da078b722b45d51 BUG: 1441927 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17050 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* build/packaging: Debian and Ubuntu don't have /usr/libexecKaleb S. KEITHLEY2017-03-132-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs in configure(.ac) Debian-based distributions don't have a /usr/libexec/ directory This issues is partially mitigated by the use of $libexecdir in some of the Makefile.am files, but even so the incorrectly defined GLUSTERFS_LIBEXECDIR results in various things such as gsyncd, glusterfind, eventsd, etc., trying to invoke other scripts and programs from a location that doesn't exist. And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as well use it appropriatedly. master change https://review.gluster.org/16880 master BZ: 1430841 Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d BUG: 1430844 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/16881 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* geo-rep: Fix xsync crawlKotresh HR2017-02-161-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If stime is set to (0, 0) on master brick root, it is expected to do complete sync ignoring the stime set on sub directories. But while initializing the stime variable for comparison, it was initailized to (-1, 0) instead of (0, 0). Fixed the same. The stime is set to (0, 0) with the 'reset-sync-time' option while deleting session. 'gluster vol geo-rep master fedora1::slave delete reset-sync-time' The scenario happens when geo-rep session is deleted as above and for some reason the session is re-established with same slave volume after deleting data on slave volume. > Change-Id: Ie5bc8f008dead637a09495adeef5577e2b33bc90 > BUG: 1422760 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/16629 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Aravinda VK <avishwan@redhat.com> Change-Id: Ie5bc8f008dead637a09495adeef5577e2b33bc90 BUG: 1422819 Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 267578ec0d6b29483a1bd402165ea8c388ad825e) Reviewed-on: https://review.gluster.org/16644 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* geo-rep: Separate slave mount logs for each connectionKotresh HR2017-01-183-1/+21
| | | | | | | | | | | | | | | | | | | | | Geo-rep worker mounts the slave volume on the slave node. If multiple worker connects to same slave node, all workers share the same mount log file. This is very difficult to debug as logs are cluttered from different mounts. Hence creating separate mount log file for each connection from worker. Each connection from worker is identified uniquely using 'mastervol uuid', 'master host', 'master brickpath', 'salve vol'. The log file name will be combination of the above. Change-Id: I67871dc8e8ea5864e2ad55e2a82063be0138bf0c BUG: 1412689 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/16384 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Handle directory sync failure as hard errorKotresh HR2017-01-132-6/+20
| | | | | | | | | | | | | | | | | | If directory creation is failed, return immediately before further processing. Allowing it to further process will fail the entire directory tree syncing to slave. Hence master will log and raise exception if it's directory failure. Earlier, master used to log the failure and proceed. Change-Id: Iba2a8b5d3d0092e7a9c8a3c2cdf9e6e29c73ddf0 BUG: 1411607 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/16364 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix log-rsync-performance config issueAravinda VK2016-12-142-12/+14
| | | | | | | | | | | | | | | | If log-rsync-performance config is not set, gconf.get_realtime will return None, Added default value as False if config file doesn't have this option set. BUG: 1393678 Change-Id: I89016ab480a16179db59913d635d8553beb7e14f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/16102 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Use Host UUID to find local Gluster nodeAravinda VK2016-12-133-43/+22
| | | | | | | | | | | | | | | | | To spawn workers for each local brick, Geo-rep was collecting all the machine IPs based on hostname and finds based on the connectivity. With this patch, Geo-rep finds local brick if host UUID matches with UUID of the brick from Volume info. BUG: 1401801 Change-Id: Ic83c65df89e43cb86346e3ede227aa84d17ffd79 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/16035 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Do not restart workers when log-rsync-performance config changeAravinda VK2016-12-071-2/+4
| | | | | | | | | | | | | | | | | | Geo-rep restarts workers when any of the configurations changed. We don't need to restart workers if tunables like log-rsync-performance is modified. With this patch, Geo-rep workers will get new "log-rsync-performance" config automatically without restart. BUG: 1393678 Change-Id: I40ec253892ea7e70c727fa5d3c540a11e891897b Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15816 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep/eventsapi: Add Master node information in Geo-rep EventsAravinda VK2016-12-024-23/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added Master node information to GEOREP_ACTIVE, GEOREP_PASSIVE, GEOREP_FAULTY and GEOREP_CHECKPOINT_COMPLETED events. EVENT_GEOREP_ACTIVE(master_node and master_node_id are new fields) { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_ACTIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "master_node": MASTER_NODE, "master_node_id": MASTER_NODE_ID, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_PASSIVE(master_node and master_node_id are new fields) { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_PASSIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "master_node": MASTER_NODE, "master_node_id": MASTER_NODE_ID, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_FAULTY(master_node and master_node_id are new fields) { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_FAULTY", "message": { "master_volume": MASTER_VOLUME_NAME, "master_node": MASTER_NODE, "master_node_id": MASTER_NODE_ID, "current_slave_host": CURRENT_SLAVE_HOST, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_CHECKPOINT_COMPLETED(master_node and master_node_id are new fields) { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_CHECKPOINT_COMPLETED", "message": { "master_volume": MASTER_VOLUME_NAME, "master_node": MASTER_NODE, "master_node_id": MASTER_NODE_ID, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH, "checkpoint_time": CHECKPOINT_TIME, "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME } } BUG: 1395660 Change-Id: Ic91af52fa248c8e982e93a06be861dfd69689f34 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15858 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix Last synced status column issue during Hybrid CrawlAravinda VK2016-11-281-7/+9
| | | | | | | | | | | | | | | | | | | During Hybrid crawl, Geo-rep maintains stime xattr in subdirectories along with the Brick root. This is done to skip directories if Geo-rep crashes before Hybrid crawl completes. Update Last synced status only when stime xattr updated in brick root. Status output will mislead if it shows sub directory stime as last synced time. BUG: 1396081 Change-Id: I5b73aee7ae4a1c1e2d1001d1f55559b9f9efd6e6 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15869 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Handle ENOENT during unlinkAravinda VK2016-11-221-2/+4
| | | | | | | | | | | | | | Do not raise traceback if a file/dir not exists during unlink or rmdir BUG: 1396062 Change-Id: Idd43ca1fa6ae6056c3cd493f0e2f151880a3968c Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15868 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep: Upgrade conf file only if it is session configAravinda VK2016-10-241-1/+4
| | | | | | | | | | | | | | Ignore config upgrade if it is template config file present in /var/lib/glusterd/geo-replication/gsyncd_template.conf BUG: 1386123 Change-Id: I2cbba3103b6801c16ff57f778a90b9a0bb2467cf Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15669 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* geo-rep: Logging improvementsAravinda VK2016-10-244-29/+55
| | | | | | | | | | | | | | | | | | | - Redundant log messages removed. - Worker and connected slave node details added in "starting worker" log - Added log for Monitor state change - Added log for Worker status change(Initializing/Active/Passive/Faulty) - Added log for Crawl status Change - Added log for config set and reset - Added log for checkpoint set, reset and completion BUG: 1359612 Change-Id: Icc7173ff3c93de4b862bdb1a61760db7eaf14271 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15684 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep/eventsapi: Additional EventsAravinda VK2016-10-185-8/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added following events EVENT_GEOREP_ACTIVE { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_ACTIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_PASSIVE { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_PASSIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH } } EVENT_GEOREP_CHECKPOINT_COMPLETED { "nodeid": NODEID, "ts": TIMESTAMP, "event": "GEOREP_ACTIVE", "message": { "master_volume": MASTER_VOLUME_NAME, "slave_host": SLAVE_HOST, "slave_volume": SLAVE_VOLUME, "brick_path": BRICK_PATH, "checkpoint_time": CHECKPOINT_TIME, "checkpoint_completion_time": CHECKPOINT_COMPLETION_TIME } } BUG: 1379330 Change-Id: I90716175868c59dd65c8d202e73e0ede90347b6a Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15630 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Kotresh HR <khiremat@redhat.com>
* eventsapi/geo-rep: Geo-rep will not work without eventsapi rpmsAravinda VK2016-09-282-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | If glusterfs-events rpm is not installed, Geo-replication will fail since it imports eventtypes. Any call to gsyncd will fail with Import error. Glusterd start fails since it runs `gsyncd.py --version` Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 29, in <module> from syncdutils import FreeObject, norm, grabpidfile, finalize File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 28, in <module> from events import eventtypes ImportError: No module named events BUG: 1378057 Change-Id: I1a9bc086c3d52449ec7296cb2f9ceb16cd41a8a4 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15539 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* build: out-of-tree builds generates files in the wrong directoryKaleb S KEITHLEY2016-09-181-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | And minor cleanup of a few of the Makefile.am files while we're at it. Rewrite the make rules to do what xdrgen does. Now we can get rid of xdrgen. Note 1. netbsd6's sed doesn't do -i. Why are we still running smoke tests on netbsd6 and not netbsd7? We barely support netbsd7 as it is. Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with libglusterfs? A cut-and-paste mistake? It has no references to symbols in libglusterfs. Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_ regex that matches the same lines as the _extended_ regex "/#(ifndef|define|endif)/". To match the extended regex sed needs to be run with -r on Linux; with -E on *BSD. However NetBSD's and FreeBSD's sed helpfully also provide -r for compatibility. Using a basic regex avoids having to use a kludge in order to run sed with the correct option on OS X. Note 4. Not copying the bit of xdrgen that inserts copyright/license boilerplate. AFAIK it's silly to pretend that machine generated files like these can be copyrighted or need license boilerplate. The XDR source files have their own copyright and license; and their copyrights are bound to be more up to date than old boilerplate inserted by a script. From what I've seen of other Open Source projects -- e.g. gcc and its C parser files generated by yacc and lex -- IIRC they don't bother to add copyright/license boilerplate to their generated files. It appears that it's a long-standing feature of make (SysV, BSD, gnu) for out-of-tree builds to helpfully pretend that the source files it can find in the VPATH "exist" as if they are in the $cwd. rpcgen doesn't work well in this situation and generates files with "bad" #include directives. E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`, you get an #include directive in the generated .c file like this: ... #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h" ... which (obviously) results in compile errors on out-of-tree build because the (generated) header file doesn't exist at that location. Compared to `rpcgen ./glusterfs3-xdr.x` where you get: ... #include "glusterfs3-xdr.h" ... Which is what we need. We have to resort to some Stupid Make Tricks like the addition of various .PHONY targets to work around the VPATH "help". Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/... looks exactly like -I$(top_srcdir)/rpc/xdr/... Don't be fooled though. And don't delete the -I$(top_builddir)/rpc/xdr/... bits Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e BUG: 1330604 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14085 Tested-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* geo-rep: Defunct tar process after syncAravinda VK2016-09-131-0/+6
| | | | | | | | | | | | | | | | | After every sync iteration with tarssh mode leaves defunct tar process. Added wait for tar process to prevent this issue. BUG: 1374286 Change-Id: I9953239ef601cc1970c814b00074b45eb00f481e Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15426 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Use configured log_level for libgfchangelog logsAravinda VK2016-09-094-2/+21
| | | | | | | | | | | | | | | | libgfchangelog was not respecting the log_level configured in Geo-replication. With this patch Libgfchangelog log level can be configured using `config changelog_log_level TRACE`. Default Changelog log level is INFO BUG: 1363965 Change-Id: Ida714931129f6a1331b9d0815da77efcb2b898e3 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15078 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Handle EISDIR error during UnlinkAravinda VK2016-09-081-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | During Rename, If Source and Target has same inode then Geo-rep unlinks source. But if source is a directory then this will fail with below traceback Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 113, in worker res = getattr(self.obj, rmeth)(*in_data[2:]) File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 772, in entry_ops os.unlink(entry) OSError: [Errno 21] Is a directory: '.gfid/12711ebf-7fdc-4f4b-9850-2d75581eb 452/New folder' With this patch, if EISDIR, rmdir is tried. Logs error in Slave log in case of ENOTEMPTY. BUG: 1365791 Change-Id: I099af4192adac5125c0a23988ceb6506f91e987f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15132 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix ESTALE/EINVAL issue during set_{xtime,stime}Aravinda VK2016-09-081-3/+6
| | | | | | | | | | | | | | | | | | Setfattr may get ESTALE/EINVAL if a file is being unlinked. To prevent worker crashing, added retry for these error messages. On second retry it will get ENOENT and that error is handled by ignoring. BUG: 1373373 Change-Id: Ic660fa13208366d57c8d3d492bbef611475e45b7 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15404 Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: Fix Geo-rep status if monitor.pid file not existsAravinda VK2016-09-081-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If monitor.pid file not exists, gsyncd fails with following traceback Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 201, in main main_i() File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 681, in main_i brick_status.print_status(checkpoint_time=checkpoint_time) File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 343, in print_status for key, value in self.get_status(checkpoint_time).items(): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 262, in get_status with open(self.monitor_pid_file, "r+") as f: IOError: [Errno 2] No such file or directory: '/var/lib/glusterd/ geo-replication/master_node_slave/monitor.pid' If Georep status command this worker's status will not be displayed since not returning expected status output. BUG: 1373741 Change-Id: I600a2f5d9617f993d635b9bc6e393108500db5f9 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15416 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix logging sync failuresAravinda VK2016-09-081-1/+8
| | | | | | | | | | | | | | | If Rsync/Tar subprocess dies, while logging error Geo-rep fails with EBADF while accessing error file. Also worker dies while accessing elines before it is set. BUG: 1372193 Change-Id: I9cfce116e8aafa4a98654f5190d40a455af8ec95 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15379 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix History post processAravinda VK2016-09-075-320/+123
| | | | | | | | | | | | | | | | This patch removes changelogsdb part of post processing since not got much performance advantage as expected. Entry stime and other logging improvements retained. BUG: 1364420 Change-Id: Ib99d23f09d96c14bc28225b47d9134260f5551bf Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15371 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: add geo-rep events for server side changesSaravanakumar Arumugam2016-08-315-25/+58
| | | | | | | | | | | | | | | | | Event Type defined in #15351 to avoid merge conflicts Add geo-rep events applicable to changes in geo-rep session in the server side. Change-Id: Ia66574d2abccad7fce6a96667efbc7c6c8903fc6 BUG: 1370445 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/15328 Tested-by: Aravinda VK <avishwan@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Alternate command to generate SSH KeysAravinda VK2016-08-312-2/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `gluster system:: execute gsec_create` is used to generate SSH Keys in all Master nodes and collect public keys in command initiated node. But this tool will not provide details if a peer node is down and unable to generate keys. New command will be introduced to create SSH Keys in all peer nodes. Usage: gluster-georep-sshkey generate or gluster-georep-sshkey generate --no-prefix Generates two SSH keys(one for gsyncd access and other for tar) in all peer nodes and collects the public keys to the local node where it is initiated. Adds `command=` prefix to common_secret.pem.pub if `--no-prefix` argument is not set. Shows status as below, +-----------+-------------+---------------+ | NODE | NODE STATUS | KEYGEN STATUS | +-----------+-------------+---------------+ | fvm2 | UP | OK | | localhost | UP | OK | +-----------+-------------+---------------+ BUG: 1356508 Change-Id: Ib202811f41f9986694f07d9eedba31db6ed4d18f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14732 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Simplify Non root user(mountbroker) SetupAravinda VK2016-08-302-2/+399
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Setting up Non root Geo-replication is troublesome. Lot of steps involved like user setup, directory creation, setting up proper permissions, editing glusterd.vol file etc. Introducing `gluster-mountbroker` command, with this tool non root user setup steps are(Run the following commands in any one Slave node), gluster-mountbroker setup <MOUNT ROOT> <GROUP> For example, gluster-mountbroker setup /var/mountbroker-root geogroup Add user using, gluster-mountbroker add <VOLUME> <USER> For example, gluster-mountbroker add slavevol geoaccount Remove user or Volume using, gluster-mountbroker remove [--volume <VOLUME>] [--user <USER>] Example, gluster-mountbroker remove --volume slavevol --user geoaccount gluster-mountbroker remove --user geoaccount gluster-mountbroker remove --volume slavevol Check the status of setup using, gluster-mountbroker status Once the changes are complete, restart glusterd in all Slave nodes. and follow the Geo-rep instructions. Change-Id: Ied3fa009df6a886b24ddd86d39283fcfcff68c82 BUG: 1343333 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14544 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Post process Data and Meta ChangelogsAravinda VK2016-08-267-186/+516
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this patch, Data and Meta GFIDs are post processed. If Changelog has UNLINK entry then remove from Data and Meta GFIDs list(If stat on GFID is ENOENT in Master). While processing Changelogs, - Collect all the data and meta operations in a temporary database - Delete all Data and Meta GFIDs which are already unlinked as per Changelogs (unlink only if stat on GFID is ENOENT) - Process all Entry operations as usual - Process data and meta operations in batch(Fetch from Db in batch) - Data sync is again batched based on number of changelogs(Default 1day changelogs). Once the sync is complete, Update last Changelog's time as last_synced time as usual. Additionally maintain entry_stime on Brick root, ignore Entry ops if changelog suffix time is less than entry_stime. If data stime is more than entry_stime, this can happen only when passive worker updates stime by itself by getting mount point stime. Use entry_stime = data_stime in this case. New configurations: max-rsync-retries - Default Value is 10 max-data-changelogs-in-batch - Max number of changelogs to be considered in a batch for syncing. Default value is 5760(4 changelogs per min * 60 min * 24 hours) max-history-changelogs-in-batch - Max number of history changelogs to be processed at once. Default value 86400(4 changelogs per min * 60 min * 24 hours * 15 days) BUG: 1364420 Change-Id: I7b665895bf4806035c2a8573d361257cbadbea17 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/15110 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* features/libgfchangelog: Log failure in gf_histroy_changelogKotresh HR2016-08-093-2/+10
| | | | | | | | | | | | | | | | | Add error logs if gf_history_changelog fails. If requested changelog range is not available, log the error and exit instead of continuing the loop and exiting in readdir without logging. Also fixed the duplicate MSGID number in 'changelog-lib-messages.h' Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd BUG: 1362151 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15064 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: address potential leak of memoryPrasanna Kumar Kalever2016-07-191-1/+9
| | | | | | | | | | | | | | | | | fix memory leak reported by clang analyzer: gsyncd.c:411:17: warning: Potential leak of memory pointed to by 'argv' in main() Change-Id: I179fb1c095cf4d331b5910df6388dc724cb4ac96 BUG: 1332419 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/14151 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* core: use readdir(3) with glibc, and associated cleanupKaleb S. KEITHLEY2016-07-181-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting with glibc-2.23 (i.e. what's in Fedora 25), readdir_r(3) is marked as deprecated. Specifically the function decl in <dirent.h> has the deprecated attribute, and now warnings are thrown during the compile on Fedora 25 builds. The readdir(_r)(3) man page (on Fedora 25 at least) and World+Dog say that glibc's readdir(3) is, and always has been, MT-SAFE as long as only one thread is accessing the directory object returned by opendir(). World+Dog also says there is a potential buffer overflow in readdir_r(). World+Dog suggests that it is preferable to simply use readdir(). There's an implication that eventually readdir_r(3) will be removed from glibc. POSIX has, apparently deprecated it in the standard, or even removed it entirely. Over and above that, our source near the various uses of readdir(_r)(3) has a few unsafe uses of strcpy()+strcat(). (AFAIK nobody has looked at the readdir(3) implemenation in *BSD to see if the same is true on those platforms, and we can't be sure of MacOS even though we know it's based on *BSD.) Change-Id: I5481f18ba1eebe7ee177895eecc9a80a71b60568 BUG: 1356998 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14838 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* geo-rep: Handle Config parser errorsAravinda VK2016-07-122-10/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | Python ConfigParser lib has two methods, readfp and read, it should be used as follows. readfp(open("defaults.conf")) read("custom.conf") ConfigParser.read(path) ignores any file errors, which is intentional since errors are handled while loading default config. Geo-rep uses only one config file(Session config in Master side and Template config on Slave side) so we should use readfp to avoid skipping OS errors. config.read is retained in case of `--config-set-rx` where glusterd creates new template config file. BUG: 1349273 Change-Id: I15a14d3743facd7b8c7af0edc70fdefaa43efd04 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14777 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* geo-rep: fix wrong argument to find_librarySaravanakumar Arumugam2016-07-071-1/+1
| | | | | | | | | | | | | | | | Argument passed to find_library function is wrong. Pass proper argument to find_library. Note: It still works as default handle is libc. Change-Id: I4657a3ef9f5249e3eec4a20582856b58168126ce BUG: 1352423 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/14852 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* georep: add reset-sync-time option for session deleteMilind Changire2016-06-283-3/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | Set the stime xattr at all the brick roots to (0,0) if the argument reset-sync-time has been provided on the command-line. To avoid testing against directory specific stime, the remote stime is assumed to be minus_infinity, if the root directory stime is set to (0,0), before the directory scan begins. This triggers a full volume resync to slave in the case of a geo-rep session recreation with the same master-slave volume pair. Command synopsis: gluster volume geo-replication <MASTERVOL> <SLAVE>::<SLAVEVOL> delete \ [reset-sync-time] Update gluster cli man page to include new sub-command reset-sync-time. Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948 BUG: 1311926 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/14051 Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Safely handle if unliked GFID not present in data listAravinda VK2016-06-191-1/+2
| | | | | | | | | | | | | | | If unlinked GFID is not present in data list to be synced then Geo-rep worker was crashing with KeyError. Handled KeyError with this patch. BUG: 1345744 Change-Id: I5a1c9ca4473e32606df2e5c7e26c95faf55d44c0 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14706 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Error message cleanupAravinda VK2016-06-151-10/+13
| | | | | | | | | | | | | | | | If ssh returns 127 that means the remote gsyncd path is wrong or push-pem failed during create. Existing error message was pointing old documentation. Change-Id: Ifbbb4a604fc0ae0fd5cb2746df6363bf28cde1e9 BUG: 1343943 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14673 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Handle stime/xtime set failuresAravinda VK2016-06-131-6/+10
| | | | | | | | | | | | | | | | | | While setting stime/xtime, if the file or directory is already deleted then Geo-rep will crash with ENOENT. With this patch, Geo-rep will ignores ENOENT since stime/xtime can't be applied on a deleted file/directory. Change-Id: I2d90569e51565f81ae53fcb23323e4f47c9e9672 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1339471 Reviewed-on: http://review.gluster.org/14529 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* glusterd/geo-rep: upgrade path when slave vol uuid involvedSaravanakumar Arumugam2016-06-031-0/+3
| | | | | | | | | | | | | | | | | | slave volume uuid is involved in identifying a geo-replication session. This patch addresses upgrade path, where existing geo-rep session is gracefully upgraded to involve slave volume uuid. Change-Id: Ib7ff5109b161592f24fc86fc7e93a407655fab86 BUG: 1337473 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14425 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix volume stop with geo-rep sessionAravinda VK2016-06-031-0/+12
| | | | | | | | | | | | | | | | | | | The slave volume uuid is appended to slave_url which is saved in 'gsync_slaves' dictionary with the commit "http://review.gluster.org/13111". But the gsyncd expects slave_url with out slave volume uuid otherwise it fails which in turn fails volume stop with geo-rep. This patch fixes the same. Change-Id: I036f4fce43115ff848daebfe1b2c58765cffab5a BUG: 1342420 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14636 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: update peers section in gsyncd confSaravanakumar Arumugam2016-05-311-3/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Once Slave volume uuid is involved as part of a geo-rep session, it is possible to create the same geo-rep session with different (slave)host. But, it reflects default values for geo-rep configuration values originally configured for old geo-rep session. Reason is, slave host is used while saving config options in gsyncd.conf. With new slave host, it is not possible to retrieve those config values. Solution: Remove slave host related information from gsyncd.conf and have only master volume and slave volume as part of peers section. Also, during upgrade from old geo-rep session, update peers section to reflect only master volume and slave volume. Change-Id: I7debf35a09a28d030b706b0c3e5d82c9b0467d0e BUG: 1340853 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/14558 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Handle Worker kill gracefully if worker already diedAravinda VK2016-05-301-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If Agent dies for any reason, monitor tries to kill Worker also. But if worker is also died then kill command raises error ESRCH: No such process. [2016-05-23 16:49:33.903965] I [monitor(monitor):326:monitor] Monitor: Changelog Agent died, Aborting Worker(/bricks/brick0/master_brick0) [2016-05-23 16:49:33.904535] E [syncdutils(monitor):276:log_raise_exception] <top>: FAIL: Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 306 in twrap tf(*aa) File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 393, in wmon slave_host, master) File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 327, in monitor os.kill(cpid, signal.SIGKILL) OSError: [Errno 3] No such process With this patch, monitor will gracefully handle if worker is already died. Change-Id: I3ae5f816a3a197343b64540cf46f5453167fb660 Signed-off-by: Aravinda VK <avishwan@redhat.com> BUG: 1339472 Reviewed-on: http://review.gluster.org/14512 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* core: assorted typos and spelling mistakes reported by Debian lintianKaleb S KEITHLEY2016-05-181-2/+2
| | | | | | | | | | | | | | | Also missing bang (!) in #!/bin/bash in shell scripts. Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b BUG: 1336793 Reported-by: Patrick Matthäi <pmatthaei@debian.org> Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14398 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd/geo-rep: slave volume uuid to identify a geo-rep sessionSaravanakumar Arumugam2016-05-122-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Currently, it is possible to create multiple geo-rep session from the Master host to Slave host(s), where Slave host(s) belonging to the same volume. For example: Consider Master Host M1 having volume tv1 and Slave volume tv2, which spans across two Slave hosts S1 and S2. Currently, it is possible to create geo-rep session from M1(tv1) to S1(tv2) as well as from M1(tv1) to S2(tv2). When the Slave Host is alone modified, it is identified as a new geo-rep session (as slave host and slave volume together are identifying Slave side). Also, it is possible to create both root and non-root geo-rep session between same Master volume and Slave volume. This should also be avoided. Solution: This multiple geo-rep session creation must be avoided and in order to avoid, use Slave volume uuid to identify a Slave. This way, we can identify whether a session is already created for the same Slave volume and avoid creating again (using different host). When the session creation is forced in the above scenario, rename the existing geo-rep session directory with new Slave Host mentioned. Change-Id: I9239759cbc0d15dad63c48b8cf62950bb687c7c8 BUG: 1294813 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13111 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Fix Entry Creation issue with non root UID/GIDAravinda VK2016-03-091-0/+20
| | | | | | | | | | | | | | | | | | | | | | | During entry_ops RENAME Geo-rep sends stat info along with the recorded info from Changelog. In Slave side if Source file exists Geo-rep renames to Target file by calling os.rename. If source file does not exists, it tries to create Target file directly using available stat info from Master. If UID and GID are different in Master for that file then stat info will have different UID/GID during Create. Geo-rep gets EACCES when it tries to create a new entry using gfid-access with different UID/GID. With this patch, Entry creation with different UID/GID is split into two operations. Create Entry with UID:0 and GID:0 and then set UID/GID. Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01 BUG: 1313303 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13542 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: hard-link rename issues on changelog replayMilind Changire2016-03-091-4/+24
| | | | | | | | | | | | | | | | | | | | | Problem: LINK + RENAME changelog when replayed after worker restart causes stale hard-links to persist since VFS returns success for RENAME if hard-links point to same inode. Solution: Worker detects RENAME being issued on hard-links to the same inode and unlinks the source file-name. Conditionally rename by verifying that the source gfid matches with the on-disk gfid on the slave. Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2 BUG: 1296174 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/13189 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com>
* geo-rep: Fix CPU utilization 100% with PopenAravinda VK2016-03-081-0/+4
| | | | | | | | | | | | | | | | | | | Errorstore is maintained by Geo-rep to collect errors from the child processes opened using Popen. If Popen.communicate is used then it closes stderr. When stderr is not available errorstore.tailer() will enter into infinite loop without gap. With this patch, sleep time added when stderr of Child process is already closed. Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94 BUG: 1315601 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/13637 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* georep: avoid creating multiple entries with same gfidMilind Changire2016-02-291-4/+12
| | | | | | | | | | | | | | | | | | | | | | Problem: CREATE + RENAME changelogs replayed by geo-replication cause stale old-name entries with same gfid on slave nodes. A gfid is a unique key in the file-system and should not be assigned to multiple entries. Solution: Create entry on slave only if lstat(gfid) at aux-mount fails. This applies to files as well as directories. Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919 BUG: 1296206 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/13316 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* geo-rep: Fix invalid disk size in gverify.shSaravanakumar Arumugam2016-02-291-7/+7
| | | | | | | | | | | | | | | | | | | | | | Problem: Due to the wrong interpretation of disk size, gverify.sh script exits in between with error "integer expression expected". This error can be observed, only if gverify.sh is run standalone. Solution: Fix the interpretation of disk size. Also, verify that master and slave's gluster version are same. Change-Id: Id095dfffd07a93a535a380f617ad215230979870 BUG: 1305839 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/13407 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
generated by cgit (git 2.34.1) at 2024-11-10 18:06:48 +0000