summaryrefslogtreecommitdiffstats
path: root/geo-replication
Commit message (Collapse)AuthorAgeFilesLines
* geo-rep: IPv6 supportAravinda VK2019-03-154-7/+38
| | | | | | | | | | | | | `address_family=inet6` needs to be added while mounting master and slave volumes in gverify script. New option introduced to gluster cli(`--inet6`) which will be used internally by geo-rep while calling `gluster volume info --remote-host=<ipv6>`. Fixes: bz#1688833 Change-Id: I1e0d42cae07158df043e64a2f991882d8c897837 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* eventsapi: Fix Python3 compatibility issuesAravinda VK2019-02-251-5/+5
| | | | | | | | | | | - Fixed Relative import and non-package import related issues. - socketserver import issues fix - Renamed installed directory name to `gfevents` from `events`(To avoid any issues with other global libs) Fixes: bz#1679406 Change-Id: I3dc38bc92b23387a6dfbcc0ab8283178235bf756 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* geo-rep : fix incorrectly formatted authorized_keysSunny Kumar2019-02-251-2/+2
| | | | | | | | | | | | | Problem : While Geo-rep setup when creating an ssh authorized_keys the geo-rep setup inserts an extra space before the "ssh-rsa" label. This gets flagged by an enterprise customer's security scan as a security violation. Solution: Remove extra space in GSYNCD_CMD & TAR_CMD. Change-Id: I956f938faef0e0883703bbc337b1dc2770e4a921 fixes: bz#1679401 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* geo-rep: Fix configparser import issueKotresh HR2019-02-051-2/+2
| | | | | | | | | | | | | | | 'configparser' is backported to python2 and can be installed using pip (pip install configparser). So trying to import 'configparser' first and later 'ConfigParser' can cause issues w.r.t unicode strings. Always try importing 'ConfigParser' first and then 'configparser'. This solves python2/python3 compat issues. Change-Id: I2a87c3fc46476296b8cb547338f35723518751cc fixes: bz#1671637 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep : fix rename sync on hybrid crawlSunny Kumar2019-01-172-13/+11
| | | | | | | | | | | | | | Problem: When geo-rep is configured as hybrid crawl directory renames are not synced to the slave. Solution: Rename sync of directory was failing due to incorrect destination path calculation. During check for existence on slave we miscalculated realpath. <host:brickpath/dir>. Change-Id: I23f1ea60e86a917598fe869d5d24f8da654d8a0a fixes: bz#1665826 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* geo-rep: Make slave volume read-only (by default)Harpreet Kaur2018-12-071-0/+1
| | | | | | | | | | | | | | | Added a command to set "features.read-only" option to a default value "on" for slave volume. Changes are made in: $SRC//extras/hook-scripts/S56glusterd-geo-rep-create-post.sh for root geo-rep and $SRC/geo-replication/src/set_geo_rep_pem_keys.sh for non-root geo-rep. Fixes: bz#1654187 Change-Id: I15beeae3506f3f6b1dcba0a5c50b6344fd468c7c Signed-off-by: Harpreet Kaur <hlalwani@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-052-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* geo-rep: Fix syncing of files with non-ascii filenamesKotresh HR2018-12-044-68/+174
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Creation of files/directories with non-ascii names fails to sync to the slave. It crashes with below traceback on slave. ... File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/repce.py", line 118, in worker res = getattr(self.obj, rmeth)(*in_data[2:]) File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/resource.py", line 709, in entry_ops [ESTALE, EINVAL, EBUSY]) File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap return call(*arg) File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 83, in lsetxattr cls.raise_oserr() File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 38, in raise_oserr raise OSError(errn, os.strerror(errn)) OSError: [Errno 12] Cannot allocate memory Cause: The length calculation arguments passed to blob creation was done before encoding. Hence was failing in gfid-access layer. Fix: It appears that the calculating lenght properly fixes this issue. But it will cause issues in other places in 'python2' and not in 'python3'. So encoding and decoding each required string to make geo-rep compatible with both 'python2' and 'python3' is a nightmare and is not fool proof. Hence kept 'python2' code as is with out encode/decode and applied encode/decode only to 'python3' Added non-ascii filename tests to regression fixes: bz#1650893 Change-Id: I35cfaf848e07b1a0b5cb93c01b98b472f08271a6 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix permissions with non-root setupKotresh HR2018-11-261-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: In non-root fail-over/fail-back(FO/FB), when slave is promoted as master, the session goes to 'Faulty' Cause: The command 'gluster-mountbroker <mountbroker-root> <group>' is run as a pre-requisite on slave in non-root setup. It modifies the permission and group of following required directories and files recursively [1] /var/lib/glusterd/geo-replication [2] /var/log/glusterfs/geo-replication-slaves In a normal setup, this is executed on slave node and hence doing it recursively is not an issue on [1]. But when original master becomes slave in non-root during FO/FB, it contains ssh public keys and modifying permissions on them causes geo-rep to fail with incorrect permissions. Fix: Don't do permission change recursively. Fix permissions for required files. fixes: bz#1651498 Change-Id: I68a744644842e3b00abc26c95c06f123aa78361d Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix traceback with symlink metadata syncKotresh HR2018-11-061-15/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | While syncing metadata, 'os.chmod', 'os.chown', 'os.utime' should be used without de-reference. But python supports only 'os.chown' without de-reference. That's mostly because Linux doesn't support 'chmod' on symlink file itself but it does support 'chown'. So while syncing metadata ops, if it's symlink we should only sync 'chown' and not do 'chmod' and 'utime'. It will lead to tracebacks with errors like EROFS, EPERM, ACCESS, ENOENT. All the three errors (EPERM, ACCESS, ENOENT) were handled except EROFS. But the way it was handled was not fool proof. The operation is tried and failure was handled based on the errors. All the errors with symlink file for 'chown', 'utime' had to be passed to safe errors list of 'errno_wrap'. This patch handles it better by avoiding 'chmod' and 'utime' if it's symlink file. fixes: bz#1646104 Change-Id: Ic354206455cdc7ab2a87d741d81f4efe1f19d77d Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Add more intelligence to automatic error handlingKotresh HR2018-10-301-24/+46
| | | | | | | | | | | | | Geo-rep's automatic error handling does gfid conflict resolution. But if there are ENOENT errors because the parent is not synced to slave, it doesn' handle them. This patch adds the intelligence to create missing parent directories on slave. It can create the missing directories upto the depth of 10. fixes: bz#1643402 Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep/scripts: Fix traceback in gluster-mountbrokerKotresh HR2018-10-301-1/+1
| | | | | | | | | | When 'gluster-mountbroker status' was issued, it crashes in a corner case with 'str object has not attribute get'. Fixed the same. fixes: bz#1643929 Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb
* georep: python2 to python3 compat - schedulerKotresh HR2018-10-301-1/+1
| | | | | | | | | 1. scheduler - Popen 2. syncdutils - corner case on failure fixes: bz#1643932 Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix issue in gfid-conflict-resolutionKotresh HR2018-10-261-17/+24
| | | | | | | | | | | | | | | | | | | | | | | | | Problem: During gfid-conflict-resolution, geo-rep crashes with 'ValueError: list.remove(x): x not in list' Cause and Analysis: During gfid-conflict-resolution, the entry blob is passed back to master along with additional information to verify it's integrity. If everything looks fine, the entry creation is ignored and is deleted from the original list. But it is crashing during removal of entry from the list saying entry not in list. The reason is that the stat information in the entry blob was modified and sent back to master if present. Fix: Send back the correct stat information for gfid-conflict-resolution. fixes: bz#1642865 Change-Id: I47a6aa60b2a495465aa9314eebcb4085f0b1c4fd Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep : fix high sev coverity isuueSunny Kumar2018-10-251-1/+0
| | | | | | | | This patch fixes CID: 1382374: USE_AFTER_FREE. Change-Id: If408f52ee291312fb83095126ebd6bb79ae95e26 updates: bz#789278 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* georep: python2 to python3 compat - syscallsKotresh HR2018-10-087-35/+88
| | | | | | | | | | | | 1. ctypes/syscalls A) arguments is expected to be encoded B) Raw conversion of return value from bytearray into string 2. struct pack/unpack - Raw converstion of string to bytearray 3. basestring -> str Updates: #411 Change-Id: I80f939adcdec0ed0022c87c0b76d057ad5559e5a Signed-off-by: Kotresh HR <khiremat@redhat.com>
* python: remove shebangs of non-main scripts and make others executableNiels de Vos2018-10-025-4/+0
| | | | | | | | | | | | | | | Some of the scripts that have a #!/usr/bin/python3 shebang do not have a main() like function. These scripts will not get executed but only imported. They do not need the shebang. A few others are not installed with 'make install', but do have a main() like function. These scripts are expected to be used by developers for different tasks (mostly code generation). Marking these scripts executable to make it easier to identify them. Change-Id: I73541471deb7e0830766b804786244e73dfe4221 Updates: #411 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* georep: python2 to python3 compat - pickleKotresh HR2018-10-021-4/+12
| | | | | | | | | | | | | | | | | | | | | | Handle py2 and py3 compatibility for pickling and unpickling. Geo-rep pickles and unpickles reading sys.stdin and sys.stdout streams. py2 and py3 compatibility expects the streams to be opened in binary mode but the sys.stdout objects are different in python2 and python3 python2: >>> type(sys.stdout) <type 'file'> python3: >>> type(sys.stdout) <class '_io.TextIOWrapper'> So in order to access binary stream, using sys.stdin.buffer in python3 Updates: #411 Change-Id: I1a633ccdddff5baf0cf05a8b493add39ddf75bd7 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: python2 and python3 compat - bytes-strKotresh HR2018-10-021-3/+5
| | | | | | | | | 1. Fix fdopen used for pid file 2. Fix sha256 checksum calculation Updates: #411 Change-Id: Ic173d104a73822c29aca260ba6de872cd8d23f86 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: python2 to python3 compatibility-mount writeKotresh HR2018-10-021-3/+9
| | | | | | | | | | python3 expects byte string for os.write. This works for both py2 and py3. Fixed the same for geo-rep mount testing code path. Updates: #411 Change-Id: I2dfedcb0869457707bcca4d2847ef0d52bff1987 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: python3 compatibility (Popen)Kotresh HR2018-09-253-12/+21
| | | | | | | | | | | | | | The file objects for python3 by default is opened in binary mode where as in python2 it's opened as text by default. The geo-rep code parses the output of Popen assuming it as text, hence used the 'universal_newlines' flag which provides backward compatibility for the same. Change-Id: I371a03b6348af9666164cb2e8b93d47475431ad9 Updates: #411 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: Fix python3 compatibility (configparser)Kotresh HR2018-09-251-5/+2
| | | | | | | | | '%' needs special handling in config and also removed duplicate misspelled 'changelog-archive-format' config. Updates: #411 Change-Id: I33621a62bdf5f781ee62e6cedec0c2df3f5d70cf Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: Fix python3 compatibility (os.pipe)Kotresh HR2018-09-253-7/+19
| | | | | | | | | | | | | | 'os.pipe' returns pair of file descriptors which are non-inheritable by child processes. But geo-rep uses te inheritable nature of pipe fds to communicate between parent and child processes. Hence wrote a compatiable pipe routine which works well both with python2 and python3 with inheritable nature. Updates: #411 Change-Id: I869d7a52eeecdecf3851d44ed400e69b32a612d9 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep : fix log message in master.pySunny Kumar2018-09-241-2/+2
| | | | | | | | This patch fixes spell mistake in log message. Change-Id: I84779c64aef6698cbc1a60ae1a82533e8a6a6e3d updates: bz#1193929 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* geo-rep: Fix issues related config setKotresh HR2018-09-183-7/+11
| | | | | | | | | | | | | | | | | | | | 1. '--ignore-mising-args' option for rsync is not being used even though the rsync version is greater than 3.1.0. Fixed the same. 2. '--existing' option for rsync is also not being used. Fixed the same. 3. geo-rep config fails to set rsync-options as the value contains '--'. Interestingly, python argsparse treats the value with '--' (e.g., --ignore-missing-args) as option. But when passed with something like --value=--ignore-missing-args, it succeeds. Fixed the same. Change-Id: Iaeb838acaff1c2920fee9c7f920c99edce13a0a1 Signed-off-by: Kotresh HR <khiremat@redhat.com> fixes: bz#1629561
* Land part 2 of clang-format changesGluster Ant2018-09-122-385/+396
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Land clang-format changesGluster Ant2018-09-121-4/+5
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* multiple: Fix python2 to python3 compatKotresh HR2018-09-072-3/+3
| | | | | | Change-Id: Iac241166d7a35dc7cc6cf07850f9f1bce38fe207 Updates: #411 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* core: switch to python3Sanju Rakonde2018-09-051-1/+1
| | | | | | | | | | Commit af0d5a9b5375a5cd87ac10b429e2b9934718ce5b changes python2 -> python3. This patch fixes a small issue introduced by it. Change-Id: Ib23c73683f570e8891f41476b661f37c89635fb5 updates: #411 Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* core: python3Kaleb S. KEITHLEY2018-09-0311-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* geo-rep: Make automatic gfid conflict resolution optionalKotresh HR2018-08-272-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | Autmatic gfid conflict resolution needs to be disabled during failover/failback as it might lead to data loss in the following scenario. 1. Master went down without syncing directory "dir1" to slave. 2. When slave is failed over to master, if a new file is written inside "dir1", creating dir1 again if not present, "dir1" ends up with different gfid on original slave. 3. When original master is up and failed back, due to automatic gfid conflict resolution, "dir1" present in original master is deleted losing all files and only new file created on original slave is restored. Hence during failover/failback, automatic gfid conflict resolution should be disabled. So in these cases, appropriate decision is taken. fixes: bz#1622076 Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I433616f5d3e13d4b6eb675475bd554ca34928573
* geo-rep : fix few coverity issues in procdiggy.cSunny Kumar2018-08-201-4/+6
| | | | | | | | | | | | | This patch fixes BAD_FREE at line number 75,105 and UNUSED_VALUE at line number 108. CID : 1274068 scan detail at [1]. [1] https://scan6.coverity.com/reports.htm#v42401/p10714/fileInstanceId=84384726&defectInstanceId=25600457&mergedDefectId=727233 Change-Id: I8dff42546204bf78c178c9dcaba534888c5354e9 updates: bz#789278 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* geo-rep: Cleanup unwanted codeKotresh HR2018-08-202-22/+0
| | | | | | | | | | | Cleanup the Active/Passive logging code which is redundant. With new status infra implmented, this is redundant as every status switch is logged by status infra. fixes: bz#1619027 Change-Id: I0a6644cb998f3520e62a5189f21e4d66acc0e7c5 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix deadlock during worker startKotresh HR2018-08-132-4/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Analysis: Monitor process spawns monitor threads (one per brick). Each monitor thread, forks worker and agent processes. Each monitor thread, while intializing, updates the monitor status file. It is synchronized using flock. The race is that, some thread can fork worker while other thread opened the status file resulting in holding the reference of fd in worker process. Cause: flock gets unlocked either by specifically unlocking it or by closing all duplicate fds referring to the file. The code was relying on fd close, hence a reference in worker/agent process by fork could cause the deadlock. Fix: 1. flock is unlocked specifically. 2. Also made sure to update status file in approriate places so that the reference is not leaked to worker/agent process. With this fix, both the deadlock and possible fd leaks is solved. fixes: bz#1614799 Change-Id: I0d1ce93072dab07d0dbcc7e779287368cd9f093d Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: fix hard-coded paths in gsyncd.conf.inKaleb S. KEITHLEY2018-07-251-2/+2
| | | | | | | | | | | | | | | | | This is part of the reason why we use autoconf (i.e. configure). For an ordinary clone+autogen.sh+configure SBIN_DIR is /usr/local/sbin; for an rpm or dpkg build it will be /usr/sbin. I wonder how many more are lurking in our sources? /usr/libexec is one that frequently bites us on Debian and Ubuntu, which don't have /usr/libexec. (But it's all Linux, right?) See https://bugzilla.redhat.com/show_bug.cgi?id=1601532 Reported-by: lohmaier+rhbz@gmail.com Change-Id: I6523894416cc06236ea1f99529efd36e957bd98e updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-227-11/+13
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* geo-rep: Fix issues with gfid conflict handlingKotresh HR2018-07-203-64/+169
| | | | | | | | | | | | | | | | | | | | | | | 1. MKDIR/RMDIR is recorded on all bricks. So if one brick succeeds creating it, other bricks should ignore it. But this was not happening. The fix rename of directories in hybrid crawl, was trying to rename the directory to itself and in the process crashing with ENOENT if the directory is removed. 2. If file is created, deleted and a directory is created with same name, it was failing to sync. Again the issue is around the fix for rename of directories in hybrid crawl. Fixed the same. If the same case was done with hardlink present for the file, it was failing. This patch fixes that too. fixes: bz#1598884 Change-Id: I6f3bca44e194e415a3d4de3b9d03cc8976439284 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix symlink rename syncing issueKotresh HR2018-07-121-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Geo-rep sometimes fails to sync the rename of symlink if the I/O is as follows 1. touch file1 2. ln -s "./file1" sym_400 3. mv sym_400 renamed_sym_400 4. mkdir sym_400 The file 'renamed_sym_400' failed to sync to slave Cause: Assume there are three distribute subvolume (brick1, brick2, brick3). The changelogs are recorded as follows for above I/O pattern. Note that the MKDIR is recorded on all bricks. 1. brick1: ------- CREATE file1 SYMLINK sym_400 RENAME sym_400 renamed_sym_400 MKDIR sym_400 2. brick2: ------- MKDIR sym_400 3. brick3: ------- MKDIR sym_400 The operations on 'brick1' should be processed sequentially. But since MKDIR is recorded on all the bricks, The brick 'brick2/brick3' processed MKDIR first before 'brick1' causing out of order syncing and created directory sym_400 first. Now 'brick1' processed it's changelog. CREATE file1 -> succeeds SYMLINK sym_400 -> No longer present in master. Ignored RENAME sym_400 renamed_sym_400 While processing RENAME, if source('sym_400') doesn't present, destination('renamed_sym_400') is created. But geo-rep stats the name 'sym_400' to confirm source file's presence. In this race, since source name 'sym_400' is present as directory, it doesn't create destination. Hence RENAME is ignored. Fix: The fix is not rely only on stat of source name during RENAME. It should stat the name and if the name is present, gfid should be same. Only then it can conclude the presence of source. fixes: bz#1600405 Change-Id: I9fbec4f13ca6a182798a7f81b356fe2003aff969 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* contrib: Remove contrib/ipaddr-py entirely.Nigel Babu2018-07-091-2/+2
| | | | | | | | This module is no longer being used. Fixes: bz#1597512 Change-Id: Ie5faf55c5961d9d7b5082c9c257351af712c41d7 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-07-093-13/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, and https://review.gluster.org/#/c/20364/ Fixes glupy.py python2isms, iteritems -> items, and some overlooked print() in georep/peer_mountbroker.in Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: Idda031c1ec975417c79323aea33e7b694e752b2a updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* geo-rep: Fix geo-rep for older versions of unshareKotresh HR2018-06-223-7/+42
| | | | | | | | | | | | | | Geo-rep mounts are private to worker. It uses mount namespace using unshare command to achieve the same. Well, the unshare command has to support '--propagation' option. So geo-rep breaks on the systems with older unshare version. The patch makes it fall back to lazy umount behaviour if the unshare does not support propagation option. fixes: bz#1589782 Change-Id: Ia614f068aede288d63ac62fea4461b1865066054 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix problems in python2 -> python3 compatKotresh HR2018-06-112-3/+3
| | | | | | | | | 1. Import configparser module correctly 2. Import thread module correctly Updates: #411 Change-Id: I522453d23c256b694fa58d285f413b8c4dd6595c Signed-off-by: Kotresh HR <khiremat@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-071-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, and https://review.gluster.org/#/c/20104/ https://review.gluster.org/#/c/20162/ This patch changes uses of map() and raise(), and a few cases of print() that were overlooked in the prior patch that fixed print. Note: Fedora packaging guidelines require explicit shebangs, so popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, zip, map, and raise have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: Id62ea491e4ab5dd390075c5c6d9d889cf6f9da27 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-048-6/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, and https://review.gluster.org/#/c/19952/ This patch adds version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: Fedora packaging guidelines require explicit shebangs, so popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, and zip have already been applied. Note: these 2to3 fixes report no changes are necessary: exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I8d393064a1837874d8b4bc87c8ce05c679664642 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* geo-rep: Fix upgrade issueKotresh HR2018-05-071-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | Cause and Analysis: The last synced changelog for entry operations is marked in current version to avoid re-processing of already processed entry operations in a batch during crash/restart of geo-rep. This was not present in previous versoins. The marker is maintained in the dictionary with the key 'last_synced_entry' and dictionary is persisted into status file. So upgrading to current version in which the marker is present was failing with KeyError. Solution: Load the dictionary with default keys first which contains all the keys including latest ones and then load the values from status file instead of doing otherwise. fixes: bz#1575490 Change-Id: Ic654e6f9a3c97f616761f1362f890352a2186fb4 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-024-9/+13
| | | | | | | | | | see https://review.gluster.org/#/c/19788/ use print fn from __future__ Change-Id: If5075d8d9ca9641058fbc71df8a52aa35804cda4 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* geo-rep: Fix syncing of symlinkKotresh HR2018-04-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Problem: If symlink is created on master pointing to current directory (e.g symlink -> ".") with non root uid or gid, geo-rep worker crashes with ENOTSUP. Cause: Geo-rep creates the symlink on slave and fixes the uid and gid using chown cmd. os.chown dereferences the symlink which is pointing to ".gfid" which is not supported. Note that geo-rep operates on aux-gfid-mount (e.g. "/mnt/.gfid/<gfid-of-symlink-file>"). Solution: The uid or gid change is acutally on symlink file. So use os.lchown, i.e, don't deference. BUG: 1567209 Change-Id: I63575fc589d71f987bef1d350c030987738c78ad updates: bz#1567209 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-1211-10/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* python: Remove all uses of find_library. Fixes #1450593Niklas Hambüchen2018-03-242-4/+2
| | | | | | | | `find_library()` doesn't consider LD_LIBRARY_PATH on Python < 3.6. Change-Id: Iee26085cb5d14061001f19f032c2664d69a378a8 BUG: 1450593 Signed-off-by: Niklas Hambüchen <mail@nh2.me>
* geo-rep: Remove lazy umount and use mount namespacesKotresh HR2018-02-224-15/+29
| | | | | | | | | | | | | Lazy umounting the master volume by worker causes issues with rsync's usage of getcwd. Henc removing the lazy umount and using private mount namespace for the same. On the slave, the lazy umount is retained as we can't use private namespace in non root geo-rep setup. Change-Id: I403375c02cb3cc7d257a5f72bbdb5118b4c8779a BUG: 1546129 Signed-off-by: Kotresh HR <khiremat@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-19 23:32:49 +0000