glusterfs.git/geo-replication, branch v5.13

geo-rep : fix mountbroker setup

2019-09-23T12:39:08+00:00

Problem:

Unable to setup mountbroker root directory while creating geo-replication
session for non-root user.

Casue:
With patch[1] which defines the max-port for glusterd one extra sapce
got added in field of 'option max-port'.
[1]. https://review.gluster.org/#/c/glusterfs/+/21872/

In geo-rep spliting of key-value pair form vol file was done on the
basis of space so this additional space caused "ValueError: too many values
to unpack".

Solution:
Use split so that it can treat consecutive whitespace as a single separator.

Backport of:
https://review.gluster.org/#/c/glusterfs/+/22716/.

>Fixes: bz#1709248
>Change-Id: Ia22070a43f95d66d84cb35487f23f9ee58b68c73
>Signed-off-by: Sunny Kumar 
>(cherry picked from commit 3dd03146bb7037ae2ebea0579d0b81be27fdd927)

Change-Id: Ia22070a43f95d66d84cb35487f23f9ee58b68c73
Fixes: bz#1750230
Signed-off-by: Sunny Kumar

geo-rep: Fix mount broker setup issue

2019-08-06T07:02:00+00:00

Even the use builtin 'type' command as in patch [1]
causes issues if argument in question is not part of PATH
environment variable for that user. This patch fixes the
same by doing source /etc/profile. This was already being
used in another part of script.

[1] https://review.gluster.org/23089

Backport of:
 > Patch: https://review.gluster.org/23136/
 > Change-Id: Iceb78835967ec6a4350983eec9af28398410c002
 > BUG: 1734738
 > Signed-off-by: Kotresh HR 

Change-Id: Iceb78835967ec6a4350983eec9af28398410c002
fixes: bz#1737716
Signed-off-by: Kotresh HR

geo-rep: Fix mount broker setup issue

2019-07-29T08:54:36+00:00

The patch [1] added validation in gverify.sh to check if the gluster
binary exists on slave by executing gluster directly on slave.  But for
non-root users, even though gluster binary is present, path is not
found when executed via ssh. Hence validate the gluster binary using
bash builtin 'type' command.

[1] https://review.gluster.org/19224

Backport of:
 > Patch: https://review.gluster.org/23089/
 > Change-Id: I93ca62c0c5b1e16263e586ddbbca8407d60ca126
 > BUG: 1731920
 > Signed-off-by: Kotresh HR 
(cherry picked from commit 2aa731a259ea457c07494e3c3edf6d5f7c02fe77)

Change-Id: I93ca62c0c5b1e16263e586ddbbca8407d60ca126
fixes: bz#1733881
Signed-off-by: Kotresh HR

eventsapi: Fix Python3 compatibility issues

2019-02-26T05:11:49+00:00

- Fixed Relative import and non-package import related issues.
- socketserver import issues fix
- Renamed installed directory name to `gfevents` from `events`(To
  avoid any issues with other global libs)

Fixes: bz#1649054
Change-Id: I3dc38bc92b23387a6dfbcc0ab8283178235bf756
Signed-off-by: Aravinda VK 
(cherry picked from commit cd68f7b88b9a2c9a4e4ff9fca61517384e54130a)

geo-rep: Fix syncing of files with non-ascii filenames

2018-12-26T16:50:01+00:00

Problem:
  Creation of files/directories with non-ascii names fails
  to sync to the slave. It crashes with below traceback on
  slave.
  ...
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/repce.py", line 118, in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/resource.py", line 709, in entry_ops
    [ESTALE, EINVAL, EBUSY])
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap
    return call(*arg)
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 83, in lsetxattr
    cls.raise_oserr()
  File "/usr/lib/x86_64-linux-gnu/glusterfs/python/syncdaemon/libcxattr.py", line 38, in raise_oserr
    raise OSError(errn, os.strerror(errn))
  OSError: [Errno 12] Cannot allocate memory

Cause:
  The length calculation arguments passed to blob creation was done before encoding. Hence
  was failing in gfid-access layer.

Fix:
  It appears that the calculating lenght properly fixes this issue. But it will cause
  issues in other places in 'python2' and not in 'python3'. So encoding and decoding
  each required string to make geo-rep compatible with both 'python2' and 'python3'
  is a nightmare and is not fool proof. Hence kept 'python2' code as is with out
  encode/decode and applied encode/decode only to 'python3'

Added non-ascii filename tests to regression

Backport of:
 > Patch: https://review.gluster.org/21668
 > BUG: 1650893
 > Change-Id: I35cfaf848e07b1a0b5cb93c01b98b472f08271a6
 > Signed-off-by: Kotresh HR 

fixes: bz#1648642
Change-Id: I35cfaf848e07b1a0b5cb93c01b98b472f08271a6
Signed-off-by: Kotresh HR

geo-rep: Fix traceback with symlink metadata sync

2018-12-12T12:54:37+00:00

While syncing metadata, 'os.chmod', 'os.chown',
'os.utime' should be used without de-reference.
But python supports only 'os.chown' without
de-reference. That's mostly because Linux
doesn't support 'chmod' on symlink file itself
but it does support 'chown'.

So while syncing metadata ops, if it's symlink
we should only sync 'chown' and not do 'chmod'
and 'utime'. It will lead to tracebacks with
errors like EROFS, EPERM, ACCESS, ENOENT.
All the three errors (EPERM, ACCESS, ENOENT)
were handled except EROFS. But the way it was
handled was not fool proof. The operation is
tried and failure was handled based on the errors.
All the errors with symlink file for 'chown',
'utime' had to be passed to safe errors list of
'errno_wrap'. This patch handles it better by
avoiding 'chmod' and 'utime' if it's symlink
file.
Backport of :
< Patch: https://review.gluster.org/21546/
> BUG: bz#1646104
> Change-Id: Ic354206455cdc7ab2a87d741d81f4efe1f19d77d
> Signed-off-by: Kotresh HR 
(cherry picked from commit 3c6cf9a4a1b46cab2dc53c1ee0afca0fe993102e)


fixes: bz#1654115
Change-Id: Ic354206455cdc7ab2a87d741d81f4efe1f19d77d
Signed-off-by: Kotresh HR

geo-rep: Fix permissions with non-root setup

2018-11-29T15:35:41+00:00

Problem:
In non-root fail-over/fail-back(FO/FB), when slave is
promoted as master, the session goes to 'Faulty'

Cause:
The command 'gluster-mountbroker  '
is run as a pre-requisite on slave in non-root setup.
It modifies the permission and group of following required
directories and files recursively

  [1] /var/lib/glusterd/geo-replication
  [2] /var/log/glusterfs/geo-replication-slaves

In a normal setup, this is executed on slave node and hence
doing it recursively is not an issue on [1]. But when original
master becomes slave in non-root during FO/FB, it contains
ssh public keys and modifying permissions on them causes
geo-rep to fail with incorrect permissions.

Fix:
Don't do permission change recursively. Fix permissions for
required files.

Backport of:
 > Patch: https://review.gluster.org/#/c/glusterfs/+/21689/
 > BUG: bz#1651498
 > Change-Id: I68a744644842e3b00abc26c95c06f123aa78361d
 > Signed-off-by: Kotresh HR 
(cherry picked from commit b2776b1ec1ad845ba568c4439bca3b57cc4d2592)


fixes: bz#1654117
Change-Id: I68a744644842e3b00abc26c95c06f123aa78361d
Signed-off-by: Kotresh HR

geo-rep: Add more intelligence to automatic error handling

2018-11-09T18:46:54+00:00

Geo-rep's automatic error handling does gfid conflict
resolution. But if there are ENOENT errors because the
parent is not synced to slave, it doesn' handle them.
This patch adds the intelligence to create missing
parent directories on slave. It can create the missing
directories upto the depth of 10.

Backport of:

 > Patch: https://review.gluster.org/21498/
 > fixes: bz#1643402
 > Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
 > Signed-off-by: Kotresh HR 
(cherry picked from commit 19775e0445411cca9ddd9d294fd54d0b6fbe6a03)

fixes: bz#1646896
Change-Id: Ic97ed1fa5899c087e404d559e04f7963ed7bb54c
Signed-off-by: Kotresh HR

geo-rep/scripts: Fix traceback in gluster-mountbroker

2018-11-08T14:36:32+00:00

When 'gluster-mountbroker status' was issued, it
crashes in a corner case with 'str object has not
attribute get'. Fixed the same.

Backport of:
> BUG: 1643929
> Signed-off-by: Kotresh HR 
> Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb
(cherry picked from commit 5987b3388126a3c5e77481913cbaa4142117d19a)

fixes: bz#1644515
Signed-off-by: Kotresh HR 
Change-Id: Iaf1a937ed0136b3b2058230c75fa89a215d8a5eb

georep: python2 to python3 compat - scheduler

2018-11-08T14:35:30+00:00

1. scheduler - Popen
2. syncdutils - corner case on failure

Backport of:
> Patch: https://review.gluster.org/21505
> BUG: 1643932
> Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10
> Signed-off-by: Kotresh HR 
(cherry picked from commit 33e96100e17e9a293db6d63d9d5449d6c2d69376)

fixes: bz#1644514
Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10
Signed-off-by: Kotresh HR