glusterfs.git/geo-replication, branch v3.8.0

geo-rep: Handle Worker kill gracefully if worker already died

2016-06-13T13:47:00+00:00

If Agent dies for any reason, monitor tries to kill Worker also. But
if worker is also died then kill command raises error ESRCH: No such
process.

[2016-05-23 16:49:33.903965] I [monitor(monitor):326:monitor] Monitor:
    Changelog Agent died, Aborting Worker(/bricks/brick0/master_brick0)
[2016-05-23 16:49:33.904535] E [syncdutils(monitor):276:log_raise_exception]
    : FAIL:
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 306 in
  twrap
    tf(*aa)
  File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 393, in
  wmon
     slave_host, master)
  File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 327, in
  monitor
     os.kill(cpid, signal.SIGKILL)
     OSError: [Errno 3] No such process

With this patch, monitor will gracefully handle if worker is already died.

> Change-Id: I3ae5f816a3a197343b64540cf46f5453167fb660
> Signed-off-by: Aravinda VK 
> BUG: 1339472
> Reviewed-on: http://review.gluster.org/14512
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Kotresh HR 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 4f4a94a35a24d781f3f0e584a8cb59c019e50d6f)

Change-Id: I3ae5f816a3a197343b64540cf46f5453167fb660
Signed-off-by: Aravinda VK 
BUG: 1341069
Reviewed-on: http://review.gluster.org/14563
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

glusterd/geo-rep: upgrade path when slave vol uuid involved

2016-06-13T10:36:56+00:00

slave volume uuid is involved in identifying a geo-replication
session.

This patch addresses upgrade path, where existing geo-rep session
is gracefully upgraded to involve slave volume uuid.

Change-Id: Ib7ff5109b161592f24fc86fc7e93a407655fab86
BUG: 1342452
Reviewed-on: http://review.gluster.org/#/c/14425/
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/14640
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Fix volume stop with geo-rep session

2016-06-06T06:51:09+00:00

Backport of http://review.gluster.org/14636

The slave volume uuid is appended to slave_url
which is saved in 'gsync_slaves' dictionary with
the commit "http://review.gluster.org/13111".
But the gsyncd expects slave_url with out slave
volume uuid otherwise it fails which in turn
fails volume stop with geo-rep. This patch
fixes the same.

Change-Id: I036f4fce43115ff848daebfe1b2c58765cffab5a
BUG: 1342634
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14645
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: update peers section in gsyncd conf

2016-06-02T10:53:27+00:00

Problem:
Once Slave volume uuid is involved as part of a geo-rep session, it is
possible to create the same geo-rep session with different (slave)host.

But, it reflects default values for geo-rep configuration values originally
configured for old geo-rep session.
Reason is, slave host is used while saving config options in gsyncd.conf.
With new slave host, it is not possible to retrieve those config values.

Solution:
Remove slave host related information from gsyncd.conf and have only master
volume and slave volume as part of peers section.

Also, during upgrade from old geo-rep session, update peers section to
reflect only master volume and slave volume.

Change-Id: I7debf35a09a28d030b706b0c3e5d82c9b0467d0e
BUG: 1341108
Reviewed-on: http://review.gluster.org/#/c/14558/
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/14565
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

glusterd/geo-rep: slave volume uuid to identify a geo-rep session

2016-05-23T08:13:58+00:00

Problem:
Currently, it is possible to create multiple geo-rep session from
the Master host to Slave host(s), where Slave host(s) belonging
to the same volume.

For example:
Consider Master Host M1 having volume tv1 and Slave volume tv2,
which spans across two Slave hosts S1 and S2.
Currently, it is possible to create geo-rep session from
M1(tv1) to S1(tv2) as well as from M1(tv1) to S2(tv2).

When the Slave Host is alone modified, it is identified as a new geo-rep
session (as slave host and slave volume together are identifying
Slave side).

Also, it is possible to create both root and non-root geo-rep session between
same Master volume and Slave volume. This should also be avoided.

Solution:
This multiple geo-rep session creation must be avoided and
in order to avoid, use Slave volume uuid to identify a Slave.
This way, we can identify whether a session is already created for
the same Slave volume and avoid creating again (using different host).

When the session creation is forced in the above scenario, rename
the existing geo-rep session directory with new Slave Host mentioned.

Change-Id: I9239759cbc0d15dad63c48b8cf62950bb687c7c8
BUG: 1336704
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13111
Reviewed-by: Kotresh HR 
Tested-by: Kotresh HR 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
(cherry picked from commit a9128cda34b1f696b717ba09fa0ac5a929be8969)
Reviewed-on: http://review.gluster.org/14372

core: assorted typos and spelling mistakes reported by Debian lintian

2016-05-18T13:55:02+00:00

Also missing bang (!) in #!/bin/bash in shell scripts.

Backport of
> http://review.gluster.org/#/c/14398/
> BUG: 1336793
> Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b

Change-Id: I003554daede9937a41010e085c74e77957c5c608
BUG: 1336794
Reported-by: Patrick Matthäi 
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14400
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

geo-rep: Fix Entry Creation issue with non root UID/GID

2016-03-09T11:02:01+00:00

During entry_ops RENAME Geo-rep sends stat info along with the
recorded info from Changelog. In Slave side if Source file exists
Geo-rep renames to Target file by calling os.rename. If source file
does not exists, it tries to create Target file directly using available
stat info from Master. If UID and GID are different in Master for that
file then stat info will have different UID/GID during Create. Geo-rep
gets EACCES when it tries to create a new entry using gfid-access with
different UID/GID.

With this patch, Entry creation with different UID/GID is split into two
operations. Create Entry with UID:0 and GID:0 and then set UID/GID.

Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01
BUG: 1313303
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13542
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: hard-link rename issues on changelog replay

2016-03-09T08:25:15+00:00

Problem:
LINK + RENAME changelog when replayed after worker restart causes stale
hard-links to persist since VFS returns success for RENAME if hard-links
point to same inode.

Solution:
Worker detects RENAME being issued on hard-links to the same inode and
unlinks the source file-name. Conditionally rename by verifying that the
source gfid matches with the on-disk gfid on the slave.

Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2
BUG: 1296174
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13189
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Fix CPU utilization 100% with Popen

2016-03-08T13:53:49+00:00

Errorstore is maintained by Geo-rep to collect errors from
the child processes opened using Popen. If Popen.communicate
is used then it closes stderr. When stderr is not available
errorstore.tailer() will enter into infinite loop without gap.

With this patch, sleep time added when stderr of Child process
is already closed.

Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94
BUG: 1315601
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13637
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

georep: avoid creating multiple entries with same gfid

2016-03-01T07:58:05+00:00

Problem:
CREATE + RENAME changelogs replayed by geo-replication cause
stale old-name entries with same gfid on slave nodes.
A gfid is a unique key in the file-system and should not be
assigned to multiple entries.

Solution:
Create entry on slave only if lstat(gfid) at aux-mount fails.
This applies to files as well as directories.

Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919
BUG: 1296206
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13316
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System