glusterfs.git/geo-replication/syncdaemon, branch v3.7.10

geo-rep: Fix Entry Creation issue with non root UID/GID

2016-03-09T11:02:36+00:00

During entry_ops RENAME Geo-rep sends stat info along with the
recorded info from Changelog. In Slave side if Source file exists
Geo-rep renames to Target file by calling os.rename. If source file
does not exists, it tries to create Target file directly using available
stat info from Master. If UID and GID are different in Master for that
file then stat info will have different UID/GID during Create. Geo-rep
gets EACCES when it tries to create a new entry using gfid-access with
different UID/GID.

With this patch, Entry creation with different UID/GID is split into two
operations. Create Entry with UID:0 and GID:0 and then set UID/GID.

Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01
BUG: 1312762
Signed-off-by: Aravinda VK 
Reviewed-on:http://review.gluster.org/13542
Reviewed-on: http://review.gluster.org/13643
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: hard-link rename issues on changelog replay

2016-03-09T08:26:27+00:00

Problem:
LINK + RENAME changelog when replayed after worker restart causes stale
hard-links to persist since VFS returns success for RENAME if hard-links
point to same inode.

Solution:
Worker detects RENAME being issued on hard-links to the same inode and
unlinks the source file-name. Conditionally rename by verifying that the
source gfid matches with the on-disk gfid on the slave.

Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2
BUG: 1296175
Reviewed-on: http://review.gluster.org/13189
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13638
Smoke: Gluster Build System 
Tested-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

georep: avoid creating multiple entries with same gfid

2016-03-08T16:34:57+00:00

Problem:
CREATE + RENAME changelogs replayed by geo-replication cause
stale old-name entries with same gfid on slave nodes.
A gfid is a unique key in the file-system and should not be
assigned to multiple entries.

Solution:
Create entry on slave only if lstat(gfid) at aux-mount fails.
This applies to files as well as directories.

Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919
BUG: 1296208
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13316
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
(cherry picked from commit 87d93fac9fcc4b258b7eb432ac4151cdd043534f)
Reviewed-on: http://review.gluster.org/13571

geo-rep: Fix CPU utilization 100% with Popen

2016-03-08T16:32:39+00:00

Errorstore is maintained by Geo-rep to collect errors from
the child processes opened using Popen. If Popen.communicate
is used then it closes stderr. When stderr is not available
errorstore.tailer() will enter into infinite loop without gap.

With this patch, sleep time added when stderr of Child process
is already closed.

Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94
BUG: 1315582
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13637
Reviewed-on: http://review.gluster.org/13645
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

geo-rep: Handling Rsync/Tar errors efficiently

2016-03-08T09:59:17+00:00

Geo-rep processes Changelogs in Batch, if one file in batch
fails with rsync error that Changelog file is reprocessed multiple times.
After MAX_RETRY, it logs all the GFIDs from that batch as Skipped.

This patch addresses following issues,
1. When Rsync/Tar fails do not parse Changelog again for retry
2. When Rsync/Tar fails do not replay Entry operations, only retry
   rsync/tar for those GFIDs
3. Log Error in Rsync/Tar only in the last Retry
4. Do not log Skipped GFIDs since Rsync/Tar errors are logged for
   only failed files.
5. Changed Entry failures as Error instead of Warning

BUG: 1313309
Change-Id: Ie134ce2572693056ab9b9008cd8aa5b5d87f7975
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12856
Reviewed-by: Kotresh HR 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Venky Shankar 
(cherry picked from commit d136a789258e8f600e536717da156a242d8ed9a5)
Reviewed-on: http://review.gluster.org/13558

geo-rep: Handle ERANGE error during listxattr

2016-03-08T09:58:37+00:00

llistxattr in Geo-rep is two syscall instead of one

SIZE = llistxattr(PATH, &BUF, 0);
BUF = create_buf(SIZE);
_ = llistxattr(PATH, &BUF, SIZE);

So if any new xattrs added just after first call by any other worker,
second syscall will fail with ERANGE error.

Now Geo-rep sends BUF with large size(256*100) and gets value
with only one syscall. Raises OSError if fails with ERANGE error
even after sending large BUF.

Change-Id: I8ade4bbe9a0a8ea908ed9dedcd3f2ff4c6fe6f51
Signed-off-by: Aravinda VK 
BUG: 1313311
Reviewed-on: http://review.gluster.org/13106
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR 
(cherry picked from commit 656a0e64e5f465561ed29297421ec150de32f2a1)
Reviewed-on: http://review.gluster.org/13560

geo-rep: Handle hardlink in Tiering based volume

2016-03-08T03:13:20+00:00

Problem:
Hardlinks are synced as Sticky bit files to Slave in
a Tiering based volume.
In a Tiering based volume, cold tier is hashed subvolume
and geo-rep captures all namespace operations in cold tier.

While syncing a file and its corresponding hardlink, it is
recorded as MKNOD in cold tier(for both) and
We end up creating two different files in Slave.

Solution:
If MKNOD with Sticky bit set is present, record it as LINK.
This way it will create a HARDLINK if source file exists (on slave),
else it will create a new file.

This way, Slave can create Hardlink file itself (instead
of creating a new file) in case of hardlink.

Change-Id: Ic50dc6e64df9ed01799c30539a33daace0abe6d4
BUG: 1302979
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13281
Reviewed-on: http://review.gluster.org/13315
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

geo-rep: Symlink Rename issue

2016-01-04T06:35:38+00:00

If ENTRY creation failed for symlink in Slave and symlink
renamed in Master. If Source not exists to Rename in Slave
Geo-rep interprets as Create of Target file. Geo-rep sends blob
of regular file to create symlink instead of sending blob of
symlink.

With this patch, Geo-rep identifies symlink and sends respective
blob.

BUG: 1292697
Change-Id: If9351974d1945141a1d3abb838b7d0de7591e48e
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12917
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Milind Changire 
Tested-by: Milind Changire 
Tested-by: NetBSD Build System 
 (cherry picked from commit 3151194fad1aed3ab340b53aeeb4e8d131538d12)
Reviewed-on: http://review.gluster.org/12998
Reviewed-by: Venky Shankar

geo-rep: Fix syntax errors in GsyncdError

2016-01-04T06:35:10+00:00

%s was not replaced by actual values in GsyncdError

BUG: 1279644
Change-Id: I3c0a10f07383ca72844a46f930b4aa3d3c29f568
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12566
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
 (cherry picked from commit 74699ddd777f7e862991cf3afad91823d30e5b84)
Reviewed-on: http://review.gluster.org/12724
Reviewed-by: Milind Changire 
Reviewed-by: Venky Shankar

geo-rep: Fix getting subvol count

2016-01-04T06:07:05+00:00

Tiering doesn't support disperse volume as hot tier,
hence xml output doesn't give 'hotdisperseCount'.
Remove the usage of 'hotdisperseCount' in geo-rep
and return 0 instead.

BUG: 1293309
Change-Id: I3f50d21cb51db91e31faebf69af4f72360420b73
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13062
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13068