glusterfs.git/geo-replication/syncdaemon, branch v3.9dev

geo-rep: Fix Entry Creation issue with non root UID/GID

2016-03-09T11:02:01+00:00

During entry_ops RENAME Geo-rep sends stat info along with the
recorded info from Changelog. In Slave side if Source file exists
Geo-rep renames to Target file by calling os.rename. If source file
does not exists, it tries to create Target file directly using available
stat info from Master. If UID and GID are different in Master for that
file then stat info will have different UID/GID during Create. Geo-rep
gets EACCES when it tries to create a new entry using gfid-access with
different UID/GID.

With this patch, Entry creation with different UID/GID is split into two
operations. Create Entry with UID:0 and GID:0 and then set UID/GID.

Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01
BUG: 1313303
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13542
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: hard-link rename issues on changelog replay

2016-03-09T08:25:15+00:00

Problem:
LINK + RENAME changelog when replayed after worker restart causes stale
hard-links to persist since VFS returns success for RENAME if hard-links
point to same inode.

Solution:
Worker detects RENAME being issued on hard-links to the same inode and
unlinks the source file-name. Conditionally rename by verifying that the
source gfid matches with the on-disk gfid on the slave.

Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2
BUG: 1296174
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13189
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Fix CPU utilization 100% with Popen

2016-03-08T13:53:49+00:00

Errorstore is maintained by Geo-rep to collect errors from
the child processes opened using Popen. If Popen.communicate
is used then it closes stderr. When stderr is not available
errorstore.tailer() will enter into infinite loop without gap.

With this patch, sleep time added when stderr of Child process
is already closed.

Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94
BUG: 1315601
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13637
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

georep: avoid creating multiple entries with same gfid

2016-03-01T07:58:05+00:00

Problem:
CREATE + RENAME changelogs replayed by geo-replication cause
stale old-name entries with same gfid on slave nodes.
A gfid is a unique key in the file-system and should not be
assigned to multiple entries.

Solution:
Create entry on slave only if lstat(gfid) at aux-mount fails.
This applies to files as well as directories.

Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919
BUG: 1296206
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13316
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

geo-rep: Handling Rsync/Tar errors efficiently

2016-02-26T10:15:01+00:00

Geo-rep processes Changelogs in Batch, if one file in batch
fails with rsync error that Changelog file is reprocessed multiple times.
After MAX_RETRY, it logs all the GFIDs from that batch as Skipped.

This patch addresses following issues,
1. When Rsync/Tar fails do not parse Changelog again for retry
2. When Rsync/Tar fails do not replay Entry operations, only retry
   rsync/tar for those GFIDs
3. Log Error in Rsync/Tar only in the last Retry
4. Do not log Skipped GFIDs since Rsync/Tar errors are logged for
   only failed files.
5. Changed Entry failures as Error instead of Warning

BUG: 1287723
Change-Id: Ie134ce2572693056ab9b9008cd8aa5b5d87f7975
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12856
Reviewed-by: Kotresh HR 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Venky Shankar

geo-rep: Handle ERANGE error during listxattr

2016-02-26T10:12:16+00:00

llistxattr in Geo-rep is two syscall instead of one

SIZE = llistxattr(PATH, &BUF, 0);
BUF = create_buf(SIZE);
_ = llistxattr(PATH, &BUF, SIZE);

So if any new xattrs added just after first call by any other worker,
second syscall will fail with ERANGE error.

Now Geo-rep sends BUF with large size(256*100) and gets value
with only one syscall. Raises OSError if fails with ERANGE error
even after sending large BUF.

Change-Id: I8ade4bbe9a0a8ea908ed9dedcd3f2ff4c6fe6f51
Signed-off-by: Aravinda VK 
BUG: 1294588
Reviewed-on: http://review.gluster.org/13106
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Handle hardlink in Tiering based volume

2016-02-26T09:44:14+00:00

Problem:
Hardlinks are synced as Sticky bit files to Slave in
a Tiering based volume.
In a Tiering based volume, cold tier is hashed subvolume
and geo-rep captures all namespace operations in cold tier.

While syncing a file and its corresponding hardlink, it is
recorded as MKNOD in cold tier(for both) and
We end up creating two different files in Slave.

Solution:
If MKNOD with Sticky bit set is present, record it as LINK.
This way it will create a HARDLINK if source file exists (on slave),
else it will create a new file.

This way, Slave can create Hardlink file itself (instead
of creating a new file) in case of hardlink.

Change-Id: Ic50dc6e64df9ed01799c30539a33daace0abe6d4
BUG: 1301032
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13281
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

geo-rep: Fix getting subvol count

2015-12-22T15:06:46+00:00

Tiering doesn't support disperse volume as hot tier,
hence xml output doesn't give 'hotdisperseCount'.
Remove the usage of 'hotdisperseCount' in geo-rep
and return 0 instead.

Change-Id: I736e29257de085a25e38eb02959caad3465ebcda
BUG: 1292084
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13062
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vivek
Reviewed-by: Aravinda VK

geo-rep: Fix getting subvol number

2015-12-21T08:28:23+00:00

Fix getting subvol number if the volume
type is tier. If the volume type was tier,
the subvol number was calculated incorrectly
and hence few of workers didn't become ACTIVE
resulting in files not being replicated from
corresponding brick. This patch addresses
the same.

Change-Id: Ic10ad7f09a0fa91b4bf2aa361dea3bd48be74853
BUG: 1292084
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12994
Tested-by: NetBSD Build System 
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System

geo-rep: Symlink Rename issue

2015-12-18T05:46:35+00:00

If ENTRY creation failed for symlink in Slave and symlink
renamed in Master. If Source not exists to Rename in Slave
Geo-rep interprets as Create of Target file. Geo-rep sends blob
of regular file to create symlink instead of sending blob of
symlink.

With this patch, Geo-rep identifies symlink and sends respective
blob.

BUG: 1289859
Change-Id: If9351974d1945141a1d3abb838b7d0de7591e48e
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12917
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Milind Changire 
Tested-by: Milind Changire 
Tested-by: NetBSD Build System