glusterfs.git/geo-replication, branch v3.8rc1

geo-rep: Fix Entry Creation issue with non root UID/GID

2016-03-09T11:02:01+00:00

During entry_ops RENAME Geo-rep sends stat info along with the
recorded info from Changelog. In Slave side if Source file exists
Geo-rep renames to Target file by calling os.rename. If source file
does not exists, it tries to create Target file directly using available
stat info from Master. If UID and GID are different in Master for that
file then stat info will have different UID/GID during Create. Geo-rep
gets EACCES when it tries to create a new entry using gfid-access with
different UID/GID.

With this patch, Entry creation with different UID/GID is split into two
operations. Create Entry with UID:0 and GID:0 and then set UID/GID.

Change-Id: I4987e3a205d8513c06fa66198cde145a87003a01
BUG: 1313303
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13542
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: hard-link rename issues on changelog replay

2016-03-09T08:25:15+00:00

Problem:
LINK + RENAME changelog when replayed after worker restart causes stale
hard-links to persist since VFS returns success for RENAME if hard-links
point to same inode.

Solution:
Worker detects RENAME being issued on hard-links to the same inode and
unlinks the source file-name. Conditionally rename by verifying that the
source gfid matches with the on-disk gfid on the slave.

Change-Id: I3ff1c30ef79e77503c8b246d46dab8ac3059ccf2
BUG: 1296174
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13189
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

geo-rep: Fix CPU utilization 100% with Popen

2016-03-08T13:53:49+00:00

Errorstore is maintained by Geo-rep to collect errors from
the child processes opened using Popen. If Popen.communicate
is used then it closes stderr. When stderr is not available
errorstore.tailer() will enter into infinite loop without gap.

With this patch, sleep time added when stderr of Child process
is already closed.

Change-Id: Ic36aabd6de35b259467d0bab7952468432867a94
BUG: 1315601
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13637
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

georep: avoid creating multiple entries with same gfid

2016-03-01T07:58:05+00:00

Problem:
CREATE + RENAME changelogs replayed by geo-replication cause
stale old-name entries with same gfid on slave nodes.
A gfid is a unique key in the file-system and should not be
assigned to multiple entries.

Solution:
Create entry on slave only if lstat(gfid) at aux-mount fails.
This applies to files as well as directories.

Change-Id: Ice3340f4ae1251c2dcef024a2388c4d33b5d4919
BUG: 1296206
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/13316
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

geo-rep: Fix invalid disk size in gverify.sh

2016-03-01T07:57:20+00:00

Problem:
        Due to the wrong interpretation of disk size, gverify.sh script
exits in between with error "integer expression expected".
This error can be observed, only if gverify.sh is run standalone.

Solution:
        Fix the interpretation of disk size.

Also, verify that master and slave's gluster version are same.

Change-Id: Id095dfffd07a93a535a380f617ad215230979870
BUG: 1305839
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/13407
Smoke: Gluster Build System 
Reviewed-by: Aravinda VK 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
CentOS-regression: Gluster Build System

geo-rep: Handling Rsync/Tar errors efficiently

2016-02-26T10:15:01+00:00

Geo-rep processes Changelogs in Batch, if one file in batch
fails with rsync error that Changelog file is reprocessed multiple times.
After MAX_RETRY, it logs all the GFIDs from that batch as Skipped.

This patch addresses following issues,
1. When Rsync/Tar fails do not parse Changelog again for retry
2. When Rsync/Tar fails do not replay Entry operations, only retry
   rsync/tar for those GFIDs
3. Log Error in Rsync/Tar only in the last Retry
4. Do not log Skipped GFIDs since Rsync/Tar errors are logged for
   only failed files.
5. Changed Entry failures as Error instead of Warning

BUG: 1287723
Change-Id: Ie134ce2572693056ab9b9008cd8aa5b5d87f7975
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/12856
Reviewed-by: Kotresh HR 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Venky Shankar

geo-rep: Handle ERANGE error during listxattr

2016-02-26T10:12:16+00:00

llistxattr in Geo-rep is two syscall instead of one

SIZE = llistxattr(PATH, &BUF, 0);
BUF = create_buf(SIZE);
_ = llistxattr(PATH, &BUF, SIZE);

So if any new xattrs added just after first call by any other worker,
second syscall will fail with ERANGE error.

Now Geo-rep sends BUF with large size(256*100) and gets value
with only one syscall. Raises OSError if fails with ERANGE error
even after sending large BUF.

Change-Id: I8ade4bbe9a0a8ea908ed9dedcd3f2ff4c6fe6f51
Signed-off-by: Aravinda VK 
BUG: 1294588
Reviewed-on: http://review.gluster.org/13106
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Handle hardlink in Tiering based volume

2016-02-26T09:44:14+00:00

Problem:
Hardlinks are synced as Sticky bit files to Slave in
a Tiering based volume.
In a Tiering based volume, cold tier is hashed subvolume
and geo-rep captures all namespace operations in cold tier.

While syncing a file and its corresponding hardlink, it is
recorded as MKNOD in cold tier(for both) and
We end up creating two different files in Slave.

Solution:
If MKNOD with Sticky bit set is present, record it as LINK.
This way it will create a HARDLINK if source file exists (on slave),
else it will create a new file.

This way, Slave can create Hardlink file itself (instead
of creating a new file) in case of hardlink.

Change-Id: Ic50dc6e64df9ed01799c30539a33daace0abe6d4
BUG: 1301032
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13281
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

geo-rep: Fix gsyncd prefix in gsec_create

2016-02-04T11:16:26+00:00

gsec_create script is generated after running ./configure

libexec dir was formed using $prefix/libexec, but in Debian based
distributions libexec dir is not present, instead they use lib
directory to store these scripts.

With this patch, full libexec path is fetched during ./configure.

BUG: 1162905
Change-Id: I9f47a38e6ab0027c7df6716136fbe0635e95a593
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13298
Smoke: Gluster Build System 
Tested-by: Raghavendra Talur 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY

geo-rep: Fix getting subvol count

2015-12-22T15:06:46+00:00

Tiering doesn't support disperse volume as hot tier,
hence xml output doesn't give 'hotdisperseCount'.
Remove the usage of 'hotdisperseCount' in geo-rep
and return 0 instead.

Change-Id: I736e29257de085a25e38eb02959caad3465ebcda
BUG: 1292084
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13062
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Vivek
Reviewed-by: Aravinda VK