glusterfs.git/geo-replication, branch release-3.6

geo-rep: gid is not set in entry ops

2014-11-13T10:12:01+00:00

uid is sent in place of gid while CREATE and
MKDIR.

BUG: 1159213
Change-Id: I711f383cef9de7467df9e60774d880eb5e1a642a
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7984
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9027

geo-rep: Failover when a Slave node goes down

2014-11-13T06:22:53+00:00

When a slave node goes down, worker in master node
can connect to different slave node and resume the operation.

Existing georep was not checking the status of slave node before
worker restart. With this patch, geo-rep worker will check the
node status using `gluster volume status` when it goes faulty.

BUG: 1159209
Change-Id: I7ca2a66ff2a438435f297b7063313214c28f3d4b
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8921
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9025

feature/geo-rep: Fix skipped files logging.

2014-11-12T12:28:32+00:00

When changelog is failed to process even after max
retries, files are skipped but were not logging.
This patch addresses this issue to log the skipped
files.

BUG: 1159198
Change-Id: I1f3d62cb61a940ee5e10e5a79e08fa65e1c15e97
Reviewed-on: http://review.gluster.org/7945
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/9022
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Fix rename of directory syncing.

2014-09-30T06:33:11+00:00

The rename of directories are captured in all distributed
brick changelogs. gsyncd processess these changelogs on
each brick parallellaly. The first changelog to get processed
will be successful. All subsequent ones will stat the 'src'
and if not present, tries to create freshly on slave. It
should be done only for files and not for directories.
Hence when this code path was hit, regular file's blob
is sent as directory's blob and gfid-access translator
was erroring out as 'Invalid blob length' with errno as
'ENOMEM'

Change-Id: I50545b02b98846464876795159d2446340155c82
BUG: 1147422
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/8865
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8880
Reviewed-by: Vijay Bellur

geo-rep: fix same file different gfid in master and slave

2014-09-30T06:32:57+00:00

While processing RENAME in changelog, if the file is unlinked
in master, then geo-rep was sending UNLINK to slave instead of
RENAME.

If rsync job fails if one of the file failed to sync in the job.
This patch adds logic to remove GFID from data list if the same
changelog has UNLINK entry for it after the DATA. Or it removes
those GFIDs during retry of changelogs processing.

BUG: 1147420
Change-Id: I982dc976397cd0ab676bb912583f66a28f821926
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8761
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8879
Reviewed-by: Vijay Bellur

geo-rep: Fixing issue with xsync upper limit

2014-09-08T19:04:23+00:00

While identifying the file/dir to sync, xtime of the file was compared
with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync,
xtime of parent directory is updated as stime. With the upper limit
condition, stime is updated as MIN(xtime_parent, xsync_upper_limit)

With this files will get missed if `xtime_of_file == xsync_upper_limit`

With this patch xtime_of_file is compared as
xtime_of_file <= xsync_upper_limit

BUG: 1138952
Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8439
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8644

geo-rep: Handle RMDIR recursively

2014-09-08T19:03:20+00:00

If RMDIR is recorded in brick changelog which is due to
self heal traffic then it will not have UNLINK entries for
child files. Geo-rep hangs with ENOTEMPTY error on slave.

Now geo-rep recursively deletes the dir if it gets ENOTEMPTY.

BUG: 1138952
Change-Id: Ie79db90c52103b39fa795bb8a096b363d450b427
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8477
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8643
Reviewed-by: Vijay Bellur

geo-rep: minimize xsync crawl usage and set upper limit to xsync crawl

2014-09-08T17:41:59+00:00

For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.

When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.

After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.

If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.

BUG: 1138952
Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8639
Reviewed-by: Vijay Bellur

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-09-03T18:01:45+00:00

Backport from master branch - http://review.gluster.org/#/c/8246/

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System

geo-rep: Avoid duplicate stat in xsync changelog processing

2014-07-09T08:52:24+00:00

When A file/dir is identified for metadata sync, it was doing
duplicate stat to get the metadata to sync.

With this patch it avoids doing one additional stat call. Xsync
performance will improve.

rsync will copy files metadata, so no need to include for
processing.

BUG: 1111490

Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8124
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar