glusterfs.git/geo-replication/syncdaemon, branch v3.6.1

geo-rep: Fix rename of directory syncing.

2014-09-30T06:33:11+00:00

The rename of directories are captured in all distributed
brick changelogs. gsyncd processess these changelogs on
each brick parallellaly. The first changelog to get processed
will be successful. All subsequent ones will stat the 'src'
and if not present, tries to create freshly on slave. It
should be done only for files and not for directories.
Hence when this code path was hit, regular file's blob
is sent as directory's blob and gfid-access translator
was erroring out as 'Invalid blob length' with errno as
'ENOMEM'

Change-Id: I50545b02b98846464876795159d2446340155c82
BUG: 1147422
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/8865
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8880
Reviewed-by: Vijay Bellur

geo-rep: fix same file different gfid in master and slave

2014-09-30T06:32:57+00:00

While processing RENAME in changelog, if the file is unlinked
in master, then geo-rep was sending UNLINK to slave instead of
RENAME.

If rsync job fails if one of the file failed to sync in the job.
This patch adds logic to remove GFID from data list if the same
changelog has UNLINK entry for it after the DATA. Or it removes
those GFIDs during retry of changelogs processing.

BUG: 1147420
Change-Id: I982dc976397cd0ab676bb912583f66a28f821926
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8761
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8879
Reviewed-by: Vijay Bellur

geo-rep: Fixing issue with xsync upper limit

2014-09-08T19:04:23+00:00

While identifying the file/dir to sync, xtime of the file was compared
with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync,
xtime of parent directory is updated as stime. With the upper limit
condition, stime is updated as MIN(xtime_parent, xsync_upper_limit)

With this files will get missed if `xtime_of_file == xsync_upper_limit`

With this patch xtime_of_file is compared as
xtime_of_file <= xsync_upper_limit

BUG: 1138952
Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8439
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8644

geo-rep: Handle RMDIR recursively

2014-09-08T19:03:20+00:00

If RMDIR is recorded in brick changelog which is due to
self heal traffic then it will not have UNLINK entries for
child files. Geo-rep hangs with ENOTEMPTY error on slave.

Now geo-rep recursively deletes the dir if it gets ENOTEMPTY.

BUG: 1138952
Change-Id: Ie79db90c52103b39fa795bb8a096b363d450b427
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8477
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8643
Reviewed-by: Vijay Bellur

geo-rep: minimize xsync crawl usage and set upper limit to xsync crawl

2014-09-08T17:41:59+00:00

For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.

When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.

After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.

If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.

BUG: 1138952
Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8639
Reviewed-by: Vijay Bellur

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-09-03T18:01:45+00:00

Backport from master branch - http://review.gluster.org/#/c/8246/

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System

geo-rep: Avoid duplicate stat in xsync changelog processing

2014-07-09T08:52:24+00:00

When A file/dir is identified for metadata sync, it was doing
duplicate stat to get the metadata to sync.

With this patch it avoids doing one additional stat call. Xsync
performance will improve.

rsync will copy files metadata, so no need to include for
processing.

BUG: 1111490

Change-Id: I79dad6375fa4742d9aaca7d9856993c184a744dc
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8124
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: History Change detector method select issue

2014-07-01T10:59:01+00:00

Geo-rep does history crawl even if change-detector is set
to xsync.

This patch fixes by taking priority to user configured
change detector.

BUG: 1113525
Change-Id: Ic6c34e187c9cb6608c9ef8a010ea07015ba60a80
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8183
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: Fix the fd leak in worker/agent spawn

2014-06-30T10:40:45+00:00

worker and agent uses pipe to communicate, if worker dies
for some reason agent should get EOF and terminate.

Each worker-agent spawning is done in thread, Due to race
if multiple workers in same node retain the pipe refs of
other workers. Hence agent will not get EOF even if
worker dies.

BUG: 1114003
Change-Id: I36b9709b9392299483606bd3ef1db764fa3f2bff
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8194
Tested-by: Justin Clift 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

feature/geo-rep: Fix to retain pause state of gsyncd on restart

2014-06-20T08:17:22+00:00

On soft reboot, geo-rep monitor is writing 'faulty' into status
file. It should not do it if previous state is paused as
glusterd depend on the state file on node restart.

Change-Id: Idd45abf13350b087371935f1b4f6e1a346433d27
BUG: 1101410
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8097
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar