glusterfs.git/geo-replication/syncdaemon/resource.py, branch v3.6.2

geo-rep: gid is not set in entry ops

2014-11-13T10:12:01+00:00

uid is sent in place of gid while CREATE and
MKDIR.

BUG: 1159213
Change-Id: I711f383cef9de7467df9e60774d880eb5e1a642a
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7984
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9027

geo-rep: Fix rename of directory syncing.

2014-09-30T06:33:11+00:00

The rename of directories are captured in all distributed
brick changelogs. gsyncd processess these changelogs on
each brick parallellaly. The first changelog to get processed
will be successful. All subsequent ones will stat the 'src'
and if not present, tries to create freshly on slave. It
should be done only for files and not for directories.
Hence when this code path was hit, regular file's blob
is sent as directory's blob and gfid-access translator
was erroring out as 'Invalid blob length' with errno as
'ENOMEM'

Change-Id: I50545b02b98846464876795159d2446340155c82
BUG: 1147422
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/8865
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8880
Reviewed-by: Vijay Bellur

geo-rep: fix same file different gfid in master and slave

2014-09-30T06:32:57+00:00

While processing RENAME in changelog, if the file is unlinked
in master, then geo-rep was sending UNLINK to slave instead of
RENAME.

If rsync job fails if one of the file failed to sync in the job.
This patch adds logic to remove GFID from data list if the same
changelog has UNLINK entry for it after the DATA. Or it removes
those GFIDs during retry of changelogs processing.

BUG: 1147420
Change-Id: I982dc976397cd0ab676bb912583f66a28f821926
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8761
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Kotresh HR 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8879
Reviewed-by: Vijay Bellur

geo-rep: Fixing issue with xsync upper limit

2014-09-08T19:04:23+00:00

While identifying the file/dir to sync, xtime of the file was compared
with xsync_upper_limit as `xtime < xsync_upper_limit` After the sync,
xtime of parent directory is updated as stime. With the upper limit
condition, stime is updated as MIN(xtime_parent, xsync_upper_limit)

With this files will get missed if `xtime_of_file == xsync_upper_limit`

With this patch xtime_of_file is compared as
xtime_of_file <= xsync_upper_limit

BUG: 1138952
Change-Id: I469e8638ab6923e518022a539a19e2d040b60eb0
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8439
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8644

geo-rep: Handle RMDIR recursively

2014-09-08T19:03:20+00:00

If RMDIR is recorded in brick changelog which is due to
self heal traffic then it will not have UNLINK entries for
child files. Geo-rep hangs with ENOTEMPTY error on slave.

Now geo-rep recursively deletes the dir if it gets ENOTEMPTY.

BUG: 1138952
Change-Id: Ie79db90c52103b39fa795bb8a096b363d450b427
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8477
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8643
Reviewed-by: Vijay Bellur

geo-rep: minimize xsync crawl usage and set upper limit to xsync crawl

2014-09-08T17:41:59+00:00

For effective handling of deletes and renames use history crawl
as much as possible. History crawl will run in loop till it syncs
all data before live changelog time.

When it uses xsync crawl(fallback when changelog not available, or
very first crawl) it sets upper limit to crawl.

After completing History crawl, it checks actual end time returned
by history api to compare with register time, if actual end is less
than register time then run history crawl one more time.

If first turn history processing time is less than the CHANGELOG
ROLLOVER TIME then sleep for the difference, After sleep if it is
guaranteed that rollover will happen and switches to live changelog
consumption without switching to xsync. This sleep is only when
history processing completed < CHANGELOG_ROLLOVER_TIME and sleep
only after the first turn, So will not affect the performance.

BUG: 1138952
Change-Id: Ida024211d312f60f0e8190805e7469b2165f00e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/8151
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8639
Reviewed-by: Vijay Bellur

feature/geo-rep: Fix to retain pause state of gsyncd on restart

2014-06-20T08:17:22+00:00

On soft reboot, geo-rep monitor is writing 'faulty' into status
file. It should not do it if previous state is paused as
glusterd depend on the state file on node restart.

Change-Id: Idd45abf13350b087371935f1b4f6e1a346433d27
BUG: 1101410
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8097
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

geo-rep: entry_ops errors handling

2014-06-12T06:52:52+00:00

Xattr.lsetxattr_l call will not raise OSError which
errno_wrap can handle, so we need to use Xattr.lsetxattr
to get proper OSError and errno_wrap ignores or retries
accordingly.

Change-Id: Ie0a777152ddbaf9ed80c977e4704974fec997bea
BUG: 1105083
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/7972
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

gsyncd / geo-rep: FSH recommended log locations

2014-06-10T16:55:55+00:00

Upgrading "working_dir" on the fly is a bit unclean yet
(though it works) as currently config upgrade does not
support "old" values to be expanded by using configuration
variables.

Change-Id: I44ed65c281f2e0ce3b6b467addc5c1c88ac674e7
BUG: 1077516
Signed-off-by: Venky Shankar 
Signed-off-by: Kotresh H R 
Signed-off-by: Aravinda VK 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/7375
Tested-by: Gluster Build System

feature/geo-rep: Fix to retain pause state of gsyncd on restart.

2014-06-06T05:58:26+00:00

A new gsyncd options '--pause-on-start' is introduced. When node
reboots, if the status is paused, gsyncd is started with this
option. After gsyncd spawns worker and agent, worker will send 
SIGSTOP to negative pid of monitor to enter pause mode.

Change-Id: I5aad82c9a9fc8c243f384940b77d25e26e520d6d
BUG: 1101410
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7885
Reviewed-by: Aravinda VK 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar