glusterfs-snapshot.git/geo-replication, branch gluster-snap

geo-rep: get new config value after config value reset

2014-01-23T17:45:30+00:00

When config.read is called it preserves the previous values
from the the previously opened config file.

For example
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read("defaults.conf")
config.read("preferences.conf")

When change in config file is identified it will open new
instance of config to avoid getting old config values.

Change-Id: Iec677e61ebd2c59c95aea94481f569d78bd913e4
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6747
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: Fix no error when slave size less than master size

2014-01-23T17:41:10+00:00

in df command output, default block-size is 1K, but BUFFER_SIZE is
in bytes, so verify will succeed even if slave size is less than
master size.

Added -B1 to df command(that is --block-size=1) to get all values in
bytes.

BUG: 1056518
Change-Id: I2384d2a6f2505d967e0538b95fbf111c15b26f30
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6746
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

geo-rep: optimizing update stime after directory synchronization

2014-01-23T05:02:38+00:00

Since xsync crawl generates new changelog when number of entries
reaches 8K or when directory is reached. If a directory has number
of files less than 8K then respective changelog file will have less
entries. Since xsync generated changelog files processed one after
the other, so syncjobs are underutilized. hence low bandwidth
utilization.

With this patch, changelog will be generated for 8K entries only,
but stime will be accumulated. Multiple dirs stime will be
updated together since the generated changelog will have entries
accross the dirs.

Change-Id: Ib0b40962a070f855f47f887d0840e412fb7928e1
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/6744
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

gsyncd / geo-rep: geo-replication fixes

2013-12-12T08:16:03+00:00

-> "threaded" hybrid crawl.
-> Enabling metatadata synchronization.
-> Handling EINVAL/ESTALE gracefully while syncing metadata.
-> Improvments to changelog crawl code.
-> Initial crawl changelog generation format.
-> No gsyncd restart when checkpoint updated.
-> Fix symlink handling in hybrid crawl.
-> Slave's xtime key is 'stime'.
-> tar+ssh as data synchronization.
-> Instead of 'raise', just log in warning level for xtime missing cases.
-> Fix for JSON object load failure
-> Get new config value after config value reset.
-> Skip already processed changelogs.
-> Saving status of each individual worker thread.
-> GFID fetch on slave for purges.
-> Add tar ssh keys and config options.
-> Fix nlink count when using backend.
-> Include "data" operation for hardlink.
-> Use changelog time prefix as slave's time.
-> Process changelogs in parallel.

Change-Id: I09fcbb2e2e418149a6d8435abd2ac6b2f015bb06
BUG: 1036539
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6404
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

geo-rep: Increasing the buffer_size for master-salve size comparision.

2013-10-24T21:23:04+00:00

The buffer size is increased to 100MB, considering the space required by slave
filesystem bookkeeping.

Change-Id: Ib296f0e03b4d7a2de6d5ff0ae9ab09a5261e3e7c
BUG: 1020154
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6102
Reviewed-by: Avra Sengupta 
Tested-by: Gluster Build System

geo-rep: logrotate: Logrotate handling

2013-10-04T04:26:00+00:00

In existing georep logrotate was implemented by handling SIGSTOP
and SIGCONT, gsyncd was failing to start again after SIGSTOP.
New approach uses WatchedFileHandler in logging, which tracks the
log file changes or logrotate. Reopens the log file if logrotate is
triggered or if same log file is updated from other process.

As per python doc:
http://docs.python.org/2/library/logging.handlers.html:
The WatchedFileHandler class, located in the logging.handlers module,
is a FileHandler which watches the file it is logging to. If the file
changes, it is closed and reopened using the file name.

A file change can happen because of usage of programs such as newsyslog
and logrotate which perform log file rotation. This handler, intended
for use under Unix/Linux, watches the file to see if it has changed
since the last emit. (A file is deemed to have changed if its device
or inode have changed.) If the file has changed, the old file stream
is closed, and the file opened to get a new stream.

Change-Id: I30f65eb1e9778b12943d6e43b60a50344a7885c6
BUG: 1012776
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/5968
Reviewed-by: Amar Tumballi 
Reviewed-by: Harshavardhana 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

geo-rep: pack the value of 'readlink()' while crawling symlinks

2013-09-20T18:46:58+00:00

without having 'readlink()' value, symlink fails on slave mounts

Change-Id: Ib84be8db77c033029ba8ba454fd629a496fc3fe1
Signed-off-by: Amar Tumballi 
BUG: 847839
Reviewed-on: http://review.gluster.org/5950
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System

geo-replication: treat MKNOD also as regular file create

2013-09-20T18:46:48+00:00

Change-Id: Iec04f642282b554a4d1b5f5c8cdc099fd001b3f4
Original-Author: Venky Shankar 
Signed-off-by: Amar Tumballi 
BUG: 847839
Reviewed-on: http://review.gluster.org/5949
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System

geo-rep: create hardlink as hardlinks in xsync crawl

2013-09-20T18:46:31+00:00

BUG: 847839
Change-Id: Ieaa754f15611392c09afcc7190b9ff9da39d7fce
Signed-off-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5934
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System

geo-rep: retry in case of ENOENT errors in entry creations

2013-09-17T10:42:19+00:00

Change-Id: I8961633a7371c941a3feee44c949d5c934eca998
Original-Author: Venky Shankar 
Signed-off-by: Amar Tumballi 
BUG: 847839
Reviewed-on: http://review.gluster.org/5933
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar