| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ibd0faefecc15b6713eda28bc96794ae58aff45aa
BUG: 847839
Original Author: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/5133
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added code to display extra information when status command
is executed.
Information shown now are
1 Number of files synced
2 crawl time
3 total sync time
4 bytes synced
bytes synced is taken from rsync output .
--stats option of rsync gives extra infor
mation about the sync.In stats output there
is a field called Total transferred file
size which states the ammount of bytes synced .
This information is parsed from stdout output
using regular expressions.Bytes synced information
can be used to calculate throughput.
Change-Id: Id9bba9fff45ee7049bb8257c6fd918e5237e05b1
BUG: 947774
Signed-off-by: sarvotham s pai <spai@redhat.com>
Reviewed-on: http://review.gluster.org/4749
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
... in gsyncd python code. Indeed, use the configuration
mechanism to set it suitably from glusterd.
Change-Id: I9fe2088b14d28588d1e64fe892740cc5755b8365
BUG: 868877
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.org/4143
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Earlier fixes to 842330 changed the generic (*a, **kw) signature,
although that was not related to the issue.
We restore the generic signature as it was used for a reason
(proxy methods that do none or only algebraic transformations
on passed arguments idiomatically have generic signature, both
to serve as visual cue and agnosticism wrt. the inner API).
Change-Id: Ib609a3a58be53d78b7f1221a3c162c6aec8fd488
BUG: 842330
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I0ae81ab01418becba83e401ec36c6db5323945e8
BUG: 842330
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3725
Tested-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes two problems with recent changes to
Geo-Replication
First:
------
Recent changes to geo-replication relies on Rsync to tranfer
extended attributes. Essentially Rsync would invoke a listxattr()
and then getxattr() the set reutrned by listxattr() and finally
transfer it to the remote slave. Xattrs like security.selinux would
create problems as they are not allowed to be set explicitly (unless
there's a rule that allows this). So, to make Rsync behave sanely we
filter out all "*.selinux*" xattrs from listxattr() (which is getxattr()
with ->name as NULL).
Second:
-------
Python's "if {..} else {..}" shortcut ".. and .. or .." was misused here.
This is a straightforward fix by interchanging last two variables (classes
in this case). Also fix a typo in sendmark_regular() definition.
Change-Id: I097b5f5d88a36c7eef5560a78d4332948a545942
BUG: 842330
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3714
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I6c329b895178545d16b0cb9f01ad116f5342f752
BUG: 841855
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3706
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I164a1d1dd5f15569afd6806834119a6844949df0
BUG: 841062
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3684
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that in said mode metadata synchronization is best effort:
rsync syncs metadata at last so if rsync is interrupted in between
xattr sync and metadata sync stages, then file will be considered
in sync
Change-Id: I1c75eab33b0a1000abf3ad36b2d484a89eeda1bd
BUG: 841062
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3683
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit is based on Venky Shankar <vshankar@>'s
original implementation. Let us first quote Venky's
description, then we summarize changes to his work.
------
First version of failover/failback.
Failback mechanism uses two exclusive modes:
* blind-sync
This mode works with xtime pairs (both master and slave) to
identify candidated to sync the original master from the slave
* wrapup-sync
This mode is similar to the normal working of gsyncd except
that orphaned entities in the gluster volume are not assigned
xtimes. This prevents un-necessary transfer of data for such
entities.
Modes can be enabled via:
gluster volume geo-replication M S config special_sync_mode blind
gluster volume geo-replication M S config special_sync_mode wrapup
To turn off the special modes (i.e. to revert to normal gsyncd behaviour) use:
gluster volume geo-replication colon-d0 192.168.1.2::colon-d config \!special_sync_mode
------
Code has been refactored to meet following goals:
- make checkpointing work with special sync modes
- move out sync mode related conditionals from the crawl
loop and make all decisions to be made at startup
time
- be intrusive to the crawl loop to smallest possible degree
(we will have to change/revisit it for other reasons,
and the complexity of that should not increase)
So, xtime parsing/updating/evaluation that's specific to
the certain special modes are represented as mixin classes;
basic operation logic is in an abstract base class.
On startup, special-sync-mode tunable is dynamically dispatched
to the corresponding mixin and the actual master class is
derived from the chosen mixin and the ABS.
Change-Id: Ic9b8448f31ad4239a8200dc689f7d713662a67de
BUG: 830497
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3541
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- gluster vol geo-rep M S conf checkpoint <LABEL|now>
sets a checkpoint with LABEL (the keyword "now" is special,
it's rendered to the label "as of <timestamp of current time>")
that's used to refer to the checkpoint in the sequel.
(Technically, gsyncd makes a note of the xtime of master's root
as of setting the checkpoint, called the "checkpoint target".)
- gluster vol geo-rep M S conf \!checkpoint
deletes the checkpoint.
- gluster vol geo-rep M S stat
if status is OK, and there is a checkpoint configured, the checkpoint
info is appended to status (either "not yet reached", or
"completed at <timestamp of completion>").
(Technically, the worker runs a thread that monitors / serializes /
verifies checkpoint status, and answers checkpoint status requests
through a UNIX socket; monitoring boils down to querying the xtime
of slave's root and comparing with the target.)
- gluster vol geo-rep M S conf log-file | xargs grep checkpoint
displays the checkpoint history. Set, delete and completion events
are logged properly.
Change-Id: I4398e0819f1504e6e496b4209e91a0e156e1a0f8
BUG: 826512
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3491
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
fix topy.
Change-Id: I84df3e850dd24d9e86713dfa401c603a84a81ca6
BUG: 763302
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3375
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Syncing of certain files can fail naturally if changes happen on
master (eg. file gets deleted). Therefore logging an error is
misleading.
Change-Id: I7b54904e5ec7c85e4e0fa1e330123d2c44c78ac5
BUG: 764510
Reported-by: Vijaykumar Koppad <vkoppad@redhat.com>
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3113
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijaykumar <vkoppad@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
In order to randomize the walk of the file tree.
Change-Id: I9fc3b83d5804914a50faae8df7dbcfed2ba6f4b4
BUG: 809675
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3079
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I2fa543b4bd317e06ea621ae968300ffb7223a68a
BUG: 771787
Signed-off-by: Csaba Henk <csaba@gluster.com>
Reviewed-on: http://review.gluster.com/2580
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When this option is set, a file deleted on master will not trigger
a delete operation on the slave. Hence, the slave will remain as a
superset of the master and can be used to recover the master in case
of crash and/or accidental deletes.
This options is not enabled by default.
Change-Id: I9244d9dfa4f38f19436036f36bec0d9c3a1f7993
BUG: 3552
Reviewed-on: http://review.gluster.com/426
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Csaba Henk <csaba@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Also add __codecheck script which can verify if source is OK at the
syntactical level with a given Python interpreter.
Change-Id: Ieff34bcd3efd1cdc0e8f9a510c05488f35897bbe
BUG: 1570
Reviewed-on: http://review.gluster.com/320
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I559e6a0709b8064cfd54c693e289c741f9c4c4ab
BUG: 1570
Reviewed-on: http://review.gluster.com/319
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use subprocess module instead of os.spawn* / ad-hoc fork/exec.
With this, we do now:
- close uneeded files in children
- watch childrens' stderr:
- have a thread which collects childrens' stderr into a ring buffer
(so that stderr pipe doesn't get stuffed)
- on command failure show stderr
- distinguish between rsync exit values, tolerate only partial errors
- if connection is broken to slave, show ssh/slave gsycd's stderr
Change-Id: Ia92f57b5bdfa47f8c44375c50cf279006a0bf69b
BUG: 2946
Reviewed-on: http://review.gluster.com/85
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Kaushik BV <kaushikbv@gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
- exceptions raised by us will be logged as single-line error messages
(full stack strace is shown only at DEBUG loglevel)
- common/well understood exceptions are mapped to "user-parsable" error logs
Change-Id: I75f1fb848483372364b2093878d9cfed576c9739
BUG: 2778
Reviewed-on: http://review.gluster.com/125
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I19f6cf539289062b39e3c0f73fa88f4ff9488349
BUG: 2778
Reviewed-on: http://review.gluster.com/80
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
- detect faulty state early
- keep the feedback fd in gsyncd module
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2537 (gsync autorestart)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2537
|
|
|
|
|
|
|
|
| |
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2736 (gsyncd hangs if crash occurs in the non-main thread)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2736
|
|
|
|
|
|
|
|
| |
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2736 (gsyncd hangs if crash occurs in the non-main thread)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2736
|
|
|
|
|
|
|
|
| |
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2644 (no volume-mark, if the behaviour persists have to check if master gsyncd is running.)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2644
|
|
|
|
|
|
|
|
|
|
|
| |
- whitespace fix
- remove superfluous imports
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 1570 (geosync related changes)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=1570
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We checked for the event of a new master being adopted by:
- checking for state change in list of masters
- checking if our chosen master is a foreign one
and if these held, then wrote out the actually available master to config
However this was flaky: if
(1) native master changed
(2) there was already a foreign master adopted
(3) actually the master was not available
then we wrote out the None to config.
This could occur if:
- on syncd startup, native master just changes from None to the volume native
[implies (1)]
- foreign master adopted from earlier run via config [implies (2)]
- actually foreign master was not available [implies (3)]
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand Avati <avati@gluster.com>
BUG: 2670 (strange behaviour with intermediate-master and slave)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2670
|
|
|
|
|
|
|
|
| |
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2537 (gsync autorestart)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2537
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- expiry check of foreign volinfo moved back to GLUSTERServer,
so that under the hood we can removexattr the expired ones;
a nice side-effect is that we can use the same dict layout
for foreign and native volinfo (ie., foreign needs no
timeout field)
- get_volinfo() is renamed to get_sys_volinfo() and most of the logic
is stripped off of it (what remained there is the check
against foreign master ambiguity)
- volinfo transition logic is cut out to an almost purely functional
static method (only impurity is the exeption raised upon
forbidden volinfo change)
- ping renamed to keep-alive, as something called "ping" is
not supposed to have payload (yeah, keep-alive is a bit fishy
on this front too, but could not come up with better...)
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2535 (gsync cascading)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2535
|
|
|
|
|
|
|
|
| |
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2535 (gsync cascading)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2535
|
|
|
|
|
|
|
|
|
|
| |
This patch allows the slave of a gsyncd to be started as the master of another slave gsyncd.
Signed-off-by: Kaushik BV <kaushikbv@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 2535 (gsync cascading)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2535
|
|
|
|
|
| |
Signed-off-by: Kaushik BV <kaushikbv@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
|
|
|
|
|
|
|
|
| |
Signed-off-by: Kaushik BV <kaushikbv@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 1570 (geosync related changes)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=1570
|
|
|
|
|
|
|
|
|
|
| |
links/dirs
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 2318 (gsyncd.py :chmod on directory not working)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2318
|
|
Signed-off-by: Csaba Henk <csaba@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 2310 (georeplication)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2310
|