| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(duplexpand): Ignore risk of overflow in computation of the realloc
object count, since we'd need a string with 2^62 space-separated tokens
to trigger that (that's w/64-bit size_t; w/32-bit it'd be easier, but
still improbable).
Change-Id: If4521afe7b46110742991dd0ee234284ef8970df
BUG: 846755
Signed-off-by: Jim Meyering <meyering@redhat.com>
Reviewed-on: http://review.gluster.com/3793
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Earlier fixes to 842330 changed the generic (*a, **kw) signature,
although that was not related to the issue.
We restore the generic signature as it was used for a reason
(proxy methods that do none or only algebraic transformations
on passed arguments idiomatically have generic signature, both
to serve as visual cue and agnosticism wrt. the inner API).
Change-Id: Ib609a3a58be53d78b7f1221a3c162c6aec8fd488
BUG: 842330
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I0ae81ab01418becba83e401ec36c6db5323945e8
BUG: 842330
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3725
Tested-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes two problems with recent changes to
Geo-Replication
First:
------
Recent changes to geo-replication relies on Rsync to tranfer
extended attributes. Essentially Rsync would invoke a listxattr()
and then getxattr() the set reutrned by listxattr() and finally
transfer it to the remote slave. Xattrs like security.selinux would
create problems as they are not allowed to be set explicitly (unless
there's a rule that allows this). So, to make Rsync behave sanely we
filter out all "*.selinux*" xattrs from listxattr() (which is getxattr()
with ->name as NULL).
Second:
-------
Python's "if {..} else {..}" shortcut ".. and .. or .." was misused here.
This is a straightforward fix by interchanging last two variables (classes
in this case). Also fix a typo in sendmark_regular() definition.
Change-Id: I097b5f5d88a36c7eef5560a78d4332948a545942
BUG: 842330
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3714
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I6c329b895178545d16b0cb9f01ad116f5342f752
BUG: 841855
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/3706
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I164a1d1dd5f15569afd6806834119a6844949df0
BUG: 841062
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3684
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that in said mode metadata synchronization is best effort:
rsync syncs metadata at last so if rsync is interrupted in between
xattr sync and metadata sync stages, then file will be considered
in sync
Change-Id: I1c75eab33b0a1000abf3ad36b2d484a89eeda1bd
BUG: 841062
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3683
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
- add two tunables for rsync: "rsync-options" and "rsync-ssh-options"
- always pass "--no-implied-dirs" to rsync
Change-Id: I3d67a4cba8cabd681edac80e6b1fb8ea322008bd
BUG: 841062
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3682
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
due to not using the proper Python keyword, errhandler thread
was possible to run into empty select
Signed-off-by: Csaba Henk <csaba@redhat.com>
BUG: 764678
Change-Id: I3c39e718e72545c27d50fd73aa6daf54062331b0
Reviewed-on: http://review.gluster.com/3560
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a command invoked by gsyncd fails, gsyncd makes a log
of what comes out on its stderr. So far the log indeterministically
broke lines at random places. Now put some effort into reconstructing
original lines and having a faithful log.
BUG: 764678
Change-Id: I16fcc75d3e0f624c10c71d9b37c937ca677087cc
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3561
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix typo: s/buf/*buf/ in test for failed *buf = realloc...
Spotted by coverity.
Change-Id: Ie1db59956e331c3691f4ed470a3fbaf2bd2e2055
BUG: 789278
Signed-off-by: Jim Meyering <meyering@redhat.com>
Reviewed-on: http://review.gluster.com/3669
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ic55c28c572c1878f8a223ae2fd25d301a7ef0c12
BUG: 789278
Signed-off-by: Jim Meyering <meyering@redhat.com>
Reviewed-on: http://review.gluster.com/3596
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I5d10eb92af753513eb5e5e3d7d2ad3ace57917d7
BUG: 789278
Signed-off-by: Jim Meyering <meyering@redhat.com>
Reviewed-on: http://review.gluster.com/3597
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are a number of nit-level issues throughout the source with
the use of localtime and ctime. While they apparently aren't causing
too many problems, apart from the one in bz 828058, they ought to be
fixed. Among the "real" problems that are fixed in this patch:
1) general localtime and ctime not MT-SAFE. There's a non-zero chance
that another thread calling localtime (or ctime) will over-write
the static data about to be used in another thread
2) localtime(& <64-bit-type>) or ctime(& <64-bit-type>) generally
not a problem on 64-bit or little-endian 32-bit. But even though
we probably have zero users on big-ending 32-bit platforms, it's
still incorrect.
3) multiple nested calls passed as params. Last one wins, i.e. over-
writes result of prior calls.
4) Inconsistent error handling. Most of these calls are for logging,
tracing, or dumping. I submit that if an error somehow occurs in
the call to localtime or ctime, the log/trace/dump still should
still occur.
5) Appliances should all have their clocks set to UTC, and all log
entries, traces, and dumps should use GMT.
6) fix strtok(), change to strtok_r()
Other things this patch fixes/changes (that aren't bugs per se):
1) Change "%Y-%m-%d %H:%M:%S" and similar to their equivalent shorthand,
e.g. "%F %T"
2) change sizeof(timestr) to sizeof timestr. sizeof is an operator,
not a function. You don't use i +(32), why use sizeof(<var>).
(And yes, you do use parens with sizeof(<type>).)
3) change 'char timestr[256]' to 'char timestr[32]' where appropriate.
Per-thread stack is limited. Time strings are never longer than ~20
characters, so why waste 220+ bytes on the stack?
Things this patch doesn't fix:
1) hodgepodge of %Y-%m-%d %H:%M:%S versus %Y/%m/%d-%H%M%S and other
variations. It's not clear to me whether this ever matters, not to
mention 3rd party log filtering tools may already rely on a
particular format. Still it would be nice to have a single manifest
constant and have every call to localtime/strftime consistently use
the same format.
Change-Id: I827cad7bf53e57b69c0173f67abe72884249c1a9
BUG: 832173
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.com/3568
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit is based on Venky Shankar <vshankar@>'s
original implementation. Let us first quote Venky's
description, then we summarize changes to his work.
------
First version of failover/failback.
Failback mechanism uses two exclusive modes:
* blind-sync
This mode works with xtime pairs (both master and slave) to
identify candidated to sync the original master from the slave
* wrapup-sync
This mode is similar to the normal working of gsyncd except
that orphaned entities in the gluster volume are not assigned
xtimes. This prevents un-necessary transfer of data for such
entities.
Modes can be enabled via:
gluster volume geo-replication M S config special_sync_mode blind
gluster volume geo-replication M S config special_sync_mode wrapup
To turn off the special modes (i.e. to revert to normal gsyncd behaviour) use:
gluster volume geo-replication colon-d0 192.168.1.2::colon-d config \!special_sync_mode
------
Code has been refactored to meet following goals:
- make checkpointing work with special sync modes
- move out sync mode related conditionals from the crawl
loop and make all decisions to be made at startup
time
- be intrusive to the crawl loop to smallest possible degree
(we will have to change/revisit it for other reasons,
and the complexity of that should not increase)
So, xtime parsing/updating/evaluation that's specific to
the certain special modes are represented as mixin classes;
basic operation logic is in an abstract base class.
On startup, special-sync-mode tunable is dynamically dispatched
to the corresponding mixin and the actual master class is
derived from the chosen mixin and the ABS.
Change-Id: Ic9b8448f31ad4239a8200dc689f7d713662a67de
BUG: 830497
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3541
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- gluster vol geo-rep M S conf checkpoint <LABEL|now>
sets a checkpoint with LABEL (the keyword "now" is special,
it's rendered to the label "as of <timestamp of current time>")
that's used to refer to the checkpoint in the sequel.
(Technically, gsyncd makes a note of the xtime of master's root
as of setting the checkpoint, called the "checkpoint target".)
- gluster vol geo-rep M S conf \!checkpoint
deletes the checkpoint.
- gluster vol geo-rep M S stat
if status is OK, and there is a checkpoint configured, the checkpoint
info is appended to status (either "not yet reached", or
"completed at <timestamp of completion>").
(Technically, the worker runs a thread that monitors / serializes /
verifies checkpoint status, and answers checkpoint status requests
through a UNIX socket; monitoring boils down to querying the xtime
of slave's root and comparing with the target.)
- gluster vol geo-rep M S conf log-file | xargs grep checkpoint
displays the checkpoint history. Set, delete and completion events
are logged properly.
Change-Id: I4398e0819f1504e6e496b4209e91a0e156e1a0f8
BUG: 826512
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3491
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Regarding issue of leftover ssh control dirs:
If master side worker is stuck in connection establishment
phase, have the monitor kill it softly (ie. first by SIGTERM,
to let it cleanup). This is trickier than sounds on first hearing,
because if worker is stuck in waiting for a RePCe answer
(in threading.Condition().wait()), then SIGTERM is ignored
(more precisely, Python holds it back for the wait and resends it to
itself when wait is over).
So instead of signalling the worker only, we send TERM to the
whole process group -- that brings down the ssh connection, which
wakes up the waiting worker, which then can cleanup. Only problem
is that monitor is also in the process group and it should not coomit
a suicide. That is taken care by setting up a one-time SIGTERM
handler in the monitor.
- Regarding slave gsyncd stuck in chdir:
Slave gsyncd is usually well behaved: if master does not send
keepalives, it takes care to exit. However, if a hang occurs
in early phase, when slave is to change to the gluster mountpoint,
no timeout is set up for that (and unlike on master side, neither
is there an external actor like the monitor to do that).
So, to manage this scenario, we do the chdir in a (supposedly)
short lived thread, and in the main thread we wait for the termination
of this thread. If that does not happen within the time limit, main
thread calls for cleanup and exit. (This logic explicitely takes the
appropriate action in the cases when chdir succeeds or when hangs;
but what about the remaining case, when chdir fails? Well in that case
the chdir thread's exception handler will put the process to
cleanup and exit route.)
Change-Id: I6ad6faa9c7b1c37084d171d1e1a756abaff9eba8
BUG: 786291
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3376
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This forces rsync to perform supposedly privileged operations on
unprivileged slaves (like chown(2)).
For consistent behavior (with gsyncd's "chown" RPC call that's
being used for symlinks and directories), we also pass
"--numeric-ids" to rsync.
Also took the chance to retire gsyncd's "--rsync-extra" option
which was there for debugging purposes (related to a resolved
issue).
Change-Id: I4ee4d0d3a8c4e0f6746d34d7722c8a567a67491c
BUG: 822121
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3426
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[This is a "forward port" of fafd5c17, http://review.gluster.com/2908]
The "finally" clause that was meant to cleanup after the
temp mount has not covered the case of getting signalled
(eg. by monitor, upon worker timing out).
So here we "outsource" the cleanup to an ephemeral child process.
Child calls setsid(2) so it won't be bothered by internal process
management. We use a pipe in between worker and the cleanup child;
when child sees the worker end getting closed, it performs the cleanup.
Worker end can get closed either because worker closes it (normal case),
or because worker has terminated (faulty case) -- thus as bonus, we get
a nice uniform handling with no need to differentiate between normal and
faulty cases.
The faulty case that was seen IRL -- ie., users of maintainance mounts
hang in chdir(2) to mount point -- can be simulated for testing purposes
by applying the following patch:
diff --git a/xlators/mount/fuse/src/fuse-bridge.c b/xlators/mount/fuse/src/fuse-bridge.c
index acd3c68..1ce5dc1 100644
--- a/xlators/mount/fuse/src/fuse-bridge.c
+++ b/xlators/mount/fuse/src/fuse-bridge.c
@@ -2918,7 +2918,7 @@ fuse_init (xlator_t *this, fuse_in_header_t *finh, void *msg)
if (fini->minor < 9)
*priv->msg0_len_p = sizeof(*finh) + FUSE_COMPAT_WRITE_IN_SIZE;
#endif
- ret = send_fuse_obj (this, finh, &fino);
+ ret = priv->client_pid_set ? 0 : send_fuse_obj (this, finh, &fino);
if (ret == 0)
gf_log ("glusterfs-fuse", GF_LOG_INFO,
"FUSE inited with protocol versions:"
Change-Id: I14bad56a60a7fa82d0104fa4b9a20f4e42a7186f
BUG: 786291
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3259
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some of the bugs to fix were found by the following stress-test:
make "glusterfs --client-pid=-1" exit immediately on slave
side.
Also fix eintr_wrap which should not "adopt" exceptions generated
by the wrapped call, by re-raising them as GsyncdError.
Change-Id: Ia0d39e0635975ebbbf98d86e1e26f3122e1ed6ff
BUG: 764678
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3258
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Don't dump stack, rather log the "glusterfs session went down" message.
If the aux glusterfs is already dead when we try to do some file
operation, we get a failure with ENOTCONN, which is already handled
as above. However, it's also possible that glusterfs dies while we
are in a syscall into it -- in that case we get ECONNABORTED, and
so far then we end up with an ugly stack strace. From now on we
take ECONNABORTAD as well into consideration.
Nb. wrt. testing: it's not easy to synthetically force the aux glusterfs
to end this way; for that we have to provoke gsyncd into intensive
synchronization. I succeeded in that with the following ruby oneliner:
ruby -rcgi -e '
Dir.chdir($*[0])
a=[]
Thread.new { loop { while a.size >= 100; File.delete a.shift; end; sleep 1 }}
loop { a<<CGI.escape(STDIN.read 10); open(a[-1], "w") {}}' MTPT < /dev/urandom
where the geo-rep master is mounted at MTPT. With this going on, deliver a
SIGKILL to the geo-rep session's aux glusterfs. (It is giving ECONNABORTED
non-deterministically, actually in the minority of cases.)
Change-Id: I24fd8d0295cdba91d8b994057a1255ca8e2d1a67
BUG: 764510
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3078
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
fix topy.
Change-Id: I84df3e850dd24d9e86713dfa401c603a84a81ca6
BUG: 763302
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3375
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
gsyncd wrapper was segfaulting as coverity fix freed
up pointer at wrong place (after it was reused)
Instead of the apporach of the original coverity fix
that added elaborate control flow to hunt down potential
leaks, here we move the code over to static allocations
in place of (the not really necessary) dynamic ones.
Change-Id: Ida3855ff4a4f4371b350d27f858f129ceed51785
BUG: 789278
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3345
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Last batch of NetBSD build fixes, makes 3.3.0qa40 operational.
Round 2: only include <sys/syslimits.h> for NetBSD
BUG: 764655
Change-Id: Icd7290f1e340675d763665a0d0c5f95bc14e0c55
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.com/3321
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ib8183d4b585465d05a7adf3a4ceae93ae1bded15
BUG: 764655
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.com/3238
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I5739e9c9ae6fe78a6defb640d630e5f918ac1295
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 789278
Reviewed-on: http://review.gluster.com/3266
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Syncing of certain files can fail naturally if changes happen on
master (eg. file gets deleted). Therefore logging an error is
misleading.
Change-Id: I7b54904e5ec7c85e4e0fa1e330123d2c44c78ac5
BUG: 764510
Reported-by: Vijaykumar Koppad <vkoppad@redhat.com>
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3113
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijaykumar <vkoppad@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Mostly to do with "-Werror=format-security" being buggy, but while we're
here we might as well fix some typos and such. Credit goes to Patrick
Matthäi <pmatthaei@debian.org> for pointing these out.
Change-Id: Ia32d1111d7c10b1f213df85d86b17a1326248ffd
BUG: 811387
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.com/3117
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
In order to randomize the walk of the file tree.
Change-Id: I9fc3b83d5804914a50faae8df7dbcfed2ba6f4b4
BUG: 809675
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/3079
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- function of actual flipping made static
- clean out references to particular namespaces from flipping logic
- namespaces involved in flipping defined at single location
- fix fnmatch(3) invocation with reversed pattern and string arguments
- instead of "user", use "system" to flip from, because latter is
free from supervision of the VFS layer (cf. attr(5))
Change-Id: I3cc5836fadcad5b237fd5c67d0dcaea63aee9164
BUG: 798716
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/2890
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I91fe16d7e5e4c21f138eab4ee0b9334aec40e41b
BUG: 765433
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/2838
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
slave configurable
It can be set through the connection-timeout tunable but we keep it hidden,
intended as a workaround for some special scenarios not for general use.
Change-Id: I31f9fa3873afa7babc2106ee34484123a01bdc57
BUG: 789078
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: http://review.gluster.com/2839
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I2fa543b4bd317e06ea621ae968300ffb7223a68a
BUG: 771787
Signed-off-by: Csaba Henk <csaba@gluster.com>
Reviewed-on: http://review.gluster.com/2580
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I2eef82faab3eed1189e3786a5dca296773e1caa0
BUG: 784498
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.com/2690
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Csaba Henk <csaba@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
This works around broken /dev/stderr on some systems.
Change-Id: I017b03082ff630c4a713ae74990e88b3fa20d0e1
BUG: 3686
Reviewed-on: http://review.gluster.com/560
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rotating geo-replication master/monitor log files from cli.
On invocation, the log file for a given master-slave session
is backed up with the current timestamp suffixed to the file
name and signal is sent to gsyncd to start logging to a new
log file.
Sample commands:
* Rotate log file for this <master>:<slave> session:
gluster volume geo-replication <master> <slave> log-rotate
* Rotate log files for all session for master volume <master>
gluster volume geo-replication <master> log-rotate
* Rotate log files for all sessions:
gluster volume geo-replication log-rotate
Change-Id: I75f641b4e082a04d5373c18583ca4a1d9651d27a
BUG: 3519
Reviewed-on: http://review.gluster.com/529
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Csaba Henk <csaba@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- space is disallowed to make rsync target unambigous for gsyncd wrapper
- *, ?, [ is disallowed so that we can tell away globs from urls
Nothing too bad would happen without these restrictions, but this way
gluster errs out early instead of producing some mystical error
further down on the way.
Change-Id: Idd4e68f7d91598a7a8e30ccbc6d395da570cdf2e
BUG: 3610
Reviewed-on: http://review.gluster.com/490
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I0d54cea72e4363eab85ade774cc918081d8036e9
BUG: 3610
Reviewed-on: http://review.gluster.com/489
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- gsyncd gets allow-network tunable which is expected to
hold a comma-separated list of IP network addresses
- for IP addess matching, bring in ipaddr module from Google
(http://code.google.com/p/ipaddr-py/, rev. trunk@225)
This will let users control master's access to slave's volumes
until we implement unprivileged geo-rep (delayed due to some
technical issues). It's also needed for the completeness of
our hardening efforts, as plain file slaves won't be able
to work with an unprivileged gsyncd.
Change-Id: I58431cba6592f8672e93ea89a5eef478905b00b9
BUG: 2825
Reviewed-on: http://review.gluster.com/488
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I2da62b34aa833b9a28728fa1db23951f28b7e538
BUG: 2825
Reviewed-on: http://review.gluster.com/462
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- require/perform rsync invocation with unprotected args
(so that target is revealed to gateway program)
- make use of some procfs wizardry to find gsyncd sibling
and match rsync target against its working directory
Change-Id: Iae1e39b0e61f22563c0f2a2e0605567e0d1902df
BUG: 2825
Reviewed-on: http://review.gluster.com/461
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this change, the suggested way of setting up a geo-sync
slave is to use an ssh key with gsyncd as a forced command
(see sshd(8)), or set gsyncd as shell. This prevents the master
in executing arbitrary commands on slave (a major security hole).
Detailed list the changes:
- All gsyncd invocations that are not done by glusterd are
considered unsafe and then we operate in so-called "restricted mode"
(see below)
- if we are invoked on purpose (ie. it's not the case that sshd forced
us to run as frontend of a remote-invoked command), we execute gsyncd.py
- if invoked by sshd as frontend command, we check the remote command
line and call the required utility if it's among the allowed ones
(rsyncd and gsyncd)
- with rsync, we check if invocation is server mode and some other
sanity measures
- with gsyncd, in restricted mode we enforce the usage of the glusterd
provided config file, and in python, we enforce operation in
server mode and some other sanity checks
Impact on using geo-rep the old way: remote file slave now also
requires a running glusterd (to pick up config from).
Missing: we not implemented check of the rsync target path.
The issue of master being able to modify arbitrary locations
is planned to be mitigated by using geo-rep with an unprivileged
user.
Change-Id: I9b5825bfe282a9ca777429aadd554d78708f1638
BUG: 2825
Reviewed-on: http://review.gluster.com/460
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This rewrite does not change functionality;
it's purpose is to prepare followup modifications which will let
all slave side helper programs being dispatched to through
gsyncd. The string processing that's required for that task would
be too much cumbersome in shell.
Change-Id: Ia7858aba5efeb5dcff16a918ea1c02253f0e49ab
BUG: 2825
Reviewed-on: http://review.gluster.com/459
Reviewed-by: Amar Tumballi <amar@gluster.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When this option is set, a file deleted on master will not trigger
a delete operation on the slave. Hence, the slave will remain as a
superset of the master and can be used to recover the master in case
of crash and/or accidental deletes.
This options is not enabled by default.
Change-Id: I9244d9dfa4f38f19436036f36bec0d9c3a1f7993
BUG: 3552
Reviewed-on: http://review.gluster.com/426
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Csaba Henk <csaba@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
gsyncd:
- mounting code is split to a direct and a mountbroker based backend
- option gluster-command gone
- new options: gluster-params, gluster-cli-options, mountbroker
- mountbroker mount backend is used if either a mountbroker label
is given through the mountbroker option, or if gsyncd is
unprivileged; in this case the username is used as label
- have gluster cli invocations log to stderr so that we don't
hit a permission issue with the logfiles
glusterd:
- do gsyncd pre-config with new options
- add option geo-replication-log-group, so if that specified
geo-rep logfile directories are given to that group (and
thus members of the given group can do logging there)
This is just WIP as geo-rep relies on trusted extended attributes
and those are not accessible for unprivileged users. Even if we
solved this issue, glusterd security settings are too coarse,
so that if we made it possible for an unprivileged gsyncd
to operate, we would open up too far.
Change-Id: Icd520b58cbadccea3fad7c0f437b99de1e22db14
BUG: 2825
Reviewed-on: http://review.gluster.com/399
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Also add __codecheck script which can verify if source is OK at the
syntactical level with a given Python interpreter.
Change-Id: Ieff34bcd3efd1cdc0e8f9a510c05488f35897bbe
BUG: 1570
Reviewed-on: http://review.gluster.com/320
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I559e6a0709b8064cfd54c693e289c741f9c4c4ab
BUG: 1570
Reviewed-on: http://review.gluster.com/319
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use subprocess module instead of os.spawn* / ad-hoc fork/exec.
With this, we do now:
- close uneeded files in children
- watch childrens' stderr:
- have a thread which collects childrens' stderr into a ring buffer
(so that stderr pipe doesn't get stuffed)
- on command failure show stderr
- distinguish between rsync exit values, tolerate only partial errors
- if connection is broken to slave, show ssh/slave gsycd's stderr
Change-Id: Ia92f57b5bdfa47f8c44375c50cf279006a0bf69b
BUG: 2946
Reviewed-on: http://review.gluster.com/85
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Kaushik BV <kaushikbv@gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
- exceptions raised by us will be logged as single-line error messages
(full stack strace is shown only at DEBUG loglevel)
- common/well understood exceptions are mapped to "user-parsable" error logs
Change-Id: I75f1fb848483372364b2093878d9cfed576c9739
BUG: 2778
Reviewed-on: http://review.gluster.com/125
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I5c5211858bdb2bd28324818362d95edd97f94207
BUG: 2778
Reviewed-on: http://review.gluster.com/81
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushik BV <kaushikbv@gluster.com>
|