summaryrefslogtreecommitdiffstats
path: root/xlators/features/marker
Commit message (Collapse)AuthorAgeFilesLines
...
* geo-rep / gsyncd: convert ignore-deletes to a mixin tooCsaba Henk2012-07-191-3/+16
| | | | | | | | | Change-Id: I164a1d1dd5f15569afd6806834119a6844949df0 BUG: 841062 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3684 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep / gsyncd: add support for sending xtimes through rsyncCsaba Henk2012-07-193-14/+35
| | | | | | | | | | | | | | Note that in said mode metadata synchronization is best effort: rsync syncs metadata at last so if rsync is interrupted in between xattr sync and metadata sync stages, then file will be considered in sync Change-Id: I1c75eab33b0a1000abf3ad36b2d484a89eeda1bd BUG: 841062 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep / gsyncd: rsync option cleanups, fixesCsaba Henk2012-07-182-2/+6
| | | | | | | | | | | | - add two tunables for rsync: "rsync-options" and "rsync-ssh-options" - always pass "--no-implied-dirs" to rsync Change-Id: I3d67a4cba8cabd681edac80e6b1fb8ea322008bd BUG: 841062 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* geo-rep / gsyncd: fixes to communication with child processesCsaba Henk2012-07-141-7/+11
| | | | | | | | | | | | due to not using the proper Python keyword, errhandler thread was possible to run into empty select Signed-off-by: Csaba Henk <csaba@redhat.com> BUG: 764678 Change-Id: I3c39e718e72545c27d50fd73aa6daf54062331b0 Reviewed-on: http://review.gluster.com/3560 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep / gsyncd: sanitize error log of external commandsCsaba Henk2012-07-141-2/+10
| | | | | | | | | | | | | | If a command invoked by gsyncd fails, gsyncd makes a log of what comes out on its stderr. So far the log indeterministically broke lines at random places. Now put some effort into reconstructing original lines and having a faithful log. BUG: 764678 Change-Id: I16fcc75d3e0f624c10c71d9b37c937ca677087cc Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3561 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gsyncd: don't dereference NULL upon failed reallocJim Meyering2012-07-131-1/+1
| | | | | | | | | | | | Fix typo: s/buf/*buf/ in test for failed *buf = realloc... Spotted by coverity. Change-Id: Ie1db59956e331c3691f4ed470a3fbaf2bd2e2055 BUG: 789278 Signed-off-by: Jim Meyering <meyering@redhat.com> Reviewed-on: http://review.gluster.com/3669 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* remove useless if-before-free (and free-like) functionsJim Meyering2012-07-131-8/+4
| | | | | | | | | | | | See comments in http://bugzilla.redhat.com/839925 for the code to perform this change. Signed-off-by: Jim Meyering <meyering@redhat.com> BUG: 839925 Change-Id: I10e4ecff16c3749fe17c2831c516737e08a3205a Reviewed-on: http://review.gluster.com/3661 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gsyncd: don't let a bogus config file make us read/write buf[-1]Jim Meyering2012-07-111-2/+4
| | | | | | | | | Change-Id: Ic55c28c572c1878f8a223ae2fd25d301a7ef0c12 BUG: 789278 Signed-off-by: Jim Meyering <meyering@redhat.com> Reviewed-on: http://review.gluster.com/3596 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* pidinfo: don't let a corrupt /proc/PID/status cause buf[-1] accessJim Meyering2012-07-021-2/+4
| | | | | | | | | Change-Id: I5d10eb92af753513eb5e5e3d7d2ad3ace57917d7 BUG: 789278 Signed-off-by: Jim Meyering <meyering@redhat.com> Reviewed-on: http://review.gluster.com/3597 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* localtime and ctime are not MT-SAFEKaleb S. KEITHLEY2012-06-291-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are a number of nit-level issues throughout the source with the use of localtime and ctime. While they apparently aren't causing too many problems, apart from the one in bz 828058, they ought to be fixed. Among the "real" problems that are fixed in this patch: 1) general localtime and ctime not MT-SAFE. There's a non-zero chance that another thread calling localtime (or ctime) will over-write the static data about to be used in another thread 2) localtime(& <64-bit-type>) or ctime(& <64-bit-type>) generally not a problem on 64-bit or little-endian 32-bit. But even though we probably have zero users on big-ending 32-bit platforms, it's still incorrect. 3) multiple nested calls passed as params. Last one wins, i.e. over- writes result of prior calls. 4) Inconsistent error handling. Most of these calls are for logging, tracing, or dumping. I submit that if an error somehow occurs in the call to localtime or ctime, the log/trace/dump still should still occur. 5) Appliances should all have their clocks set to UTC, and all log entries, traces, and dumps should use GMT. 6) fix strtok(), change to strtok_r() Other things this patch fixes/changes (that aren't bugs per se): 1) Change "%Y-%m-%d %H:%M:%S" and similar to their equivalent shorthand, e.g. "%F %T" 2) change sizeof(timestr) to sizeof timestr. sizeof is an operator, not a function. You don't use i +(32), why use sizeof(<var>). (And yes, you do use parens with sizeof(<type>).) 3) change 'char timestr[256]' to 'char timestr[32]' where appropriate. Per-thread stack is limited. Time strings are never longer than ~20 characters, so why waste 220+ bytes on the stack? Things this patch doesn't fix: 1) hodgepodge of %Y-%m-%d %H:%M:%S versus %Y/%m/%d-%H%M%S and other variations. It's not clear to me whether this ever matters, not to mention 3rd party log filtering tools may already rely on a particular format. Still it would be nice to have a single manifest constant and have every call to localtime/strftime consistently use the same format. Change-Id: I827cad7bf53e57b69c0173f67abe72884249c1a9 BUG: 832173 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3568 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: use buf->ia_gfid in all the lookup callbacksRaghavendra Bhat2012-06-192-10/+23
| | | | | | | | | | | | | | | | | * In general use buf->ia_gfid for gfid instead of inode's gfid in the callbacks of the fops where new inode is created (such as create, mkdir, mknod, symlink). In the callback path inode would not be having the gfid within it, if it is not yet linked to the inode table which happens in protocol/server. Change-Id: I75b348ad152a1bcd634a4c2db34aca97956ccb21 BUG: 822067 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.com/3567 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pranithk@gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gsyncd / geo-rep : failover/failbackCsaba Henk2012-06-133-66/+341
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit is based on Venky Shankar <vshankar@>'s original implementation. Let us first quote Venky's description, then we summarize changes to his work. ------ First version of failover/failback. Failback mechanism uses two exclusive modes: * blind-sync This mode works with xtime pairs (both master and slave) to identify candidated to sync the original master from the slave * wrapup-sync This mode is similar to the normal working of gsyncd except that orphaned entities in the gluster volume are not assigned xtimes. This prevents un-necessary transfer of data for such entities. Modes can be enabled via: gluster volume geo-replication M S config special_sync_mode blind gluster volume geo-replication M S config special_sync_mode wrapup To turn off the special modes (i.e. to revert to normal gsyncd behaviour) use: gluster volume geo-replication colon-d0 192.168.1.2::colon-d config \!special_sync_mode ------ Code has been refactored to meet following goals: - make checkpointing work with special sync modes - move out sync mode related conditionals from the crawl loop and make all decisions to be made at startup time - be intrusive to the crawl loop to smallest possible degree (we will have to change/revisit it for other reasons, and the complexity of that should not increase) So, xtime parsing/updating/evaluation that's specific to the certain special modes are represented as mixin classes; basic operation logic is in an abstract base class. On startup, special-sync-mode tunable is dynamically dispatched to the corresponding mixin and the actual master class is derived from the chosen mixin and the ABS. Change-Id: Ic9b8448f31ad4239a8200dc689f7d713662a67de BUG: 830497 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3541 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep: checkpointingCsaba Henk2012-06-133-19/+172
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - gluster vol geo-rep M S conf checkpoint <LABEL|now> sets a checkpoint with LABEL (the keyword "now" is special, it's rendered to the label "as of <timestamp of current time>") that's used to refer to the checkpoint in the sequel. (Technically, gsyncd makes a note of the xtime of master's root as of setting the checkpoint, called the "checkpoint target".) - gluster vol geo-rep M S conf \!checkpoint deletes the checkpoint. - gluster vol geo-rep M S stat if status is OK, and there is a checkpoint configured, the checkpoint info is appended to status (either "not yet reached", or "completed at <timestamp of completion>"). (Technically, the worker runs a thread that monitors / serializes / verifies checkpoint status, and answers checkpoint status requests through a UNIX socket; monitoring boils down to querying the xtime of slave's root and comparing with the target.) - gluster vol geo-rep M S conf log-file | xargs grep checkpoint displays the checkpoint history. Set, delete and completion events are logged properly. Change-Id: I4398e0819f1504e6e496b4209e91a0e156e1a0f8 BUG: 826512 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3491 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep / gsyncd: further cleanup refinementsCsaba Henk2012-05-244-5/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Regarding issue of leftover ssh control dirs: If master side worker is stuck in connection establishment phase, have the monitor kill it softly (ie. first by SIGTERM, to let it cleanup). This is trickier than sounds on first hearing, because if worker is stuck in waiting for a RePCe answer (in threading.Condition().wait()), then SIGTERM is ignored (more precisely, Python holds it back for the wait and resends it to itself when wait is over). So instead of signalling the worker only, we send TERM to the whole process group -- that brings down the ssh connection, which wakes up the waiting worker, which then can cleanup. Only problem is that monitor is also in the process group and it should not coomit a suicide. That is taken care by setting up a one-time SIGTERM handler in the monitor. - Regarding slave gsyncd stuck in chdir: Slave gsyncd is usually well behaved: if master does not send keepalives, it takes care to exit. However, if a hang occurs in early phase, when slave is to change to the gluster mountpoint, no timeout is set up for that (and unlike on master side, neither is there an external actor like the monitor to do that). So, to manage this scenario, we do the chdir in a (supposedly) short lived thread, and in the main thread we wait for the termination of this thread. If that does not happen within the time limit, main thread calls for cleanup and exit. (This logic explicitely takes the appropriate action in the cases when chdir succeeds or when hangs; but what about the remaining case, when chdir fails? Well in that case the chdir thread's exception handler will put the process to cleanup and exit route.) Change-Id: I6ad6faa9c7b1c37084d171d1e1a756abaff9eba8 BUG: 786291 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3376 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep / gsyncd: add "--super" to rsync invocationCsaba Henk2012-05-242-2/+1
| | | | | | | | | | | | | | | | | | | | This forces rsync to perform supposedly privileged operations on unprivileged slaves (like chown(2)). For consistent behavior (with gsyncd's "chown" RPC call that's being used for symlinks and directories), we also pass "--numeric-ids" to rsync. Also took the chance to retire gsyncd's "--rsync-extra" option which was there for debugging purposes (related to a resolved issue). Change-Id: I4ee4d0d3a8c4e0f6746d34d7722c8a567a67491c BUG: 822121 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3426 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep / gsyncd: fix cleanup of temporary mountsCsaba Henk2012-05-211-17/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [This is a "forward port" of fafd5c17, http://review.gluster.com/2908] The "finally" clause that was meant to cleanup after the temp mount has not covered the case of getting signalled (eg. by monitor, upon worker timing out). So here we "outsource" the cleanup to an ephemeral child process. Child calls setsid(2) so it won't be bothered by internal process management. We use a pipe in between worker and the cleanup child; when child sees the worker end getting closed, it performs the cleanup. Worker end can get closed either because worker closes it (normal case), or because worker has terminated (faulty case) -- thus as bonus, we get a nice uniform handling with no need to differentiate between normal and faulty cases. The faulty case that was seen IRL -- ie., users of maintainance mounts hang in chdir(2) to mount point -- can be simulated for testing purposes by applying the following patch: diff --git a/xlators/mount/fuse/src/fuse-bridge.c b/xlators/mount/fuse/src/fuse-bridge.c index acd3c68..1ce5dc1 100644 --- a/xlators/mount/fuse/src/fuse-bridge.c +++ b/xlators/mount/fuse/src/fuse-bridge.c @@ -2918,7 +2918,7 @@ fuse_init (xlator_t *this, fuse_in_header_t *finh, void *msg) if (fini->minor < 9) *priv->msg0_len_p = sizeof(*finh) + FUSE_COMPAT_WRITE_IN_SIZE; #endif - ret = send_fuse_obj (this, finh, &fino); + ret = priv->client_pid_set ? 0 : send_fuse_obj (this, finh, &fino); if (ret == 0) gf_log ("glusterfs-fuse", GF_LOG_INFO, "FUSE inited with protocol versions:" Change-Id: I14bad56a60a7fa82d0104fa4b9a20f4e42a7186f BUG: 786291 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3259 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep / gsyncd: fixes regarding the command invocation frameworkCsaba Henk2012-05-192-9/+25
| | | | | | | | | | | | | | | | Some of the bugs to fix were found by the following stress-test: make "glusterfs --client-pid=-1" exit immediately on slave side. Also fix eintr_wrap which should not "adopt" exceptions generated by the wrapped call, by re-raising them as GsyncdError. Change-Id: Ia0d39e0635975ebbbf98d86e1e26f3122e1ed6ff BUG: 764678 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3258 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep / gsyncd: recognize ECONNABORTED as termination of aux glusterfsCsaba Henk2012-05-191-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Don't dump stack, rather log the "glusterfs session went down" message. If the aux glusterfs is already dead when we try to do some file operation, we get a failure with ENOTCONN, which is already handled as above. However, it's also possible that glusterfs dies while we are in a syscall into it -- in that case we get ECONNABORTED, and so far then we end up with an ugly stack strace. From now on we take ECONNABORTAD as well into consideration. Nb. wrt. testing: it's not easy to synthetically force the aux glusterfs to end this way; for that we have to provoke gsyncd into intensive synchronization. I succeeded in that with the following ruby oneliner: ruby -rcgi -e ' Dir.chdir($*[0]) a=[] Thread.new { loop { while a.size >= 100; File.delete a.shift; end; sleep 1 }} loop { a<<CGI.escape(STDIN.read 10); open(a[-1], "w") {}}' MTPT < /dev/urandom where the geo-rep master is mounted at MTPT. With this going on, deliver a SIGKILL to the geo-rep session's aux glusterfs. (It is giving ECONNABORTED non-deterministically, actually in the minority of cases.) Change-Id: I24fd8d0295cdba91d8b994057a1255ca8e2d1a67 BUG: 764510 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3078 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* geo-rep / gsyncd: typo fixCsaba Henk2012-05-191-1/+1
| | | | | | | | | | | fix topy. Change-Id: I84df3e850dd24d9e86713dfa401c603a84a81ca6 BUG: 763302 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3375 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* geo-rep / gsyncd.c: fix coverity fixCsaba Henk2012-05-182-86/+36
| | | | | | | | | | | | | | | | | gsyncd wrapper was segfaulting as coverity fix freed up pointer at wrong place (after it was reused) Instead of the apporach of the original coverity fix that added elaborate control flow to hunt down potential leaks, here we move the code over to static allocations in place of (the not really necessary) dynamic ones. Change-Id: Ida3855ff4a4f4371b350d27f858f129ceed51785 BUG: 789278 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3345 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* NetBSD build fixesEmmanuel Dreyfus2012-05-151-0/+4
| | | | | | | | | | | | | Last batch of NetBSD build fixes, makes 3.3.0qa40 operational. Round 2: only include <sys/syslimits.h> for NetBSD BUG: 764655 Change-Id: Icd7290f1e340675d763665a0d0c5f95bc14e0c55 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.com/3321 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* features/marker: use the gfid from the stat structure instead of inodeRaghavendra Bhat2012-05-141-1/+6
| | | | | | | | | | | | | | | in fresh lookup, the inode would not have linked to the inode table until the fop reaches back to protocol/server, thus it would not contain the gfid within it (gfid would still be null). So use the stat structure to get the gfid in lookup callback instead of inode's gfid. Change-Id: Id70277f0228f3db64b05d613108cfb4f070197e6 BUG: 791087 Signed-off-by: Raghavendra Bhat <raghavendrabhat@gluster.com> Reviewed-on: http://review.gluster.com/3323 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* NetBSD build fixesEmmanuel Dreyfus2012-05-112-0/+3
| | | | | | | | | | | Change-Id: Ib8183d4b585465d05a7adf3a4ceae93ae1bded15 BUG: 764655 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.com/3238 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* geo-replication: coverity resource leak fixesAmar Tumballi2012-05-092-14/+64
| | | | | | | | | Change-Id: I5739e9c9ae6fe78a6defb640d630e5f918ac1295 Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 789278 Reviewed-on: http://review.gluster.com/3266 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: fix possible NULL access in strcmp()Amar Tumballi2012-04-161-3/+9
| | | | | | | | | | | | | for a given setxattr call, loc->path can be NULL now, handle that case properly Change-Id: Ic56ec81aa8eb62aafdb524bf22f6d84390ab1d4b Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 812767 Reviewed-on: http://review.gluster.com/3154 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* glusterd/rebalance: Start process with xlator option client-pid -3shishir gowda2012-04-161-1/+2
| | | | | | | | | | | | | This would prevent updation of xtime by the marker. Signed-off-by: shishir gowda <shishirng@gluster.com> Change-Id: I11e287511026326271c2926e55993e48b32018f6 BUG: 812287 Reviewed-on: http://review.gluster.com/3144 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Csaba Henk <csaba@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* geo-rep / gsyncd: log sync failures as warningsCsaba Henk2012-04-131-1/+1
| | | | | | | | | | | | | | | Syncing of certain files can fail naturally if changes happen on master (eg. file gets deleted). Therefore logging an error is misleading. Change-Id: I7b54904e5ec7c85e4e0fa1e330123d2c44c78ac5 BUG: 764510 Reported-by: Vijaykumar Koppad <vkoppad@redhat.com> Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3113 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijaykumar <vkoppad@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* Fix compiler warnings and typos from Debian build.Jeff Darcy2012-04-101-1/+1
| | | | | | | | | | | | | Mostly to do with "-Werror=format-security" being buggy, but while we're here we might as well fix some typos and such. Credit goes to Patrick Matthäi <pmatthaei@debian.org> for pointing these out. Change-Id: Ia32d1111d7c10b1f213df85d86b17a1326248ffd BUG: 811387 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.com/3117 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep / gsyncd: shuffle directory entries in crawlCsaba Henk2012-04-051-0/+2
| | | | | | | | | | | In order to randomize the walk of the file tree. Change-Id: I9fc3b83d5804914a50faae8df7dbcfed2ba6f4b4 BUG: 809675 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/3079 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* quota: log msgs indicating NULL parent inode in DEBUG level.Raghavendra G2012-04-031-2/+2
| | | | | | | | | | | | with name-less lookups its common to hit situations where parent inode is NULL. Signed-off-by: Raghavendra G <raghavendra@gluster.com> Change-Id: I980b14c3d4691e850ae15768f73cf1b7d867e930 BUG: 809032 Reviewed-on: http://review.gluster.com/3071 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* core: adding extra data for fopsAmar Tumballi2012-03-224-207/+230
| | | | | | | | | | | | | with this change, the xlator APIs will have a dictionary as extra argument, which is passed between all the layers. This can be utilized for overloading in some of the operations. Change-Id: I58a8186b3ef647650280e63f3e5e9b9de7827b40 Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 782265 Reviewed-on: http://review.gluster.com/2960 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: Replacing -1 with GF_CLIENT_PID_GSYNCD as part of code cleanup.Mohammed Junaid2012-03-201-1/+1
| | | | | | | | | Change-Id: Ia665060bff636b372904a726aba5249de9ade124 BUG: 769494 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.com/2971 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Csaba Henk <csaba@redhat.com>
* features/marker: Nameless lookup related fixesRaghavendra G2012-03-141-11/+13
| | | | | | | | | | | | while filling a loc structure, marker should not error out if loc->name is NULL or loc->path is just a gfid. Change-Id: Ie3a9cffde2d17da0377f1e41de93b099a9133abe BUG: 801364 Signed-off-by: Raghavendra G <raghavendra@gluster.com> Reviewed-on: http://review.gluster.com/2938 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cleanup and fix xattr namespace flipCsaba Henk2012-03-071-1/+1
| | | | | | | | | | | | | | | | - function of actual flipping made static - clean out references to particular namespaces from flipping logic - namespaces involved in flipping defined at single location - fix fnmatch(3) invocation with reversed pattern and string arguments - instead of "user", use "system" to flip from, because latter is free from supervision of the VFS layer (cf. attr(5)) Change-Id: I3cc5836fadcad5b237fd5c67d0dcaea63aee9164 BUG: 798716 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/2890 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/marker: GFID file handle based backend related fixesv3.3.0qa26Raghavendra G2012-03-072-2/+5
| | | | | | | | | | | | * make sure loc->gfid is filled. Change-Id: I5304e650d9fc181ee1f3b65be7f2ff3847d9722e BUG: 790389 Signed-off-by: Raghavendra G <raghavendra@gluster.com> Reviewed-on: http://review.gluster.com/2888 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendrabhat@gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* geo-rep / syncdaemon: determine suitable xattr namespace based on privilegeCsaba Henk2012-03-052-6/+7
| | | | | | | | | Change-Id: I91fe16d7e5e4c21f138eab4ee0b9334aec40e41b BUG: 765433 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/2838 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* geo-rep / syncdaemon: make the timeout for establishing the connection to ↵Csaba Henk2012-03-012-1/+2
| | | | | | | | | | | | | | slave configurable It can be set through the connection-timeout tunable but we keep it hidden, intended as a workaround for some special scenarios not for general use. Change-Id: I31f9fa3873afa7babc2106ee34484123a01bdc57 BUG: 789078 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.com/2839 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* features/marker: Use loc.inode to obtain parent if loc.parent isn't set.Vijay Bellur2012-02-221-1/+13
| | | | | | | | | Change-Id: I89dddb19168a3ffb269afc9f6bcb2d7e7c9a69cd BUG: 790389 Signed-off-by: Vijay Bellur <vijay@gluster.com> Reviewed-on: http://review.gluster.com/2802 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendrabhat@gluster.com>
* mempool: adjustments in pool sizesAmar Tumballi2012-02-221-1/+1
| | | | | | | | | | | | | | | | | * while creating 'rpc_clnt', the caller knows what would be the ideal load on it, so an extra argument to set some pool sizes * while creating 'rpcsvc', the caller knows what would be the ideal load of it, so an extra argument to set request pool size * cli memory footprint is reduced Change-Id: Ie245216525b450e3373ef55b654b4cd30741347f Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 765336 Reviewed-on: http://review.gluster.com/2784 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* core: utilize mempool for frame->local allocationsAmar Tumballi2012-02-216-55/+48
| | | | | | | | | | | | | | | in each translator, which uses 'frame->local', we are using GF_CALLOC/GF_FREE, which would be costly considering the number of allocation happening in a lifetime of 'fop'. It would be good to utilize the mem pool framework for xlator's local structures, so there is no allocation overhead. Change-Id: Ida6e65039a24d9c219b380aa1c3559f36046dc94 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 765336 Reviewed-on: http://review.gluster.com/2772 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* features/marker: copy the gfid to loc from oplocal->loc.inodev3.3.0qa22Raghavendra Bhat2012-02-141-0/+1
| | | | | | | | | Change-Id: I8cfb70c163a3c4a0bd682678f71b38b5bae20ab4 BUG: 790393 Signed-off-by: Raghavendra Bhat <raghavendrabhat@gluster.com> Reviewed-on: http://review.gluster.com/2750 Reviewed-by: Vijay Bellur <vijay@gluster.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/marker: set loc->gfid in setxattr initiated from marker.Vijay Bellur2012-02-141-6/+23
| | | | | | | | | Change-Id: I7abb62272b7c8bb25ad37cbc13da7b4713924b7a BUG: 787151 Signed-off-by: Vijay Bellur <vijay@gluster.com> Reviewed-on: http://review.gluster.com/2736 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* core: add an extra flag to readv()/writev() APIAmar Tumballi2012-02-141-2/+2
| | | | | | | | | | | | needed to implement a proper handling of open flag alterations using fcntl() on fd. Change-Id: Ic280d5db6f1dc0418d5c439abb8db1d3ac21ced0 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 782265 Reviewed-on: http://review.gluster.com/2723 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* geo-rep: gsyncd: fix up fallback xtime for orphans on master sideCsaba Henk2012-02-071-7/+8
| | | | | | | | | | Change-Id: I2fa543b4bd317e06ea621ae968300ffb7223a68a BUG: 771787 Signed-off-by: Csaba Henk <csaba@gluster.com> Reviewed-on: http://review.gluster.com/2580 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushik BV <kaushikbv@gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* features/marker,quota: Incorporate changes to support nameless lookupRaghavendra Bhat2012-02-023-1/+83
| | | | | | | | | | Change-Id: Ic5f00a9891bd835ebee5a3e103ef0f75d0b7fc25 BUG: 783925 Signed-off-by: Raghavendra Bhat <raghavendrabhat@gluster.com> Reviewed-on: http://review.gluster.com/2702 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amar@gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* geo-rep: gsyncd: Python3 compat fixesVenky Shankar2012-01-261-3/+4
| | | | | | | | | | Change-Id: I2eef82faab3eed1189e3786a5dca296773e1caa0 BUG: 784498 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.com/2690 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Csaba Henk <csaba@redhat.com>
* core: get xattrs also as part of readdirpAmar Tumballi2012-01-252-2/+46
| | | | | | | | | | | | | readdirp_req() call sends a dict_t * as an argument, which contains all the xattr keys for which the entries got in readdirp_rsp() are having xattr value filled dictionary. Change-Id: I8b7e1290740ea3e884e67d19156ce849227167c0 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 765785 Reviewed-on: http://review.gluster.com/771 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* core: change lk-owner as a 1k bufferAmar Tumballi2012-01-241-1/+1
| | | | | | | | | | | | | so, NLM can send the lk-owner field directly to the locks translators, while doing the same effort, also enabled sending maximum of 500 aux gid over protocol. Change-Id: I87c2514392748416f7ffe21d5154faad2e413969 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 767229 Reviewed-on: http://review.gluster.com/779 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* core: GFID filehandle based backend and anonymous FDsAnand Avati2012-01-201-14/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. What -------- This change introduces an infrastructure change in the filesystem which lets filesystem operation address objects (inodes) just by its GFID. Thus far GFID has been a unique identifier of a user-visible inode. But in terms of addressability the only mechanism thus far has been the backend filesystem path, which could be derived from the GFID only if it was cached in the inode table along with the entire set of dentry ancestry leading up to the root. This change essentially decouples addressability from the namespace. It is no more necessary to be aware of the parent directory to address a file or directory. 2. Why ------- The biggest use case for such a feature is NFS for generating persistent filehandles. So far the technique for generating filehandles in NFS has been to encode path components so that the appropriate inode_t can be repopulated into the inode table by means of a recursive lookup of each component top-down. Another use case is the ability to perform more intelligent self-healing and rebalancing of inodes with hardlinks and also to detect renames. A derived feature from GFID filehandles is anonymous FDs. An anonymous FD is an internal USABLE "fd_t" which does not map to a user opened file descriptor or to an internal ->open()'d fd. The ability to address a file by the GFID eliminates the need to have a persistent ->open()'d fd for the purpose of avoiding the namespace. This improves NFS read/write performance significantly eliminating open/close calls and also fixes some of today's limitations (like keeping an FD open longer than necessary resulting in disk space leakage) 3. How ------- At each storage/posix translator level, every file is hardlinked inside a hidden .glusterfs directory (under the top level export) with the name as the ascii-encoded standard UUID format string. For reasons of performance and scalability there is a two-tier classification of those hardlinks under directories with the initial parts of the UUID string as the directory names. For directories (which cannot be hardlinked), the approach is to use a symlink which dereferences the parent GFID path along with basename of the directory. The parent GFID dereference will in turn be a dereference of the grandparent with the parent's basename, and so on recursively up to the root export. 4. Development --------------- 4a. To leverage the ability to address an inode by its GFID, the technique is to perform a "nameless lookup". This means, to populate a loc_t structure as: loc_t { pargfid: NULL parent: NULL name: NULL path: NULL gfid: GFID to be looked up [out parameter] inode: inode_new () result [in parameter] } and performing such lookup will return in its callback an inode_t populated with the right contexts and a struct iatt which can be used to perform an inode_link () on the inode (without a parent and basename). The inode will now be hashed and linked in the inode table and findable via inode_find(). A fundamental change moving forward is that the primary fields in a loc_t structure are now going to be (pargfid, name) and (gfid) depending on the kind of FOP. So far path had been the primary field for operations. The remaining fields only serve as hints/helpers. 4b. If read/write is to be performed on an inode_t, the approach so far has been to: fd_create(), STACK_WIND(open, fd), fd_bind (in callback) and then perform STACK_WIND(read, fd) etc. With anonymous fds now you can do fd_anonymous (inode), STACK_WIND (read, fd). This results in great boost in performance in the inbuilt NFS server. 5. Misc ------- The inode_ctx_put[2] has been renamed to inode_ctx_set[2] to be consistent with the rest of the codebase. Change-Id: Ie4629edf6bd32a595f4d7f01e90c0a01f16fb12f BUG: 781318 Reviewed-on: http://review.gluster.com/669 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* features/marker: xtime should not be updated for modifications done by ↵Junaid2012-01-041-1/+7
| | | | | | | | | | | | | special client. Change-Id: Ia577422dedb94a1febeceb2a50cabf61d48cb714 BUG: 769494 Signed-off-by: Junaid <junaid@gluster.com> Reviewed-on: http://review.gluster.com/2559 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushik BV <kaushikbv@gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Csaba Henk <csaba@gluster.com>
generated by cgit (git 2.34.1) at 2024-05-12 08:15:29 +0000