summaryrefslogtreecommitdiffstats
path: root/libglusterfs
Commit message (Collapse)AuthorAgeFilesLines
* quota/marker: use smaller stacksize in synctask for marker updationvmallika2015-07-142-10/+28
| | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/11499/ Default stacksize that synctask uses is 2M. For marker we set it to 16k Also move market xlator close to io-threads to have smaller stack > Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651 > BUG: 1207735 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: Id1cb6288a38d370956cc47aed5253ff95f04c966 BUG: 1229282 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11595 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* refcount: correct the documentationNiels de Vos2015-07-141-4/+4
| | | | | | | | | | | | | | | | | | | | | | | The only check that _gf_ref_get() needs is "== 0" for detecting a failure. The actual return value is not guaranteed to be the number of active refences (they can change in other threads anyway). Cherry picked from commit c7f309116d8fa62f6b9fd6ff2902e8ce4bfa192d: > BUG: 1163543 > Change-Id: I8801601eab37046f5a5ee0bce5a62606115ca151 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/11328 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Change-Id: I8801601eab37046f5a5ee0bce5a62606115ca151 BUG: 1242515 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11647 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
* mem-pool,stack,store,syncop,timer/libglusterfs : Porting to a new logging ↵Mohamed Ashiq2015-07-149-155/+201
| | | | | | | | | | | | | | | | | | | | | | framework Backport of http://review.gluster.org/10827 Cherry picked from 2b9b3ef3b646989bbc0412dca187b3f5fcad3283 >Change-Id: Idd3dcaf7eeea5207b3a5210676ce3df64153197f >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10827 >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> Change-Id: Idd3dcaf7eeea5207b3a5210676ce3df64153197f BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11478 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* defaults,globals,iobuf,latency,logging,options,xlator/libglusterfs : porting ↵Mohamed Ashiq2015-07-138-179/+231
| | | | | | | | | | | | | | | | | | | | | | to a new logging framework Backport of http://review.gluster.org/10826 Cherry picked from ef374774b8727b2f76915a1f88b26c83c77dde43 >Change-Id: If6a55186cddc3d1c4d22e3d56b45358b84feeb49 >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10826 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Change-Id: If6a55186cddc3d1c4d22e3d56b45358b84feeb49 BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11405 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* quota/marker: fix spurious failure afr-quota-xattr-mdata-heal.tvmallika2015-07-122-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/11583 During quota-update process if inode info is present in size-xattr and missing in contri-xattrs, then in function '_mq_get_metadata', we set contri-size as zero (on error -2, which means usage info present, but inode info missing). With this we are calculating wrong delta and updating the same. With this patch we are ignoring errors if inode info in xattrs are missing > Change-Id: I7940a0e299b8bb425b5b43746b1f13f775c7fb92 > BUG: 1241153 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: Ie85fa84b5362ae179cc43402bd6a3a6d96a04b81 BUG: 1241831 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11614 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
* quota/marker: set lk_owner when taking lock on parent in renamevmallika2015-07-121-0/+19
| | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/11561/ Before doing a rename operation marker takes inode lock on the file parent, here lk_owner is NULL and this can cause accounting problem with multiple rename on the same directory This patch fix the problem by setting lk_owner > Change-Id: Ibb789e39b2833e425d0a5fca85282ff1465206cb > BUG: 1240598 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: Ibd3abaee2dd5d132906a79a518fbcedcdefc4552 BUG: 1241487 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11601 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* features/bitrot: cleanup, v2Venky Shankar2015-07-092-0/+14
| | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11148 This patch uses "cleanup, v1" infrastrcuture to cleanup scrubber (data structures, threads, timers, etc..) on brick disconnection. Signer is not cleaned up yet: probably would be done as part of another patch. Change-Id: I78a92b8a7f02b2f39078aa9a5a6b101fc499fd70 BUG: 1226666 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/11540 Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* protocol/server: Add null check to gf_client_putRaghavendra G2015-07-091-0/+6
| | | | | | | | | | | | Change-Id: I8bab3cd7387f89743e15e7569f0bc83a7df3c754 BUG: 1240603 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/11550 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11562 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* call-stub,circ-buff,client_t,compat,dict/libglusterfs : Porting to a new ↵Mohamed Ashiq2015-07-096-182/+256
| | | | | | | | | | | | | | | | | | | | | | | logging framework Backport of http://review.gluster.org/10828 Cherry picked from a1e32fbcfbfaf6e4c63e140b3b90a80dc748a269 >Change-Id: Ie7d180e0ab2fed1270d66504606d1b2522884020 >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10828 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Change-Id: Ie7d180e0ab2fed1270d66504606d1b2522884020 BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11406 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* event,parse-utils,quota-common-utils,rbthash/libglusterfs : Porting to new ↵Mohamed Ashiq2015-07-097-104/+126
| | | | | | | | | | | | | | | | | | | | | | logging framework Backport of http://review.gluster.org/10823 Cherry-picked from a9b6933ef097d2a81ce21a8aeda2acc569cd1509 >Change-Id: I13ae9a97fe442c06cf50fe77c63718ea0858dc4d >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10823 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Change-Id: I13ae9a97fe442c06cf50fe77c63718ea0858dc4d BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11404 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* libglusterfs: Use GF_CALLOC/GF_FREE instead of CALLOC/FREEPranith Kumar K2015-07-082-144/+147
| | | | | | | | | | | | | | | | | | | | | - Also removed numbers for the types as the string form of type is printed in statedump now, so the numbers are not needed anymore. >Change-Id: I6e8c15a1dc8cb6187842f96f1d46ec0f26a602b4 >BUG: 1237381 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/11495 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Vijay Bellur <vbellur@redhat.com> >(cherry picked from commit e7f2547f89dbcd90cdb3714f63620a36bdc2ef3a) Change-Id: I60dbe14ba0c9855982e235001ee86747e3fa87a0 BUG: 1238476 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/11501 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* fd,inode/libglusterfs : porting to a new logging frameworkMohamed Ashiq2015-07-073-107/+172
| | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10825/ Cherry picked from d32b221747cf6742cffaff5a60e16269dffd04cc >Change-Id: I4beba3b50456f802824374b6e3fa8079d72f2c00 >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10825 >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Change-Id: I4beba3b50456f802824374b6e3fa8079d72f2c00 BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11402 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* libgfapi: send explicit lookups on inodes linked in readdirpRaghavendra Bhat2015-07-062-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11236 If the inode is linked via readdirp, then the consuners of gfapi which are using handles (got either in lookup or readdirp) might not send an explicit lookup on that object again (ex: NFS, samba, USS). If there is a replicate volume where the replicas of the object are not in sync, then readdirp followed by fops might lead data being served from the subvolume which is not in sync with latest data. And since lookup is needed to trigger self-heal on that object the consumers might keep getting wrong data until an explicit lookup is not done. Fuse handles this situation by sending an explicit lookup by itself (fuse xlator) on those inodes which are linked via readdirp, whenever a fop comes on that inode. The same procedure is done in gfapi as well to address this situation. Thanks to shyam(srangana@redhat.com) for valuable inputs Change-Id: I4230fae8e0b01a95c056282b08ed30832d4804a7 BUG: 1240190 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11545 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* bit-rot : New logging framework for bit-rot log messageMohamed Ashiq2015-07-011-6/+6
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/10297 Cherry picked from 2f0d36d16c241365760aaa6d857b7a4d438e1042 >Change-Id: I83c494f2bb60d29495cd643659774d430325af0a >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10297 >Tested-by: Venky Shankar <vshankar@redhat.com> >Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> >Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Venky Shankar <vshankar@redhat.com> Change-Id: I83c494f2bb60d29495cd643659774d430325af0a BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11379 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* stack: use list_head for managing framesKrishnan Parthasarathi2015-07-013-91/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROBLEM -------- statedump requests that traverse call frames of all call stacks in execution may race with a STACK_RESET on a stack. This could crash the corresponding glusterfs process. For e.g, recently we observed this in a regression test case tests/basic/afr/sparse-self-heal.t. FIX --- gf_proc_dump_pending_frames takes a (TRY_LOCK) call_pool->lock before iterating through call frames of all call stacks in progress. With this fix, STACK_RESET removes its call frames under the same lock. Additional info ---------------- This fix makes call_stack_t to use struct list_head in place of custom doubly-linked list implementation. This makes call_frame_t manipulation easier to maintain in the context of STACK_WIND et al. BUG: 1234408 Change-Id: I7e43bccd3994cd9184ab982dba3dbc10618f0d94 Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/11095 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> (cherry picked from commit 79e4c7b2fad6db15863efb4e979525b1bd4862ea) Reviewed-on: http://review.gluster.org/11352
* common-utils/libglusterfs : port log messages to a new frameworkMohamed Ashiq Liyazudeen2015-07-013-587/+408
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/10824/ http://review.gluster.org/11400/ Cherry picked from 515a4c4e08eb9fcd92c9cca693ef9978b8a1b05c >Change-Id: Iee1c083774c988375a7261cfd6d510ed4c574de2 >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10824 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Kaushal M <kaushal@redhat.com> Change-Id: Iee1c083774c988375a7261cfd6d510ed4c574de2 BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11289 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* Logging: Porting the performance translatorarao2015-06-291-3/+42
| | | | | | | | | | | | | | | | logs to new logging framework > Change-Id: Ie6aaf8d30bd4457bb73c48e23e6b1dea27598644 > BUG: 1194640 > Signed-off-by: arao <arao@redhat.com> BUG: 1217722 Change-Id: I0856c43dbf8c0a1aa084d4478c9bdf3f41dfc0b8 Signed-off-by: arao <arao@redhat.com> Reviewed-on: http://review.gluster.org/11442 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/ctr: Ignore creation of T file and Ctr Lookup heal improvememntsDan Lambright2015-06-273-20/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a back port of 11334 1) Ignore creation of T file in ctr_mknod 2) Ignore lookup for T file in ctr_lookup 3) Ctr_lookup: a. If the gfid and pgfid in empty dont record b. Decreased log level for multiple heal attempts c. Inode/File heal happens after an expiry period, which is configurable. d. Hardlink heal happens after an expiry period, which is configurable. > Change-Id: Id8eb5092e78beaec22d05f5283645081619e2452 > BUG: 1235269 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/11334 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia28a5cf975e41d318906f707deca447aaa35630f BUG: 1236288 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/11446 Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com>
* glusterd/ afr : set afr pending xattrs on replace brickAnuradha2015-06-271-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/10076/ This patch is part one change to prevent data loss in a replicate volume on doing a replace-brick commit force operation. Problem: After doing replace-brick commit force, there is a chance that self heal happens from the replaced (sink) brick rather than the source brick leading to data loss. Solution: During the commit phase of replace brick, after old brick is brought down, create a temporary mount and perform setfattr operation (on virtual xattr) indicating AFR to mark the replaced brick as sink. As a part of this change replace-brick command is being changed to use mgmt_v3 framework rather than op-state-machine framework. Many thanks to Krishnan Parthasarathi for helping me out on this. Change-Id: If0d51b5b3cef5b34d5672d46ea12eaa9d35fd894 BUG: 1232173 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/11253 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* storage/posix: Introduce flag instructing posix to perform prestat, writev ↵Krutika Dhananjay2015-06-271-0/+1
| | | | | | | | | | | | | | and poststat atomically Backport of: http://review.gluster.org/11345 Change-Id: I0d7e3a851c744777083974ec4cdb01b08c23727b BUG: 1236271 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11439 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* ec: Porting messages to new logging frameworkNandaja Varma2015-06-271-0/+4
| | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/10465/ cherry-picked from commit b0b9eaea9dbb4e9a535f5e969defc4556a9e2204 >Change-Id: Ia05ae750a245a37d48978e5f37b52f4fb0507a8c >BUG: 1194640 >Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Change-Id: Ia05ae750a245a37d48978e5f37b52f4fb0507a8c BUG: 1217722 Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com> Reviewed-on: http://review.gluster.org/11429 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* core: add "gf_ref_t" for common refcounting structuresNiels de Vos2015-06-263-2/+217
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Checks for compiler supported atomic operations comes from client_t.h. An example usage of this change can be found in adding reference counting to "struct auth_cache_entry" in http://review.gluster.org/11023 Basic usage looks like this: #include "refcount.h" struct my_struct { GF_REF_DECL; ... /* more members */ } void my_destructor (void *data) { struct my_struct *my_ptr = (struct my_struct *) data; ... /* do some more cleanups */ GF_FREE (my_ptr); } void init_ptr (struct parent *parent) { struct my_struct *my_ptr = malloc (sizeof (struct my_struct)); GF_REF_INIT (my_ptr, my_destructor); /* refcount is set to 1 */ ... /* my_ptr probably gets added to some parent structure */ parent_add_ptr (parent, my_ptr); } void do_something (struct parent *parent) { struct my_struct *my_ptr = NULL; /* likely need to lock parent, depends on its access pattern */ my_ptr = parent_remove_first_ptr (parent); /* unlock parent */ ... /* do something */ GF_REF_PUT (my_ptr); /* calls my_destructor on refcount = 0 */ } URL: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/11202 Change-Id: Idb98a5861a44c31676108ed8876db12c320912ef BUG: 1235939 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11022 Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> (cherry picked from commit ea1d2b0045e2c2ad82a2162a58f9fc36f9d07b20) Reviewed-on: http://review.gluster.org/11421 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* gfdb/libglusterfs : Port log messages to a new frameworkMohamed Ashiq2015-06-265-272/+352
| | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10819/ Cherry picked from 2847b03a38515a625e36d52c7e0a1d37107d72ba >Change-Id: I61874561fdf2c175d2b3eec0c85c25f12dc60552 >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10819 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Joseph Fernandes Change-Id: I61874561fdf2c175d2b3eec0c85c25f12dc60552 BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11284 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/afr: Pick gfid from poststat during fresh lookup for read child ↵Krutika Dhananjay2015-06-252-0/+25
| | | | | | | | | | | | | | calculation Backport of: http://review.gluster.org/11373 Change-Id: I3ddc70cb0e7dbd1ef8adb352393b5ec16464fc94 BUG: 1212842 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11391 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* Porting new log messages for posixHari Gowtham2015-06-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/9893/ Cherry picked from 59e1c79ac6953e552fbdc8c627f5dc1d02eeacc8 >Change-Id: I29bdeefb755805858e3cb1817b679cb6f9a476a9 >BUG: 1194640 >Signed-off-by: Hari Gowtham <hgowtham@dhcp35-85.lab.eng.blr.redhat.com> >Reviewed-on: http://review.gluster.org/9893 >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Signed-off-by: Hari Gowtham <hgowtham@redhat.com> Change-Id: I29bdeefb755805858e3cb1817b679cb6f9a476a9 BUG: 1217722 Signed-off-by: Hari Gowtham <hgowtham@dhcp35-85.lab.eng.blr.redhat.com> Signed-off-by: Hari Gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/11279 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* libglusterfs: Introducing new logging messageMohamed Ashiq2015-06-222-1/+1957
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/9898 Cherry picked from a2a370db6db80e9365d0777701786ce706957f42 >Change-Id: I137f1b7805895810b8e6f0a70a183782bf472bf5 >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/9898 >Reviewed-by: Niels de Vos <ndevos@redhat.com> >Tested-by: Niels de Vos <ndevos@redhat.com> >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Change-Id: I137f1b7805895810b8e6f0a70a183782bf472bf5 BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11223 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* libglusterfs: avoid crash due to ctx being NULLHumble Devassy Chirammal2015-06-191-1/+27
| | | | | | | | | | | | | | | | >Reviewed-on: http://review.gluster.org/10155 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Niels de Vos <ndevos@redhat.com> >Reviewed-by: Vijay Bellur <vbellur@redhat.com> >(cherry picked from commit b855aa062927ea4845f56d2d1a1a2a8af3dc4192) BUG : 1233559 Change-Id: Ib667ed42f0b598baddb829b448d6efc3d8e044f1 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/10969 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* features/bitrot: tuanble object signing waiting time value for bitrotGaurav Kumar Garg2015-06-171-0/+3
| | | | | | | | | | | | | | | | | Currently bitrot using 120 second waiting time for object to be signed after all fop's released. This signing waiting time value should be tunable. Command for changing the signing waiting time will be #gluster volume bitrot <VOLNAME> signing-time <waiting time value in second> Change-Id: I89f3121564c1bbd0825f60aae6147413a2fbd798 BUG: 1231832 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/11105 (cherry picked from commit 554fa0c1315d0b4b78ba35a2d332d7ac0fd07d48) Reviewed-on: http://review.gluster.org/11235 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* changetimerecorder : port log messages to a new frameworkMohamed Ashiq2015-06-173-6/+502
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10938/ Cherry picked from 00f9a61fe8884062c141edd662424625d349a377 >Change-Id: I66e7ccc5e62482c3ecf0aab302568e6c9ecdc05d >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10938 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Joseph Fernandes >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Change-Id: I66e7ccc5e62482c3ecf0aab302568e6c9ecdc05d BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11195 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes
* libglusterfs: update glfs-message header for reserved segmentsHumble Devassy Chirammal2015-06-121-4/+36
| | | | | | | | | | | | | | | | | | all the logging patches will be dependent on this segment allocation. Sending this change as a seperate one to avoid the conflicts. This also include backport of http://review.gluster.org/#/c/10420/ Change-Id: Iedd72ab6dc3526f1a6b01828807b5e6b9edcba90 BUG: 1194640 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/10400 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> (cherry picked from commit cd4e9abee1270c6400037912aacbbf171c6897c0) Reviewed-on: http://review.gluster.org/10670 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* cluster/dht: Fix dht_setxattr to follow files under migrationNithya Balachandran2015-06-041-2/+3
| | | | | | | | | | | | | | | | If a file is under migration, then any xattrs created on it are lost post migration of the file. This is because the xattrs are set only on the cached subvol of the source and as the source is under migration, it becomes a linkto file post migration. Change-Id: Ib8e233b519cf954e7723c6e26b38fa8f9b8c85c0 BUG: 1225839 Signed-off-by: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/10968 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: Bump op version and max op version for 3.7.2Shyam2015-06-031-1/+4
| | | | | | | | | | | | | As 3.7.1 is released, and a DHT configuration option needs higher op version, bumping the gluster op-version to 3.7.2 (or 30702). Change-Id: Iaed9e49b86a195653ddca55994e2c2398b2ee3bc BUG: 1227887 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/11071 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* libglusterfs: Copy d_len and dict as well into dst direntKrutika Dhananjay2015-06-032-3/+23
| | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/11026 Also, added memory allocation failure checks in light of the comments received @ http://review.gluster.org/#/c/10809/2/libglusterfs/src/gf-dirent.c, and http://review.gluster.org/#/c/10809/1/xlators/features/shard/src/shard.c Change-Id: Iad4736bba69e6977390f76d0c9eb64217a005b80 BUG: 1227576 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11052 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* glusterd: Bump op version and max op version for 3.7.1Shyam2015-06-021-1/+4
| | | | | | | | | | | | | | | As 3.7 is released, and a DHT configuration option needs higher op version, bumping the gluster op-version to 3.7.1 (or 30701). Change-Id: I9747cf93b41be72e43077ed8e977e21eed99ccc3 BUG: 1225999 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-upstream-on: http://review.gluster.org/10849 Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/10975 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* quota: quota.conf backward compatibility fixvmallika2015-06-012-57/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/10889/ > In release-3.7 the format of quota.conf is changed. > There is a backward compatibility issues during upgrade > 1) There can be an issue when peer sync between node-3.6 and node-3.7 > 2) If the user sets/removes limit, there is will different format of > file in node-3.6 and node-3.7 > > This patch fixes the issue: > 1) restrict the user to execute command quota enable, limit-usage, remove > 2) write quota.conf in older format if op-version is less than 3.6 > > Change-Id: Ib76f5a0a85394642159607a105cacda743e7d26b > BUG: 1223739 > Signed-off-by: vmallika <vmallika@redhat.com> > Reviewed-on: http://review.gluster.org/10889 > Tested-by: NetBSD Build System > Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Change-Id: I3d5075b96de1e125d30b0f6045389573aaaaecdb BUG: 1226153 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/10985 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* tier/tier.t: Fixing tier.t crash in regression runsJoseph Fernandes2015-05-302-8/+17
| | | | | | | | | | | | | | | | | | | | | | | 1) If the database file exists a. Dont try re-creating the db schema b. Dont try re-configuring the db. 2) Dont assert in fini_db () when connection is NULL >> Signed-off-by: Joseph Fernandes <josferna@redhat.com> >> Reviewed-on: http://review.gluster.org/10870 >> Tested-by: NetBSD Build System >> Tested-by: Gluster Build System <jenkins@build.gluster.com> >> Reviewed-by: Vijay Bellur <vbellur@redhat.com> >> Signed-off-by: Joseph Fernandes <josferna@redhat.com> >>Change-Id: Idd2833f55caaa6b3a77d935d877d6c4d2994da6a Change-Id: Ia2e044fce4e41bccc8fdada1cb21f240fdbd55df BUG: 1225077 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10992 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* protocol/client,server: Move EEXIST logs in mkdir and mknod to DEBUG levelKrutika Dhananjay2015-05-281-0/+4
| | | | | | | | | | | Backport of: http://review.gluster.org/10791 Change-Id: I096296f0b97f62f49577ca698ae34e28cce4a4b4 BUG: 1225919 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10973 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/bit-rot-stub: versioning of objects in write/truncate fop instead ↵Raghavendra Bhat2015-05-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | of open * This patch brings in the changes where object versioning is done in write and truncate fops instead of tracking them in open and create fops. This model works for both regular and anonymous fds. It also removes the race associated with open calls, create and lookups. This patch follows the below method for object versioning and notifications: Before sending writev on the fd, increase the ongoing version first. This makes anonymous fd write similar to the regular fd write by having the ongoing version increased before doing the write. Do following steps to do versioning: 1) For anonymous fds set the fd context (so that release is invoked) and add the fd context to the list maintained in the inode context. For regular fds the above think would have been done in open itself. 2) Increase the on-disk ongoing version 3) Increase the in memory ongoing version and mark inode as non-dirty 3) Once versioning is successfully done send write operation. If versioning fails, then fail the write fop. 5) In writev_cbk mark inode as modified. > Change-Id: I7104391bbe076d8fc49b68745d2ec29a6e92476c > BUG: 1207979 > Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> > Reviewed-on: http://review.gluster.org/10233 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I4bb86989b5fab02b9ed2950798b1a80e566f1024 BUG: 1220041 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/10722 Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com> Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/bitrot: Throttle filesystem scrubberVenky Shankar2015-05-104-24/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces multithreaded filesystem scrubber based on throttling option configured for a particular volume. The implementation "logically" breaks scanning and scrubbing with the number of scrubber threads auto-configured depending upon the throttle configuration. Scanning (crawling) is left single threaded (per brick) with entries scrubbed in bulk. On reaching this "bulk" watermark, scanner waits until entries are scrubbed. Bricks for a particular volume have a set of thread(s) assigned for scrubbing, with entries for each brick scrubbed in a round robin fashion to avoid scrub "stalls" when a brick (out of N bricks) is under active scrubbing. This mechanism helps us implement "pause/resume" with ease: all one need to do is to cleanup scrubber threads and let the main scanner thread "wait" untill scrubbing is resumed (where the scrubber thread(s) are spawned again), therefore continuing where we left off (unless we restart the deamons, where crawl initiates from root directory again, but I guess that's OK). [ NOTE: Throttling is optional for the signer daemon, without which it runs full throttle. However, passing "-DBR_RATE_LIMIT_SIGNER" predefined in CFLAGS enables CPU throttling (during checksum calculation) thereby avoiding high CPU usage. ] Subsequent patches would introduce CPU throttling during hash calculation for scrubber. > Change-Id: I5701dd6cd4dff27ca3144ac5e3798a2216b39d4f > BUG: 1207020 > Signed-off-by: Venky Shankar <vshankar@redhat.com> > Reviewed-on: http://review.gluster.org/10511 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I5a125b2d0ac7dafd3e278b7fe4c6c9dd07af76dd Signed-off-by: Venky Shankar <vshankar@redhat.com> BUG: 1220041 Reviewed-on: http://review.gluster.org/10720 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
* features/bitrot: Follow xattr naming conventionsVenky Shankar2015-05-101-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | Instead of "trusted.glusterfs.bit-rot.*" use "trusted.bit-rot.*" NOTE: With this patch, data on existing volumes would be resigned (which should be OK as of now since we do not expect many users as of now :-)) > Change-Id: I926c7bca266a9c8f2cb35d57c4d0359aa5cecfa0 > BUG: 1170075 > Signed-off-by: Venky Shankar <vshankar@redhat.com> > Reviewed-on: http://review.gluster.org/10181 > Tested-by: NetBSD Build System > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I3c18d7dc2db4beaca6e8d8d231b4171a7b18795f Signed-off-by: Venky Shankar <vshankar@redhat.com> BUG: 1220041 Reviewed-on: http://review.gluster.org/10718 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
* core: Global timer-wheelVenky Shankar2015-05-104-3/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instantiate a process wide global instance of the timer wheel data structure. Spawning glusterfs* process with option arg "--global-timer-wheel" instantiates a global instance of timer-wheel under global context (->ctx). Translators can make use of this process wide instance [via a call to glusterfs_global_timer_wheel()] instead of maintaining an instance of their own and possibly consuming more memory. Linux kernel too has a single instance of timer wheel where subsystems such as IO, networking, etc.. make use of. Bitrot daemon would be early consumers of this: bitrot translator instances for multiple volumes would track objects belonging to their respective bricks in this global expiry tracking data structure. This is also a first step to move GlusterFS timer mechanism to use timer-wheel. > Change-Id: Ie882df607e07acaced846ea269ebf1ece306d6ae > BUG: 1170075 > Signed-off-by: Venky Shankar <vshankar@redhat.com> > Reviewed-on: http://review.gluster.org/10380 > Tested-by: NetBSD Build System > Reviewed-by: Vijay Bellur <vbellur@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Change-Id: I35c840daa9996a059699f8ea5af54c76ede7e09c Signed-off-by: Venky Shankar <vshankar@redhat.com> Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> BUG: 1220041 Reviewed-on: http://review.gluster.org/10716 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* CTR/Libgfdb: Log typo fixJoseph Fernandes2015-05-101-2/+2
| | | | | | | | | | | Log typo fix for CTR Xlator and Libgfdb Change-Id: Ia39069a5ce9c48bbee937f1b5c5d749a30c9ac56 BUG: 1220100 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10742 Reviewed-by: N Balachandran <nbalacha@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* dht: make lookup-unhashed=auto do something actually usefulJeff Darcy2015-05-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | The key concept here is to determine whether a directory is "clean" by comparing its last-known-good topology to the current one for the volume. These are stored as "commit hashes" on the directory and the volume root respectively. The volume's commit hash changes whenever a brick is added or removed, and a fix-layout is done. A directory's commit hash changes only when a full rebalance (not just fix-layout) is done on it. If all bricks are present and have a directory commit hash that matches the volume commit hash, then we can assume that every file is in its "proper" place. Therefore, if we look for a file in that proper place and don't find it, we can assume it's not on any other subvolume and *safely* skip the global (broadcast to all) lookup. Change-Id: Id6ce4593ba1f7daffa74cfab591cb45960629ae3 BUG: 1220064 Reviewed-on-master: http://review.gluster.org/#/c/7702/ Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/10729 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* bitrot/scrub: fix induced throttling in syncop_ftw_throttle()Venky Shankar2015-05-091-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Failing to reset scanning counter causes "incorrect" delay of around 50 seconds per directory entry. This causes scrubber to run extremely slowly. [ NOTE: This is a temporary fix. With the introduction of token bucket based throttling, inducing throttle via sleep() call would be unneeded. ] Also, fix logging messages in scrubber to log brick and full path of the object which is identified/marked as corrupted. > Change-Id: Id501bd15dcdbd8a09613f80f9d84050304740027 > BUG: 1170075 > Signed-off-by: Venky Shankar <vshankar@redhat.com> > Reviewed-on: http://review.gluster.org/10375 > Tested-by: NetBSD Build System > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> > Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com> Change-Id: I78f227f52f12549d62ecb35cbb70121424f7c2a7 BUG: 1220041 Reviewed-on: http://review.gluster.org/10714 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* core: use reference counting for mem_acct structuresJeff Darcy2015-05-0910-174/+193
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When freeing memory, our memory-accounting code expects to be able to dereference from the (previously) allocated block to its owning translator. However, as we have already found once in option validation and twice in logging, that translator might itself have been freed and the dereference attempt causes on of our daemons to crash with SIGSEGV. This patch attempts to fix that as follows: * We no longer embed a struct mem_acct directly in a struct xlator, but instead allocate it separately. * Allocated memory blocks now contain a pointer to the mem_acct instead of the xlator. * The mem_acct structure contains a reference count, manipulated in both the normal and translator allocate/free code using atomic increments and decrements. * Because it's now a separate structure, we can defer freeing the mem_acct until its reference count reaches zero (either way). * Some unit tests were disabled, because they embedded their own copies of the implementation for what they were supposedly testing. Life's too short to spend time fixing tests that seem designed to impede progress by requiring a certain implementation as well as behavior. Change-Id: Id929b11387927136f78626901729296b6c0d0fd7 BUG: 1219026 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/10417 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10723 Tested-by: NetBSD Build System
* cluster/afr : Prevent inode-evict during split-brain resolutionAnuradha2015-05-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/#/c/10134/ 1) Provided setfattr command to set timeout for split-brain choice. 2) If split-brain inspection/resolution is being done from the mount for a file, ref the inode when split-brain-choice is set. This inode will be unconditionally unref-ed after timeout seconds set by the user/default otherwise. 3) Updated the doc and testcase to reflect the changes. Change-Id: I15c9037dee28855f21e680e7e3632e1f48dba4e1 BUG: 1219388 Reviewed-on: http://review.gluster.org/10134 Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Signed-off-by: Anuradha <atalur@redhat.com> Reviewed-on: http://review.gluster.org/10679
* cluster/ec: data heal implementation for ecPranith Kumar K2015-05-081-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | Data self-heal: 1) Take inode lock in domain 'this->name:self-heal' on 0-0 range (full file), So that no other processes try to do self-heal at the same time. 2) Take inode lock in domain 'this->name' on 0-0 range (full file), 3) perform fxattrop+fstat and get the xattrs on all the bricks 3) Choose the brick with ec->fragment number of same version as source 4) Truncate sinks 5) Unlock lock taken in 2) 5) For each block take full file lock, Read from sources write to the sinks, Unlock 6) Take full file lock and see if the file is still sane copy i.e. File didn't become unusable while the bricks are offline. Update mtime to before healing 7) xattrop with -ve values of 'dirty' and difference of highest and its own version values for version xattr 8) unlock lock acquired in 6) 9) unlock lock acquired in 1) Change-Id: I6f4d42cd5423c767262c9d7bb5ca7767adb3e5fd BUG: 1216303 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10384 Reviewed-on: http://review.gluster.org/10692 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cluster/ec: Fix dictionary compare functionPranith Kumar K2015-05-082-1/+91
| | | | | | | | | | | | | | | | | | | | | If both dicts are NULL then equal. If one of the dicts is NULL but the other has only ignorable keys then also they are equal. If both dicts are non-null then check if for each non-ignorable key, values are same or not. value_ignore function is used to skip comparing values for the keys which must be present in both the dictionaries but the value could be different. geo-rep's stime xattr doesn't need to be present in list xattr but when getxattr comes on stime xattr even if there aren't enough responses with the xattr we should still give out an answer which is maximum of the stimes available. Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d BUG: 1216303 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/10078 Reviewed-on: http://review.gluster.org/10690 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System
* geo-rep: rename handling in dht volumeNithya Balachandran2015-05-082-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: Glusterfs changelogs are stored in each brick, which records the changes happened in that brick. Georep will run in all the nodes of master and processes changelogs "independently". Processing changelogs is in brick level, but all the fops will be replayed on "slave mount" point. Problem: With a DHT volume, in changelog "internal fops" are NOT recorded. For Rename case, Rename is recorded in "hashed" brick changelog. (DHT's internal fops like creating linkto file, unlink is NOT recorded). This lead us to inconsistent rename operations. For example, Distribute volume created with Two bricks B1, B2. //Consider master volume mounted @ /mnt/master and following operations executed: cd /mnt/master touch f1 // f1 falls on B1 Hash mv f1 f2 // f2 falls on B2 Hash // Here, Changelogs are recorded as below: @B1 CREATE f1 @B2 RENAME f1 f2 Here, race exists between Brick B1 and B2, say B2 will get executed first. Source file f1 itself is "NOT PRESENT", so it will go ahead and create f2 (Current implementation). We have this problem When rename falls in another brick and file is unlinked in Master. Similar kind of issue exists in following case too(multiple rename): CREATE f1 RENAME f1 f2 RENAME f2 f1 Solution: Instead of carrying out "changelogging" at "HASHED volume", carry out at the "CACHED volume". This way we have rename operations carried out where actual files are present. So,Changelog recorded as : @B1 CREATE f1 RENAME f1 f2 credit: sarumuga@redhat.com PS: Some of the races as the one below are _NOT_ fixed by this patch * f1 and f2 exist. B1 and B2 are their respective cached subvols. For both files hashed-subvol == cached-subvol * mv f1 f2 on master. * B1 has change-log entry of rename f1 f2 * rebalance migrates f2 from B1 and B2 * mv f2 f1 on master. * B2 has change-log entry of rename f2 f1 Since changelog entries (rename f1 f2) and (rename f2 f1) are processed independently by gsyncds, which of either f1 and f2 survives on slave is subject to race. Note that on master its file f1 with name f1 which survived. On slave it can be either file f1 with name f1 or file f2 with name f2 based on who wins the race of processing changelog. BUG: 1219412 Change-Id: I43725d69635e2ce065135691ef629014e8df7d50 Original-Author: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/10410 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Reviewed-on: http://review.gluster.org/10628 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ctr/xlator: Named lookup heal of pre-existing files, before ctr was ON.Joseph Fernandes2015-05-081-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The CTR xlator records file meta (heat/hardlinks) into the data. This works fine for files which are created after ctr xlator is switched ON. But for files which were created before CTR xlator is ON, CTR xlator is not able to record either of the meta i.e heat or hardlinks. Thus making those files immune to promotions/demotions. Solution: The solution that is implemented in this patch is do ctr-db heal of all those pre-existent files, using named lookup. For this purpose we use the inode-xlator context variable option in gluster. The inode-xlator context variable for ctr xlator will have the following, a. A Lock for the context variable b. A hardlink list: This list represents the successful looked up hardlinks. These are the scenarios when the hardlink list is updated: 1) Named-Lookup: Whenever a named lookup happens on a file, in the wind path we copy all required hardlink and inode information to ctr_db_record structure, which resides in the frame->local variable. We dont update the database in wind. During the unwind, we read the information from the ctr_db_record and , Check if the inode context variable is created, if not we create it. Check if the hard link is there in the hardlink list. If its not there we add it to the list and send a update to the database using libgfdb. Please note: The database transaction can fail(and we ignore) as there already might be a record in the db. This update to the db is to heal if its not there. If its there in the list we ignore it. 2) Inode Forget: Whenever an inode forget hits we clear the hardlink list in the inode context variable and delete the inode context variable. Please note: An inode forget may happen for two reason, a. when the inode is delete. b. the in-memory inode is evicted from the inode table due to cache limits. 3) create: whenever a create happens we create the inode context variable and add the hardlink. The database updation is done as usual by ctr. 4) link: whenever a hardlink is created for the inode, we create the inode context variable, if not present, and add the hardlink to the list. 5) unlink: whenever a unlink happens we delete the hardlink from the list. 6) mknod: same as create. 7) rename: whenever a rename happens we update the hardlink in list. if the hardlink was not present for updation, we add the hardlink to the list. What is pending: 1) This solution will only work for named lookups. 2) We dont track afr-self-heal/dht-rebalancer traffic for healing. > http://review.gluster.org/#/c/10370/ > Cherry picked from commit cb11dd91a6cc296e4a3808364077f4eacb810e48 > Change-Id: Ia4bbaf84128ad6ce8c3ddd70bcfa82894c79585f > BUG: 1212037 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/10370 > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I367aa46c3f4b8f912248fb8be75866507f2538df BUG: 1219075 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10370 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10615 Tested-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-01 20:08:30 +0000