summaryrefslogtreecommitdiffstats
path: root/xlators/features/bit-rot/src/stub
Commit message (Collapse)AuthorAgeFilesLines
* bitrot/stub: Fix memory leak gf_br_stub_mt_version_tMohammed Rafi KC2020-09-302-31/+42
| | | | | | | | | | The data type gf_br_stub_mt_version_t is leaking because the data type has been added to the dict as static type and the memory is not free'd from anywhere else Change-Id: Ib10134f41f45a39be849af1708b1960c77fb62fd Fixes: #1427 Signed-off-by: Mohammed Rafi KC <rafi.kavungal@iternity.com>
* bitrot: Make number of signer threads configurableKotresh HR2020-02-071-0/+1
| | | | | | | | | | | | | The number of signing process threads (glfs_brpobj) is set to 4 by default. The recommendation is to set it to number of cores available. This patch makes it configurable as follows gluster vol bitrot <volname> signer-threads <count> fixes: bz#1797869 Change-Id: Ia883b3e5e34e0bc8d095243508d320c9c9c58adc Signed-off-by: Kotresh HR <khiremat@redhat.com>
* xlator/bit-rot-stub-helpers: structure loggingyatipadia2020-01-143-71/+76
| | | | | | | | convert gf_msg() to gf_smsg() Change-Id: Id03b6a2f6817101650ea6eeffb10e91c09e6ef80 Updates: #657 Signed-off-by: yatipadia <ypadia@redhat.com>
* xlator/bit-rot-stub: structure loggingyatipadia2019-12-272-148/+141
| | | | | | | | convert gf_msg() to gf_smsg() Change-Id: I3072ba4305b720bea053688d140660d7d5c2d267 Updates: #657 Signed-off-by: yatipadia <ypadia@redhat.com>
* Multiple files: make root gfid a static variableYaniv Kaul2019-10-141-9/+2
| | | | | | | | | | | | | In many places we use it, compare to it, etc. It could be a static variable, as it really doesn't change. I think it's better than initializing to 0 and then doing gfid[15] = 1 or other tricks. I think there are additional oppportunuties to make more variables static. This is an attempt at an easy one. Change-Id: I7f23a30a94056d8f043645371ab841cbd0f90d19 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* graph/shd: Use top down approach while cleaning xlatorMohammed Rafi KC2019-06-271-0/+1
| | | | | | | | | | | | | | We were cleaning xlator from botton to top, which might lead to problems when upper xlators trying to access the xlator object loaded below. One such scenario is when fd_unref happens as part of the fini call which might lead to calling the releasedir to lower xlator. This will lead to invalid mem access Change-Id: I8a6cb619256fab0b0c01a2d564fc88287c4415a0 Updates: bz#1716695 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
* multiple files: another attempt to remove includesYaniv Kaul2019-06-141-1/+0
| | | | | | | | | | | | | | | | | | There are many include statements that are not needed. A previous more ambitious attempt failed because of *BSD plafrom (see https://review.gluster.org/#/c/glusterfs/+/21929/ ) Now trying a more conservative reduction. It does not solve all circular deps that we have, but it does reduce some of them. There is just too much to handle reasonably (dht-common.h includes dht-lock.h which includes dht-common.h ...), but it does reduce the overall number of lines of include we need to look at in the future to understand and fix the mess later one. Change-Id: I550cd001bdefb8be0fe67632f783c0ef6bee3f9f updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* features/bit-rot-stub: clean the mutex after cancelling the signer threadRaghavendra Bhat2019-04-172-7/+59
| | | | | | | | | | | | | | | | | | When bit-rot feature is disabled, the signer thread from the bit-rot-stub xlator (the thread which performs the setxattr of the signature on to the disk) is cancelled. But, if the cancelled signer thread had already held the mutex (&priv->lock) which it uses to monitor the queue of files to be signed, then the mutex is never released. This creates problems in future when the feature is enabled again. Both the new instance of the signer thread and the regular thread which enqueues the files to be signed will be blocked on this mutex. So, as part of cancelling the signer thread, unlock the mutex associated with it as well using pthread_cleanup_push and pthread_cleanup_pop. Change-Id: Ib761910caed90b268e69794ddeb108165487af40 updates: bz#1700078 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* feature/bitrot: Avoid thread creation if xlator is not enabledMohit Agrawal2019-01-311-8/+64
| | | | | | | | | | | | | Problem: Avoid thread creation for bitrot-stub for a volume if feature is not enabled Solution: Before thread creation check the flag if feature is enabled Updates: #475 Change-Id: I2c6cc35bba142d4b418cc986ada588e558512c8e Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com>
* features/bit-rot: do not send version and signature keys in dictRaghavendra Bhat2019-01-082-10/+53
| | | | | | | | | | | In lookup, if the file has been marked as bad, then bit-rot-stub was sending the version and signature xattr values as well in the response dictinary. This is not needed. Only bad file marker has to be sent. Change-Id: Id59c02e9857577c60849fd28ef657f71e0b15207 fixes: bz#1664122 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* all: add xlator_api to many translatorsAmar Tumballi2018-12-061-0/+14
| | | | | | Fixes: #164 Change-Id: I93ad6f0232a1dc534df099059f69951e1339086f Signed-off-by: Amar Tumballi <amarts@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-055-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* all: fix warnings on non 64-bits architecturesXavi Hernandez2018-10-102-4/+4
| | | | | | | | | | When compiling in other architectures there appear many warnings. Some of them are actual problems that prevent gluster to work correctly on those architectures. Change-Id: Icdc7107a2bc2da662903c51910beddb84bdf03c0 fixes: bz#1632717 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* features/bit-rot: NULL pointer deferencing clang fixSheetal Pamecha2018-10-082-5/+13
| | | | | | | | | | | Problem: local could be NULL Added condition checks to address this issue Updates: bz#1622665 Change-Id: I7be7dacc5386a77441385240b43f22d85074b69d Signed-off-by: Sheetal Pamecha <sheetal.pamecha08@gmail.com>
* Land part 2 of clang-format changesGluster Ant2018-09-122-3137/+3088
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Land clang-format changesGluster Ant2018-09-125-438/+414
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* bit-rot xlator: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-313-16/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | xlators/features/bit-rot/src/bitd/bit-rot-scrub-status.c xlators/features/bit-rot/src/stub/bit-rot-stub-helpers.c xlators/features/bit-rot/src/stub/bit-rot-stub.c xlators/features/bit-rot/src/stub/bit-rot-stub.h strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Ensure sprintf() results do not truncate. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN or sizeof() for const strings. - move ret from int32 to int. Compile-tested only! Change-Id: Ib9b923c45d2d59ac15a105410e8160b252061018 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-222-12/+12
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* make posix return errors when gfid2path key is absentRaghavendra Bhat2018-05-181-23/+15
| | | | | | Change-Id: I3a8d452d00560dac5e0b7ff0b1835d1f20a59f91 updates: bz#1570962 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* features/bitrot: print the path of the corrupted objectsRaghavendra Bhat2018-05-043-1/+235
| | | | | | | | | | | | | | | | | | | | | | | | | Currently "gluster volume bitrot <volume name> scrub status" gives the list of the corrupted objects (files as of now). But only the gfids of those corrupted objects are seen and one has to do getfattr, find etc operations to get the actual path of those objects for removal etc. This change makes an attempt to print the path of those files as much as possible. * Try to get the path using the on disk gfid2path xattr. * If the above operation fails, then go for in memory path (provided that the object has its dentry properly created and linked in the inode table of the brick where the corrupted object is present) So the gfid to path resolution is a soft resolution, i.e. based on the inode and dentry cache in the brick's memory. If the path cannot be obtained via inode table also, then only gfid is printed. Change-Id: Ie9a30307f43a49a2a9225821803c7d40d231de68 fixes: bz#1570962 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* glusterfsd: Memleak in glusterfsd process while brick mux is onMohit Agrawal2018-02-271-13/+9
| | | | | | | | | | | | | | | | | | Problem: At the time of stopping the volume while brick multiplex is enabled memory is not cleanup from all server side xlators. Solution: To cleanup memory for all server side xlators call fini in glusterfs_handle_terminate after send GF_EVENT_CLEANUP notification to top xlator. BUG: 1544090 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Note: Run all test-cases in separate build (https://review.gluster.org/19574) with same patch after enable brick mux forcefully, all test cases are passed. Change-Id: Ia10dc7f2605aa50f2b90b3fe4eb380ba9299e2fc
* Revert "glusterfsd: Memleak in glusterfsd process while brick mux is on"Mohit Agrawal2018-02-191-9/+13
| | | | | | | | | | | There are still remain some code paths where cleanup is required while brick mux is on.I will upload a new patch after resolve all code paths. This reverts commit b313d97faa766443a7f8128b6e19f3d2f1b267dd. BUG: 1544090 Change-Id: I26ef1d29061092bd9a409c8933d5488e968ed90e Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* build: add --without-server optionNiels de Vos2018-02-191-0/+2
| | | | | | | | | | | | | | | | With Gluster 4.0 we will not provide the server components for EL6 and older. At one point Gluster 4.x will get GlusterD2, which requires Golang tools in the distribution. EL6 does not contain these at the moment. With this change, it is possible to `./configure --without-server` which prevents building glusterd and the xlators for the bricks. Building RPMs can pass `--without server` and the glusterfs-server sub-package will not be created. Change-Id: I97f5ccf9f2c76e60d9af83915fc59fae57ad6d25 BUG: 1074947 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* glusterfsd: Memleak in glusterfsd process while brick mux is onMohit Agrawal2018-02-151-13/+9
| | | | | | | | | | | | | Problem: At the time of stopping the volume while brick multiplex is enabled memory is not cleanup from all server side xlators. Solution: To cleanup memory for all server side xlators call fini in glusterfs_handle_terminate after send GF_EVENT_CLEANUP notification to top xlator. BUG: 1544090 Change-Id: Ifa1525e25b697371276158705026b421b4f81140 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* feature/bitrot: remove internal xattrs from lookup cbkRavishankar N2017-12-192-7/+21
| | | | | | | | | | | | | | | | | | | | | Problem: afr requests all xattrs in lookup via the list-xattr key. If bitrot is enabled and later disabled, or if the bitrot xattrs were present due to an older version of bitrot which used to create the xattrs without enabling the feature, the xattrs (trusted.bit-rot.version in particular) was not getting filtered and ended up reaching the client stack. AFR, on noticing different values of the xattr across bricks of the replica, started triggering spurious metadata heals. Fix: Filter all internal xattrs in bitrot xlator before unwinding lookup, (f)getxattr. Thanks to Kotresh for the help in RCA'ing. Change-Id: I5bc70e4b901359c3daefc67b8e4fa6ddb47f046c BUG: 1524365 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* all: Simplify component message id's definitionXavier Hernandez2017-12-141-254/+43
| | | | | | | | | This patch creates a new way of defining message id's that is easier and less error prone because it doesn't require so many manual changes each time a new component is defined or a new message created. Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* features/bitrot: Add default value for stub export optionKaushal M2017-11-291-1/+2
| | | | | | | Updates #302 Change-Id: Ife78e15ad6300f09a820cbc25f43f214dc5e611d Signed-off-by: Amar Tumballi <amarts@redhat.com>
* feature/bitrot: stub xlators opts for GD2Kotresh HR2017-11-171-0/+5
| | | | | | Updates #302 Change-Id: I320eabf0c83295e90a312316a8373ccf5bf91dc4 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* features/bitrot: Fix coverity issuesKotresh HR2017-11-093-13/+28
| | | | | | | | | | | | | 1. br_update_scrub_finish_time: BUFFER_SIZE_WARNING 2. br_read_bad_object_dir : DEADCODE 3. bit-rot.c: init : RESOURCE_LEAK 4. br_stub_fsetxattr : STACK_USE 5. br_stub_setxattr : STACK_USE 6. bit-rot-stub.c: init : BUFFER_SIZE_WARNING Change-Id: Ie620f431bd7548fedae2152aa756ccdcd89ddf89 Signed-off-by: Kotresh HR <khiremat@redhat.com> BUG: 789278
* xlator/bitrot: flood of -Wformat-truncation warnings with gcc-7.1Kaleb S. KEITHLEY2017-10-043-70/+26
| | | | | | | | | | | | | | | | Starting in Fedora 26 which has gcc-7.1.x, -Wformat-trunction is enabled with -Wformat, resulting in a flood of new warnings. This many warnings is a concern because it makes it hard(er) to see other warnings that should be addressed. An example is at https://kojipkgs.fedoraproject.org//packages/glusterfs/3.12.0/1.fc28/data/logs/x86_64/build.log For more info see https://review.gluster.org/#/c/18267/ Change-Id: I7792d94da1e8109f3aaa857a94be40f2d2402684 BUG: 1492851 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* libglusterfs: Name threads on creationRaghavendra Talur2017-07-191-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set names to threads on creation for easier debugging. Output of top -H -p <PID-OF-GLUSTERFSD> Before: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd After: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustertimer 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustermemsweep 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc0 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc1 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll0 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteridxwrker 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteriotwr0 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrssign 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrswrker 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterclogecon 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd0 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd1 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd2 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixjan 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixfsy 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll1 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll2 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixhc Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703 BUG: 1254002 Updates: #271 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: https://review.gluster.org/11926 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* glusterfs: Not able to mount running volume after enable brick mux and ↵Mohit Agrawal2017-05-311-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | stopped any volume Problem: After enabled brick mux if any volume has down and then try ot run mount with running volume , mount command is hung. Solution: After enable brick mux server has shared one data structure server_conf for all associated subvolumes.After down any subvolume in some ungraceful manner (remove brick directory) posix xlator sends GF_EVENT_CHILD_DOWN event to parent xlatros and server notify updates the child_up to false in server_conf.When client is trying to communicate with server through mount it checks conf->child_up and it is FALSE so it throws message "translator are not yet ready". From this patch updated structure server_conf to save child_up status for xlator wise. Another improtant correction from this patch is cleanup threads from server side xlators after stop the volume. BUG: 1453977 Change-Id: Ic54da3f01881b7c9429ce92cc569236eb1d43e0d Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17356 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* features/bitrot: Fix glusterfsd crashKotresh HR2017-05-291-8/+15
| | | | | | | | | | | | | | | | | | With object versioning being optional, it can so happen the bitrot stub context is not always set. When it's not found, it's initialized. But was not being assigned to use in the local function. This was leading for brick crash. Fixed the same. Change-Id: I0dab6435cdfe16a8c7f6a31ffec1a370822597a8 BUG: 1454317 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17357 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* libglusterfs/stack.h: reduce duplication of codeAmar Tumballi2017-04-271-5/+5
| | | | | | | | | | | | | | | | * Use STACK_UNWIND_STRICT everywhere. * Provide STACK_WIND_COMMON as both STACK_WIND_COOKIE and STACK_WIND differ by just 1 line and 1 option. Updates gluster/glusterfs#137 Change-Id: Ifbb6b9c4702b02f4a02834824f509fd10c78f0ce Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/16915 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* features/bit-rot-stub: bring in optional versioningRaghavendra Bhat2017-04-183-57/+290
| | | | | | | | | | | | | | | | * As of now bit-rot-stub does versioning always. This leads lots of getxattr calls being made in lookups. So make object versioning optional. Change-Id: I83713e45ae59fb28004bb3cfa008f2d69edebbfa BUG: 1359599 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/14442 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/bit-rot-stub: use the correct spelling of quarantine for bad objectsRaghavendra Bhat2017-01-303-5/+20
| | | | | | | | | | | | | | | | container The directory for containing the list of bad objects was named "quanrantine" instead of "quarantine" Change-Id: I8c20381ac637201d9d1a224f5223e8dfbed53f1e BUG: 1401571 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: https://review.gluster.org/16027 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* build: out-of-tree builds generates files in the wrong directoryKaleb S KEITHLEY2016-09-181-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | And minor cleanup of a few of the Makefile.am files while we're at it. Rewrite the make rules to do what xdrgen does. Now we can get rid of xdrgen. Note 1. netbsd6's sed doesn't do -i. Why are we still running smoke tests on netbsd6 and not netbsd7? We barely support netbsd7 as it is. Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with libglusterfs? A cut-and-paste mistake? It has no references to symbols in libglusterfs. Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_ regex that matches the same lines as the _extended_ regex "/#(ifndef|define|endif)/". To match the extended regex sed needs to be run with -r on Linux; with -E on *BSD. However NetBSD's and FreeBSD's sed helpfully also provide -r for compatibility. Using a basic regex avoids having to use a kludge in order to run sed with the correct option on OS X. Note 4. Not copying the bit of xdrgen that inserts copyright/license boilerplate. AFAIK it's silly to pretend that machine generated files like these can be copyrighted or need license boilerplate. The XDR source files have their own copyright and license; and their copyrights are bound to be more up to date than old boilerplate inserted by a script. From what I've seen of other Open Source projects -- e.g. gcc and its C parser files generated by yacc and lex -- IIRC they don't bother to add copyright/license boilerplate to their generated files. It appears that it's a long-standing feature of make (SysV, BSD, gnu) for out-of-tree builds to helpfully pretend that the source files it can find in the VPATH "exist" as if they are in the $cwd. rpcgen doesn't work well in this situation and generates files with "bad" #include directives. E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`, you get an #include directive in the generated .c file like this: ... #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h" ... which (obviously) results in compile errors on out-of-tree build because the (generated) header file doesn't exist at that location. Compared to `rpcgen ./glusterfs3-xdr.x` where you get: ... #include "glusterfs3-xdr.h" ... Which is what we need. We have to resort to some Stupid Make Tricks like the addition of various .PHONY targets to work around the VPATH "help". Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/... looks exactly like -I$(top_srcdir)/rpc/xdr/... Don't be fooled though. And don't delete the -I$(top_builddir)/rpc/xdr/... bits Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e BUG: 1330604 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14085 Tested-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* feature/bitrot: Fix recovery of corrupted hardlinkKotresh HR2016-09-081-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When a file with hardlink is corrupted in ec volume, the recovery steps mentioned was not working. Only name and metadata was healing but not the data. Cause: The bad file marker in the inode context is not removed. Hence when self heal tries to open the file for data healing, it fails with EIO. Background: The bitrot deletes inode context during forget. Briefly, the recovery steps involves following steps. 1. Delete the entry marked with bad file xattr from backend. Delete all the hardlinks including .glusters hardlink as well. 2. Access the each hardlink of the file including original from the mount. The step 2 will send lookup to the brick where the files are deleted from backend and returns with ENOENT. On ENOENT, server xlator forgets the inode if there are no dentries associated with it. But in case hardlinks, the forget won't be called as dentries (other hardlink files) are associated with the inode. Hence bitrot stube won't delete it's context failing the data self heal. Fix: Bitrot-stub should delete the inode context on getting ENOENT during lookup. Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1 BUG: 1373520 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/15408 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* bit-rot: fix unused variable warnings/errorsKaleb S. KEITHLEY2016-08-291-7/+2
| | | | | | | | | | | | | | | | | http://review.gluster.org/14085 fixes a/the "leak" - via the generated rpc/xdr headers - of pragmas that mask these warnings. However 14085 won't pass the smoke test until all the warnings are fixed. Change-Id: I566fcbcae5aec575bfca40975b941c53546d4d97 BUG: 1369124 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/15245 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* features/bitrot: Move throttling code to libglusterfsKotresh HR2016-07-181-4/+0
| | | | | | | | | | | | | | | Since throttling is a separate feature by itself, move throttling code to libglusterfs. Change-Id: If9b99885ceb46e5b1865a4af18b2a2caecf59972 BUG: 1352019 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/14846 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* core: use readdir(3) with glibc, and associated cleanupKaleb S. KEITHLEY2016-07-181-14/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting with glibc-2.23 (i.e. what's in Fedora 25), readdir_r(3) is marked as deprecated. Specifically the function decl in <dirent.h> has the deprecated attribute, and now warnings are thrown during the compile on Fedora 25 builds. The readdir(_r)(3) man page (on Fedora 25 at least) and World+Dog say that glibc's readdir(3) is, and always has been, MT-SAFE as long as only one thread is accessing the directory object returned by opendir(). World+Dog also says there is a potential buffer overflow in readdir_r(). World+Dog suggests that it is preferable to simply use readdir(). There's an implication that eventually readdir_r(3) will be removed from glibc. POSIX has, apparently deprecated it in the standard, or even removed it entirely. Over and above that, our source near the various uses of readdir(_r)(3) has a few unsafe uses of strcpy()+strcat(). (AFAIK nobody has looked at the readdir(3) implemenation in *BSD to see if the same is true on those platforms, and we can't be sure of MacOS even though we know it's based on *BSD.) Change-Id: I5481f18ba1eebe7ee177895eecc9a80a71b60568 BUG: 1356998 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14838 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* features/bit-rot-stub: get frame->local before unwindingRaghavendra Bhat2016-03-091-3/+5
| | | | | | | | | | | | | | | | | | In bit-rot-stub, if unlink fails, then it was unwinding directly. Then it was trying to cleanup local. But local would be NULL, since it was unwinding directly without getting the value of frame->local. The NULL cleanup of local was causing the brick process to crash. Change-Id: I8544ba73b2e8dc0c50b1a53ff8027d85588d087b BUG: 1315465 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/13628 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* features/bitrot: do not remove the quarantine handle in forgetRaghavendra Bhat2016-02-291-8/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If an object is marked as bad, then an entry is corresponding to the bad object is created in the .glusterfs/quarantine directory to help scrub status. The entry name is the gfid of the corrupted object. The quarantine handle is removed in below 2 cases. 1) When protocol/server revceives the -ve lookup on an entry whose inode is there in the inode table (it can happen when the corrupted object is deleted directly from the backend for recovery purpose) it sends a forget on the inode and bit-rot-stub removes the quarantine handle in upon getting the forget. refer to the below commit f853ed9c61bf65cb39f859470a8ffe8973818868: http://review.gluster.org/12743) 2) When bit-rot-stub itself realizes that lookup on a corrupted object has failed with ENOENT. But with step1, there is a problem when the bit-rot-stub receives forget due to lru limit exceeding in the inode table. In such cases, though the corrupted object is not deleted (either from the mount point or from the backend), the handle in the quarantine directory is removed and that object is not shown in the bad objects list in the scrub status command. So it is better to follow only 2nd step (i.e. bit-rot-stub removing the handle from the quarantine directory in -ve lookups). Also the handle has to be removed when a corrupted object is unlinked from the mount point itself. Change-Id: Ibc3bbaf4bc8a5f8986085e87b729ab912cbf8cf9 BUG: 1308961 Original author: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13472 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/bitrot: add check for corrupted object in f{stat}Venky Shankar2016-01-101-32/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | Check for corrupted objects is done bt bitrot stub component for data operations and such fops are denied processing by returning EIO. These checks were not done for operations such as get/set extended attribute, stat and the likes - IOW, stub only blocked pure data operations. However, its necessary to have these checks for certain other fops, most importantly stat (and fstat). This is due to the fact that clients could possibly get stale stat information (such as size, {a,c,m}time) resulting in incorrect operation of the application that rely on these fields. Note that, the data that replication would take care of fetching good (and correct) data, but the staleness of stat information could lead to data inconsistencies (e.g., rebalance, tier). Change-Id: I5a22780373b182a13f8d2c4ca6b7d9aa0ffbfca3 BUG: 1296399 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/13120 Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/bitrot: Fail node-uuid getxattr if file is marked badKotresh HR2016-01-071-0/+22
| | | | | | | | | | | | | | | If xattr is node-uuid and the inode is marked bad, fail getxattr and fgetxattr with EIO. Returning EIO would result in AFR to choose correct node-uuid coresponding to the subvolume where the good copy of the file resides. Change-Id: I45a42ca38f8322d2b10f3c4c48dc504521162b42 BUG: 1294786 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/13116 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* build: export minimum symbols from xlators for correct resolutionKaleb S KEITHLEY2015-12-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | Revisiting http://review.gluster.org/#/c/11814/, which unintentionally introduced warnings from libtool about the xlator .so names. According to [1], the -module option must appear in the Makefile.am file(s); if -module is defined in a macro, e.g. in configure(.ac), then libtool will not recognize that this is a module and will emit a warning. [1] http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078 BUG: 1248669 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13003 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* core: add preadv, pwritev, pread, pwrite syscall wrappersKaleb S KEITHLEY2015-12-221-2/+2
| | | | | | | | | | | | | add additional system calls plus pick up a couple missed unwrapped system calls that seem to have slipped into the master branch. Change-Id: If268ccd5e9a139ac3ffd38293c67cd2f62ea5b58 BUG: 1289258 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12895 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* afr: handle bad objects during lookup/inode_refreshRavishankar N2015-12-201-0/+57
| | | | | | | | | | | | | | | | | If an object (file) is marked bad by bitrot, do not consider the brick on which the object is present as a potential read subvolume for AFR irrespective of the pending xattr values. Also do not consider the brick containing the bad object while performing afr_accuse_smallfiles(). Otherwise if the bad object's size is bigger, we may end up considering that as the source. Change-Id: I4abc68e51e5c43c5adfa56e1c00b46db22c88cf7 BUG: 1290965 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/12955 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* features/bit-rot-stub: delete the link for bad object in quarantine directoryRaghavendra Bhat2015-12-164-2/+95
| | | | | | | | | | | | | | | | | | When the bad object is deleted (as of now manually from the backend itself), along with its gfid handle, the entry for the bad object in the quarantne directory is left as it is (it also can be removed manually though). But the next lookup of the object upon not finding it in the backend, sends forget on the in-memory inode. If the stale link for the gfid still exists in the quarantine directory, bir-rot-stub will unlink the entry in its forget or in the next failed lookup on that object with errno being ENOENT. Change-Id: If84292d3e44707dfa11fa29023b3d9f691b8f0f3 BUG: 1285241 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/12743 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/bit-rot: Fix NULL dereferencePranith Kumar K2015-11-261-4/+10
| | | | | | | | | | | | | | | | | | Problem: By the time br_stub_worker is accessing this->private in it's thread, 'init' may not have set 'this->private = priv'. This leads to NULL dereference leading to brick crash. Fix: Set this->private before launching these threads. Change-Id: Ic797eb195fdd0c70d19f28d0b97bc0181fd3dd2f BUG: 1285616 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/12754 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Venky Shankar <vshankar@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-18 14:21:03 +0000