summaryrefslogtreecommitdiffstats
path: root/contrib
Commit message (Collapse)AuthorAgeFilesLines
* core: implement a global thread poolXavi Hernandez2019-02-184-0/+1534
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements a thread pool that is wait-free for adding jobs to the queue and uses a very small locked region to get jobs. This makes it possible to decrease contention drastically. It's based on wfcqueue structure provided by urcu library. It automatically enables more threads when load demands it, and stops them when not needed. There's a maximum number of threads that can be used. This value can be configured. Depending on the workload, the maximum number of threads plays an important role. So it needs to be configured for optimal performance. Currently the thread pool doesn't self adjust the maximum for the workload, so this configuration needs to be changed manually. For this reason, the global thread pool has been made optional, so that volumes can still use the thread pool provided by io-threads. To enable it for bricks, the following option needs to be set: config.global-threading = on This option has no effect if bricks are already running. A restart is required to activate it. It's recommended to also enable the following option when running bricks with the global thread pool: performance.iot-pass-through = on To enable it for a FUSE mount point, the option '--global-threading' must be added to the mount command. To change it, an umount and remount is needed. It's recommended to disable the following option when using global threading on a mount point: performance.client-io-threads = off To enable it for services managed by glusterd, glusterd needs to be started with option '--global-threading'. In this case all daemons, like self-heal, will be using the global thread pool. Currently it can only be enabled for bricks, FUSE mounts and glusterd services. The maximum number of threads for clients and bricks can be configured using the following options: config.client-threads config.brick-threads These options can be applied online and its effect is immediate most of the times. If one of them is set to 0, the maximum number of threads will be calcutated as #cores * 2. Some distributions use a very old userspace-rcu library (version 0.7) for this reason, some header files from version 0.10 have been copied into contrib/userspace-rcu and are used if the detected version is 0.7 or older. An additional change has been made to io-threads to prevent that threads are started when iot-pass-through is set. Change-Id: I09d19e246b9e6d53c6247b29dfca6af6ee00a24b updates: #532 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* timer-wheel: run the timer function outside of locked regionAmar Tumballi2019-01-091-1/+8
| | | | | | | | | Surprizingly this also reduced 80% CPU overhead 'perf record' tool reported in posix_spin_lock in a brick-mux test volume initialization process. updates: bz#1193929 Change-Id: I4e1df60d6fd094105c312df39f1527d3f07bed68 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-055-14/+14
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* multiple files: remove unndeeded memset()Yaniv Kaul2018-08-291-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a squash of multiple commits: contrib/fuse-lib/misc.c: remove unneeded memset() All flock variables are properly set, no need to memset it. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I8e0512c5a88daadb0e587f545fdb9b32ca8858a2 libglusterfs/src/{client_t|fd|inode|stack}.c: remove some memset() I don't think there's a need for any of them. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I2be9ccc3a5cb5da51a92af73488cdabd1c527f59 libglusterfs/src/xlator.c: remove unneeded memset() All xl->mem_acct members are properly set, no need to memset it. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I7f264cd47e7a06255a3f3943c583de77ae8e3147 xlators/cluster/afr/src/afr-self-heal-common.c: remove unneeded memset() Since we are going over the whole array anyway, initialize it properly, to either 1 or 0. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ied4210388976b6a7a2e91cc3de334534d6fef201 xlators/cluster/dht/src/dht-common.c: remove unneeded memset() Since we are going over the whole array anyway it is initialized properly. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Idc436d2bd0563b6582908d7cbebf9dbc66a42c9a xlators/cluster/ec/src/ec-helpers.c: remove unneeded memset() Since we are going over the whole array anyway it is initialized properly. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I81bf971f7fcecb4599e807d37f426f55711978fa xlators/mgmt/glusterd/src/glusterd-volgen.c: remove some memset() I don't think there's a need for any of them. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I476ea59ba53546b5153c269692cd5383da81ce2d xlators/mgmt/glusterd/src/glusterd-geo-rep.c: read() in 4K blocks The current 1K seems small. 4K is usually better (in Linux). Also remove a memset() that I don't think is needed between reads. Only compile-tested! Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I5fb7950c92d282948376db14919ad12e589eac2b xlators/storage/posix/src/posix-{gfid-path|inode-fd-ops}.c: remove memset() before sys_*xattr() functions. I don't see a reason to memset the array sent to the functions sys_llistxattr(), sys_lgetxattr(), sys_lgetxattr(), sys_flistxattr(), sys_fgetxattr(). (Note: it's unclear to me why we are calling sys_*txattr() functions with XATTR_VAL_BUF_SIZE-1 size instead of XATTR_VAL_BUF_SIZE ). Only compile-tested! Change-Id: Ief2103b56ba6c71e40ed343a93684eef6b771346 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* {mount-common|fusermount|mount_darwin|umountd}.c: strncpy()->sprintf(), ↵Yaniv Kaul2018-08-244-5/+7
| | | | | | | | | | | | | | | | | | | | | reduce strlen()'s strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN (sizeof() ) for const strings. Compile-tested only! Change-Id: I369209b36d8356c3fe00d32f8bf56e74cf9963db updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* contrib: Remove gf_mkostemp copied from GLibCShyamsundarR2018-08-161-107/+0
| | | | | | | | | | | | | | | | | | gf_mkostemp is borrowed from GLibC a long time back, we now have mkostemp or mkstemp alternatives with all distributions and versions that we care for. As a result removing this from the contrib directory and modifying the one instance that is still using the same. This is part of code cleanup as we cleaned up coverity SECUR_TEMP errors. Updates: bz#1193929 Change-Id: I1ad7863043cdb0845c53748f5a0522e767079130 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* contrib/xxhash: update to latest xxHash (0.6.5)Yaniv Kaul2018-08-163-471/+700
| | | | | | | | | | | | | | | | | | | | | | Update to latest xxHash, which is supposed to faster with small keys. Specifically, updated to 3064d42e7d74b0921bdd1818395d9cb37bb8976a, which is a bit higher than 0.6.5. Compiled hopefully with namespace (XXH_NAMESPACE=GF_), which allows to use XXH() funcs with no fear they'll 'leak' from our library. Only compile tested! xxhsum is modified to display messages which was conflicting with regression tests (TAP harness). So modified the gfid2path_fuse.t and gfid2path_nfs.t to adhere to that. updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I35cea5cc93f338c1023ac2c9bc6d7d13225a967b
* build: remove bundled arg-standaloneNiels de Vos2018-07-2819-6954/+0
| | | | | | | | | | | | | | libargp or argp-standalone is available on all commonly used distributions. There is no need to bundle an unmaintained version of argp-standalone in this repository anymore. FreeBSD places the argp.h file in /usr/local/include when argp-standalone is installed. This path is not added to CPPFLAGS by default, so thats done in configure.ac as well. Change-Id: I384a53ab0a008ec9d48fd83afeaf8fbc197e91ee Fixes: bz#1609337 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: remove uuid from contrib/Niels de Vos2018-07-2716-1878/+0
| | | | | | | | | | | | | | Bundling libuuid is not needed anymore, all current distributions provide it now. Some OS's provide their own uuid_*() functions in libc. These may not be fully compatible with libuuid.so found on Linux systems. In that case, either e2fsprogs-libuuid can be installed, or support for the native uuid_*() functions can be added to libglusterfs/src/compat-uuid.h. Change-Id: Icfa48caea81307a3bca549364969c2038911942b Fixes: bz#1607319 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-224-12/+12
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* contrib: Remove contrib/ipaddr-py entirely.Nigel Babu2018-07-098-3273/+0
| | | | | | | | This module is no longer being used. Fixes: bz#1597512 Change-Id: Ie5faf55c5961d9d7b5082c9c257351af712c41d7 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-131-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, and https://review.gluster.org/#/c/20207/ This patch changes uses of has_key() as suggested by the 2to3 utility. Note: Fedora packaging guidelines require explicit shebangs, so popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, zip, map, and raise have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Updates: #411 Change-Id: I79bda20f1583a0a1bb0320667498f4c137de93b3 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-071-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, and https://review.gluster.org/#/c/20104/ This patch changes uses of xrange() to range(), as suggested by the python 2to3 utility. https://www.geeksforgeeks.org/range-vs-xrange-python/ In Python 3, there is no xrange , but the range function behaves like xrange in Python 2. (My concern is that range() in python2 may behave differently until we "throw the switch" to switch to python3.) Note: Fedora packaging guidelines require explicit shebangs, so popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I16ae9f4e3a4fd02a0623fb6f9fdb7aaf65f2a8a9 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-302-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/ and https://review.gluster.org/#/c/19871/ Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, zip Note: these 2to3 fixes report no changes are necessary: exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Any 2to3 fixes not in the above two lists have more extensive changes which will follow in separate patches. most unicode changes suggested by 2to3 will need to be applied at the same time as changing the shebangs from python2 to python3. Prashanth notes that unicode strings in py2 need 'u' prefix; unicode strings in py3 3.0, 3.1, and 3.2 a 'u' prefix will throw an error, but in py3 3.3+ it is legal (or just ignored). All Linux dists we care about have 3.3 or later so we can leave 'u' prefixes on unicode strings. Change-Id: I49bba2f328b0ee24b9a8115a7183be979981563e updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* Don't use hardcoded /sbin, /usr/bin etc. paths. Fixes #1450546Niklas Hambüchen2018-05-031-4/+4
| | | | | | | | | Instead, rely on programs to be in PATH, as gluster already does in many places across its code base. Change-Id: Id21152fe42f5b67205d8f1571b0656c4d5f74246 BUG: 1450546 Signed-off-by: Niklas Hambuechen <mail@nh2.me>
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-123-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* tests: merge stdout and stderr outputMilind Changire2018-01-301-4/+4
| | | | | | | | | Add -m argument to 'prove' command-line. Fix xxhsum.c debugging output to keep 'prove' happy. Change-Id: I03485d5f5e43ae4ce454a275ec98bc18384ed00c Signed-off-by: Milind Changire <mchangir@redhat.com>
* fuse-lib: checked_return coverity fixKartik_Burmee2017-11-161-1/+1
| | | | | | | | | | | | issue: Calling "chdir" without checking return value function: gf_fuse_unmount_daemon fix: typecasted return value of function to void. Change-Id: I5f06fbe886a35c2d4c9f763eeb01771e9451f232 BUG: 789278 Signed-off-by: Kartik_Burmee <kburmee@redhat.com>
* Fix clang/gcc warning -Wformat-extra-argsMichael Scherer2017-09-071-2/+4
| | | | | | | | | | | | | | | umountd.c:59:61: warning: data argument not used by format string [-Wformat-extra-args] "Cannot access %s\n", path, strerror (errno)); Change-Id: If1622d5b806ce6795ad2f84f2f2874227811dd96 BUG: 1488906 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/18219 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Michael Scherer <misc@fedoraproject.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* contrib/fuse-lib: Fix compiler warning within switch caseAnoop C S2017-08-091-1/+3
| | | | | | | | | | | | | | | | | | | | | Following warning is shown when compiled with gcc v7: mount.c: In function ‘gf_fuse_unmount_daemon’: mount.c:98:22: warning: statement will never be executed [-Wswitch-unreachable] char c = 0; ^ CCLD fuse.la We just need to move the relevant declaration statements to that switch case where it is valid and is being used. Change-Id: I8e50cc7cfcfc3bc88218cd69abbf516c08ee1648 Updates: #259 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: https://review.gluster.org/17878 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* contrib/xxhash: Add xxhash libraryKotresh HR2017-06-303-0/+2429
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xxhash is a faster non-cryptographic hash. https://github.com/Cyan4973/xxHash Release Taken: "xxHash v0.6.2" -------------- Files added: contrib/xxhash/xxhash.c contrib/xxhash/xxhash.h contrib/xxhash/xxhsum.c Modifications to source: ------------------------ Following functions and data types got 'GF_' prefix as below to avoid any form of name collisions in future. ---- Functions ---- GF_XXH_versionNumber GF_XXH32 GF_XXH32_createState GF_XXH32_freeState GF_XXH32_copyState GF_XXH32_reset GF_XXH32_update GF_XXH32_digest GF_XXH32_canonicalFromHash GF_XXH32_hashFromCanonical GF_XXH64 GF_XXH64_createState GF_XXH64_freeState GF_XXH64_copyState GF_XXH64_reset GF_XXH64_update GF_XXH64_digest GF_XXH64_canonicalFromHash GF_XXH64_hashFromCanonical ---- Data Types ---- GF_XXH_errorcode GF_XXH32_state_t* GF_XXH32_canonical_t* GF_XXH32_hash_t GF_XXH64_state_t* GF_XXH64_canonical_t* GF_XXH64_hash_t It is linked with libglusterfs.so. A wrapper funtion is also added for the easy usage in common-utils.c. xxhash can be used for the all the usecases where a faster non-cryptographic hash is required. gfid to path infra would be using this for now. NOTE: ---- The gluster coding guidelines check is ignored as maintaining it further would be difficult. Updates: #253 Change-Id: Ib143f90d91d4ee99864a10246d5983e92900173b Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17641 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* fuse: implement "-oauto_unmount"Csaba Henk2017-05-232-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | libfuse has an auto_unmount option which, if enabled, ensures that the file system is unmounted at FUSE server termination by running a separate monitor process that performs the unmount when that occurs. (This feature would probably better be called "robust auto-unmount", as FUSE servers usually do try to unmount their file systems upon termination, it's just this mechanism is not crash resilient.) This change implements that option and behavior for glusterfs. Note that "auto unmount" (robust or not) is a leaky abstraction, as the kernel cannot guarantee that at the path where the FUSE fs is mounted is actually the toplevel mount at the time of the umount(2) call, for multiple reasons, among others, see: fuse-devel: "fuse: feasible to distinguish between umount and abort?" http://fuse.996288.n3.nabble.com/fuse-feasible-to-distinguish-between-umount-and-abort-tt14358.html https://github.com/libfuse/libfuse/issues/122 Updates #153 Change-Id: Ia4432580c9fd2c156d9c73c3a44f4bfd42437599 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: https://review.gluster.org/17230 Tested-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* libglusterfs: extract some functionality to functionsCsaba Henk2017-05-231-25/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | - code in run.c to close all file descriptors, except for specified ones is extracted to int close_fds_except (int *fdv, size_t count); - tokenizing and editing a string that consists of comma-separated tokens (as done eg. in mount_param_to_flag() of contrib/fuse/mount.c is abstacted into the following API: char *token_iter_init (char *str, char sep, token_iter_t *tit); gf_boolean_t next_token (char **tokenp, token_iter_t *tit); void drop_token (char *token, token_iter_t *tit); Updates #153 Change-Id: I7cb5bda38f680f08882e2a7ef84f9142ffaa54eb Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: https://review.gluster.org/17229 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* contrib/timerwheel: bad 32-bit, use builtin fls(), fix copyrightKaleb S. KEITHLEY2017-05-152-71/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's bad form to remove other people's copyright and license when you copy their source for your own use. Defining BITS_PER_LONG as 64 is incorrect on 32-bit platforms. The mismatch between the unsigned long of the timer and the int param to fls() means on 64-bit platforms that any bits set in the high 32-bits of the the timer are lost/ignored. gf_tw_find_last_bit() is meant to find the last bit in an array of longs. It's overkill for gluster's timerwheel where we only ever pass a single long; replacing it with a direct call to fls() which is renamed to gf_tw_fls() The timer routines are slightly modified from the kernel timer functions that first appeared circa 2.6.x in .../kernel/timer.c AFAICT. find_last_bit() comes from the (linux) kernel (.../lib/find_bit.c in 4.x kernels, .../lib/find_last_bit.c in 3.x kernels) but as noted above, it is removed with this patch. __fls() comes from the linux kernel (.../include/asm-generic/ bitops/{__fls.h,builtin-__fls.h} Restoring/updating the copyright and license to the version from the 4.x kernel find_bit.c. (timer.c does not have a license, __fls.h and builtin-__fls.h do not have a copyright or license, but the whole kernel is licensed under GPLv2 anyway.) Change-Id: I2d2defccf1ccc74f55d99e94212747a36a1dff35 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/17146 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* Revert "contrib/timerwheel: probable bug on 32-bit, use __builtin_ffs()"Shyamsundar Ranganathan2017-05-011-12/+5
| | | | | | | | | | | | | | This reverts commit c92b8347aea8ce78ca3fbc49b88f5adadc98509b. Commit is not ready for a merge! Change-Id: I3b3b52f7bfb4781dd42160e2b1059b4cdeb17956 Reviewed-on: https://review.gluster.org/17147 Tested-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* contrib/timerwheel: probable bug on 32-bit, use __builtin_ffs()Kaleb S. KEITHLEY2017-05-011-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | Simply always defining BITS_PER_LONG as 64 seems like it's almost certainly wrong on 32-bit platforms and could potentially result in incorrect results. fls and, e.g., __builtin_ffs() return the same answer for any given input, making it seem like the name fls (find last set) is a misnomer and ffs (find first set, starting from the lsb) is the more accurate name. Using __builtin_ffs() causes the compiler (in intel) to emit code with the bsf (bit scan forward) insn, which is approx 3x faster than the code in ffs(), at least on the machine I tried it on. (Even so, it takes 10M+ iterations for the speed difference to be measurable. Choosing the "faster" implementation seems like a no-brainer, even if there may not be any significant gain by doing so.) Change-Id: I1616dda1a5b76f208ba737a713877c1673131e33 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/17142 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* fuse: clean up mount flag processingCsaba Henk2017-04-274-19/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In general, when one invokes a mount helper program -- basically anything that mounts something based on its command line, so thinking of mount(8), mount.<fs-type> or fusermount, but also of FUSE servers in general, including glusterfs -- the command line arguments that are to affect mount(2) are mapped to a bitmask called the mount flags, which is passed to mount(2), so that the kernel can interpret the flag bits and adjusts properties of the mount accordingly. There is a traditional syntax for this mechanism as implemented in mount(8): one passes "-ocomma,separated,mount,options" and the individual option name strings are mapped to flag bits in mount(8). FUSE further explores this idea and typically the FUSE server command lines allow further option names to be used in the "-ooption,name,list" which are then separated from the kernel sanctioned option names (to which we'll refer as "system mount options") and are passed to a platform specific lower level fuse mount helper interface. The separation of system mount option names and FUSE specific option names is also platform specific, so the general mount interface function, which in case of glusterfs is gf_fuse_mount(), should abstract this away. Therefore we change the signature of this function from int gf_fuse_mount (const char *mountpoint, char *fsname, unsigned long mountflags, char *mnt_param, pid_t *mtab_pid, int status_fd); to int gf_fuse_mount (const char *mountpoint, char *fsname, char *mnt_param, pid_t *mtab_pid, int status_fd); and deal with flag extraction in platform specific mount code. Note that the sole purpose of the mountflags argument was to indicate read-only mounting. The other system mount option names were expected to reside in the comma-separated mnt_param string, but they were not properly processed (see the referred BUG). With the new gf_fuse_mount signature read-only mounting is to be indicated as a "ro" component in mnt_param. - For Darwin, which has a dedicated, separate gf_fuse_mount implementation, gf_fuse_mount was ignoring mountflags, so only the signature had to to be adjusted. However, as bonus, we gain read-only support for Darwin, which was missing so far, given that it was indicated via the ignored mountflags. Darwin's low level mount helper relies on the "ro" component of the option string, which agrees with the new calling convention of gf_fuse_mount. - On Linux, system mount option name handling (apart from the distinguished read-only option) used to have the inadvertent side effect of adding "nosuid,nodev" as indicated in BUG; since Ia89d975d1e27fcfa5ab2036ba546aa8fa0d2d1b0 this side effect is removed, but system mount option name handling was left broken (passing system mount options other than "ro" fails to mount). - On other platforms, system mount option name handling is broken (expect for the distinguished read-only option). As of this change, in the general (non-Darwin) implementation of gf_fuse_mount we take care of proper separation of system mount names and their conversion to mount flags. For Linux, we adopt the conversion table from FUSE upstream. For other systems we just provide a best effort to support those system mount options which are understood across all Unices (nosuid,nodev,noatime,noexec,ro). (This can be improved later to provide proper plaform support.) BUG: 1297182 Change-Id: I5d10b5df46feba7a02bf5bf1018db69e6b52260a Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: https://review.gluster.org/16313 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Amar Tumballi <amarts@redhat.com>
* cli: keep 'gluster volume status detail' consistentXavier Hernandez2017-01-191-111/+155
| | | | | | | | | | | | | | | | | | | | | | | | | | The output of the command 'gluster volume status <volname> detail' is not consistent between operating systems. On linux hosts it shows the file system type, the device name, mount options and inode size of each brick. However the same command executed on a FreeBSD host doesn't show all this information, even for bricks stored on a linux. Additionally, for hosts other than linux, this information is shown as 'N/A' many times. This has been fixed to show as much information as it can be retrieved from the operating system. The file contrib/mount/mntent.c has been mostly rewriten because it contained many errors that caused mount information to not be retrieved on some operating systems. Change-Id: Icb6e19e8af6ec82255e7792ad71914ef679fc316 BUG: 1411334 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/16371 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* fuse: limit fusermount fallback to EPERM casesCsaba Henk2017-01-171-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are two mount mechanims for fuse: 1) Call mount(2) syscall directly -- implemented by fuse_mount_sys 2) Call out to fusermount(1) helper utilty to do the mount -- implemented by fuse_mount_fusermount [Note: both libfuse and glusterfs ships a variant of this helper utility; named, respectively, fusermount and fusermount-glusterfs. The two has diverged, and are not compatible at the moment.] The intended use of 1) is privileged mounting, ie. when root is invoking the glusterfs client. (It cannot work for non-privileged users as the kernel enforces privilege for mount(2), more precisely, caller context needs CAP_SYS_ADMIN, see capabilities(7).) The intended use of 2) is unprivileged mountig, ie. when the glusterfs client is invoked by an unprivileged user. The helper utility is a setuid binary, so it can perform mount(2) on behalf of the user. The main mount routine, gf_fuse_mount, calls fuse_mount_sys first, and if that fails, tries also with fuse_mount_fusermount. This is what we call "fusermount fallback". However, in the light of the above remarks about intended use, this logic should apply if the fuse_mount_fusermount fails because of a privilege shortage, ie. with error "Operation not permitted" (errno EPERM). So far the fallback was unconditional (masking bugs of fuser_mount_sys, as it happens in referred BUG). Now we add the "errno == EPERM" condition. BUG: 1297182 Change-Id: Ia89d975d1e27fcfa5ab2036ba546aa8fa0d2d1b0 Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: http://review.gluster.org/15766 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* build: out-of-tree builds generates files in the wrong directoryKaleb S KEITHLEY2016-09-181-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | And minor cleanup of a few of the Makefile.am files while we're at it. Rewrite the make rules to do what xdrgen does. Now we can get rid of xdrgen. Note 1. netbsd6's sed doesn't do -i. Why are we still running smoke tests on netbsd6 and not netbsd7? We barely support netbsd7 as it is. Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with libglusterfs? A cut-and-paste mistake? It has no references to symbols in libglusterfs. Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_ regex that matches the same lines as the _extended_ regex "/#(ifndef|define|endif)/". To match the extended regex sed needs to be run with -r on Linux; with -E on *BSD. However NetBSD's and FreeBSD's sed helpfully also provide -r for compatibility. Using a basic regex avoids having to use a kludge in order to run sed with the correct option on OS X. Note 4. Not copying the bit of xdrgen that inserts copyright/license boilerplate. AFAIK it's silly to pretend that machine generated files like these can be copyrighted or need license boilerplate. The XDR source files have their own copyright and license; and their copyrights are bound to be more up to date than old boilerplate inserted by a script. From what I've seen of other Open Source projects -- e.g. gcc and its C parser files generated by yacc and lex -- IIRC they don't bother to add copyright/license boilerplate to their generated files. It appears that it's a long-standing feature of make (SysV, BSD, gnu) for out-of-tree builds to helpfully pretend that the source files it can find in the VPATH "exist" as if they are in the $cwd. rpcgen doesn't work well in this situation and generates files with "bad" #include directives. E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`, you get an #include directive in the generated .c file like this: ... #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h" ... which (obviously) results in compile errors on out-of-tree build because the (generated) header file doesn't exist at that location. Compared to `rpcgen ./glusterfs3-xdr.x` where you get: ... #include "glusterfs3-xdr.h" ... Which is what we need. We have to resort to some Stupid Make Tricks like the addition of various .PHONY targets to work around the VPATH "help". Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/... looks exactly like -I$(top_srcdir)/rpc/xdr/... Don't be fooled though. And don't delete the -I$(top_builddir)/rpc/xdr/... bits Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e BUG: 1330604 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14085 Tested-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* qemu-block: deprecated/defunct, remove from treeKaleb S KEITHLEY2016-03-07109-39377/+0
| | | | | | | | | | | | | | | | | qemu-block xlator is not used by anyone, or so I'm told. It's also substantially out of date. There's little reason to keep it in our sources. (And FedoraProject doesn't like bundled software either.) Change-Id: I4aeb2fdfd962ec6d93de6bae126874121272220a Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13473 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* fuse: add support for SEEK_HOLE and SEEK_DATA through lseek()Niels de Vos2016-02-101-1/+16
| | | | | | | | | | | | | | | | | | | | The Linux FUSE kernel module has gained support for passing SEEK_HOLE and SEEK_DATA on through lseek(). This can greatly improve performance when working with sparse files. Linux FUSE introduced support for lseek() with version 4.5. The commit in mainline Linux is 0b5da8db145bfd44266ac964a2636a0cf8d7c286. URL: http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/14752 Change-Id: I12496d788e59461a3023ddd30e0ea3179007f77e BUG: 1220173 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11474 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* fuse: update fuse_kernel.h to version 23Ravishankar N2016-02-061-3/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following changes were made upstream: - add FUSE_WRITEBACK_CACHE - add time_gran to fuse_init_out - add reserved space to fuse_init_out - add FATTR_CTIME - add ctime and ctimensec to fuse_setattr_in - add FUSE_RENAME2 request - add FUSE_NO_OPEN_SUPPORT flag Including these changes will make it easier to backport support for lseek(). Because the fuse_init_out structure changed its size, older versions of FUSE would fail initializing. When an older version of FUSE is detected, the fuse_init_out structure is reduced to the previous size. This is harmless, as the attributes that are not passed, are not used for earlier versions anyway. BUG: 1220173 Change-Id: I58c74e161638b2d4ce12fc91a206fdc1b96de14d Signed-off-by: Ravishankar N <ravishankar@redhat.com> [ndevos: splitted from http://review.gluster.org/11474 old version fuse_init_out size correction] Reviewed-on: http://review.gluster.org/11537 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* build: fix ecdh.h and dh.h depsMilind Changire2015-11-161-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | openssl.ecdh.h and openssl/dh.h are not available on all platforms. This patch adds check to autoconf and updates relevant source files. Add missing #include "config.h" to socket.c to make HAVE_OPENSSL_DH_H and HAVE_OPENSSL_ECDH_H macros available. Definitions for UTIME_OMIT and UTIME_NOW in contrib/qemu/util/oslib-posix.c have been selected from /usr/include/bits/stat.h on Fedora 22 SSL context options SSL_OP_NO_TICKET and SSL_OP_NO_COMPRESSION are now conditionally set by testing their presence. glusterfs.spec.in file now adds CFLAGS=-DUSE_INSECURE_OPENSSL for RHEL < 6 in the %build section. Change-Id: Ie32a950dad77bb0f09b4ba53edb3e1f3147056f3 BUG: 1258883 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/12517 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* build: Mac OS X build issues, no spinlock, need sys_lgetxattrKaleb S. KEITHLEY2015-07-051-1/+1
| | | | | | | | | | | | use regular locks, use our syscall wrappers in libglusterfs Change-Id: I7e0d00956366806af041b69b65d1f169aa0d2ae2 BUG: 1238793 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/11515 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* features/bitrot: cleanup, v2Venky Shankar2015-06-252-3/+9
| | | | | | | | | | | | | | | This patch uses "cleanup, v1" infrastrcuture to cleanup scrubber (data structures, threads, timers, etc..) on brick disconnection. Signer is not cleaned up yet: probably would be done as part of another patch. Change-Id: I78a92b8a7f02b2f39078aa9a5a6b101fc499fd70 BUG: 1231619 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/11148 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* contrib/timer-wheel: fix deadlock in del_timer()Venky Shankar2015-06-021-1/+1
| | | | | | | | | | | | | | | | | commit eaf3bfa added mod_timers() and successfully screwed up del_timer() by incorrectly wrapping it within double lock blocks. del_timer() was included before the above commit for the sake of timer API completion, thankfully noone used it till now. Change-Id: I07a454a216cf09dbb84777a23630e74a1e7f2830 BUG: 1227449 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/11050 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* contrib/timer-wheel: mod_timer() and friendsVenky Shankar2015-05-282-5/+89
| | | | | | | | | | | | | | | | | | | Couple of timer-wheel api's to modify timer expiry times: mod_timer() mod_timer_pending() Both the api's perform almost the same job with one minute difference: mod_timer_pending() modifies timer expiry only if the timer is pending (i.e. being tracked in timer-wheel). Change-Id: Iae64934854ccfd6b081b849bff998ae3c3021bac BUG: 1224596 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10892 Tested-by: NetBSD Build System Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* fuse: fix return value check for setuidPrasanna Kumar Kalever2015-05-161-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | setuid() sets the effective user ID of the calling process. If the effective UID of the caller is root, the real UID and saved set-user-ID are also set. On success, zero is returned. On error, -1 is returned, and errno is set appropriately. there are cases where setuid() can fail even when the caller is UID 0; it is a grave security error to omit checking for a failure return from setuid(). if an environment limits the number of processes a user can have, setuid() might fail if the target uid already is at the limit. Fix is to check return value of setuid. Change-Id: I7aa5ab5e347603c69dc93188417cc4f4c81ffc75 BUG: 1221490 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/10780 Reviewed-by: Prasanna Kumar Kalever Tested-by: Prasanna Kumar Kalever Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
* core: Global timer-wheelVenky Shankar2015-04-261-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Instantiate a process wide global instance of the timer wheel data structure. Spawning glusterfs* process with option arg "--global-timer-wheel" instantiates a global instance of timer-wheel under global context (->ctx). Translators can make use of this process wide instance [via a call to glusterfs_global_timer_wheel()] instead of maintaining an instance of their own and possibly consuming more memory. Linux kernel too has a single instance of timer wheel where subsystems such as IO, networking, etc.. make use of. Bitrot daemon would be early consumers of this: bitrot translator instances for multiple volumes would track objects belonging to their respective bricks in this global expiry tracking data structure. This is also a first step to move GlusterFS timer mechanism to use timer-wheel. Change-Id: Ie882df607e07acaced846ea269ebf1ece306d6ae BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/10380 Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-0411-59/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* contrib/timer-wheel: import linux kernel timer-wheelVenky Shankar2015-03-183-0/+417
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch imports timer-wheel[1] algorithm from the linux kernel (~/kernel/time/timer.c) with some modifications. Timer-wheel is an efficent way to track millions of timers for expiry. This is a variant of the simple but RAM heavy approach of having a list (timer bucket) for every future second. Timer-wheel categorizes every future second into a logarithmic array of arrays. This is done by splitting the 32 bit "timeout" value into fixed "sliced" bits, thereby each category has a fixed size array to which buckets are assigned. A classic split would be 8+6+6+6 (used in this patch) which results in 256+64+64+64 == 512 buckets. Therefore, the entire 32 bit futuristic timeouts have been mapped into 512 buckets. [ NOTE: There are other possible splits, such as "8+8+8+8", but this patch sticks to the widely used and tested default. ] Therfore, the first category "holds" timers whose expiry range is between 1..256, the next cateogry holds 257..16384, third category 16385..1048576 and so on. When timers are added, unless it's in the first category, timers with different timeouts could end up in the same bucket. This means that the timers are "partially sorted" -- sorted in their highest bits. The expiry code walks the first array of buckets and exprires any pending timers (1..256). Next, at time value 257, timers in the first bucket of the second array is "cascaded" onto the first category and timers are placed into respective buckets according to the thier timeout values. Cascading "brings down" the timers timeout to the coorect bucket of their respective category. Therefore, timers are sorted by their highest bits of the timeout value and then by the lower bits too. [1] https://lwn.net/Articles/152436/ Change-Id: I1219abf69290961ae9a3d483e11c107c5f49c4e3 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9707 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>
* build, glusterd: Add liburcu to build systemKaushal M2015-02-231-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds liburcu related checks to the build system and updates the spec file to require 'userspace-rcu'. liburcu >= 0.7 is required to build GlusterFS, but 0.8 and above is preferred. For cases when liburcu 0.7.x is the available version, some function definitions (currently just one) from liburcu-0.8.6 have been made available in /contrib/userspace-rcu/. This change was developed on the git branch at [1]. This commit is a combination of the following commits on the development branch. a5cd6bd Add userspace-rcu checks to configure.ac fe5ced3 Add URCU libs to glusterd libtool flags 1e43302 Add local definition of cds_list_add_tail_rcu for liburcu-0.7 98da755 Move local definition of cds_list_add_tail_rcu into contrib 8c44dfd Update spec file to include userspace-rcu0466e33 Rename rculist-additional.h to rculist-extra.h 947c7b3 Add rculist-extra.h to dist 19f32ad Address review comments 9605/1 [1]: https://github.com/kshlm/glusterfs/tree/urcu Change-Id: Ifbb617d0dacce8fa01214f894badb9d8cdcaf56f BUG: 1191030 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/9605 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
* Do not use umount -l on non Linux systemsEmmanuel Dreyfus2015-01-021-2/+4
| | | | | | | | | | | | | | Lazy unmount are only supported on Linux. Force umount instead, since this code path is used in emergency exit anyway. On NetBSD, just have the filesystem calling exit, the kernel will unmount. BUG: 1129939 Change-Id: If623ebf60b7a747ea7e78034b6d71ec2241dea4a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/9334 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* core: fix Ubuntu code audit (cppcheck) resultsKaleb S. KEITHLEY2014-11-251-14/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See also http://review.gluster.org/#/c/7693/, BZ 1091677 AFAICT these are false positives: [geo-replication/src/gsyncd.c:100]: (error) Memory leak: str [geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv [xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde [xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr Test program: [extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments. [tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments. the remainder are fixed with this change-set: [cli/src/cli-rpc-ops.c:8883]: (error) Possible null pointer dereference: local [cli/src/cli-rpc-ops.c:8886]: (error) Possible null pointer dereference: local [contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'. [contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'. [xlators/cluster/dht/src/dht-rebalance.c:1734]: (error) Possible null pointer dereference: ctx [xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local [xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1718]: (error) Possible null pointer dereference: command [xlators/mgmt/glusterd/src/glusterd-replace-brick.c:942]: (error) Resource leak: file [xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1026]: (error) Resource leak: file [xlators/mgmt/glusterd/src/glusterd-sm.c:249]: (error) Possible null pointer dereference: new_ev_ctx [xlators/mgmt/glusterd/src/glusterd-snapshot.c:6917]: (error) Possible null pointer dereference: volinfo [xlators/mgmt/glusterd/src/glusterd-utils.c:4517]: (error) Possible null pointer dereference: this [xlators/mgmt/glusterd/src/glusterd-utils.c:6662]: (error) Possible null pointer dereference: this [xlators/mgmt/glusterd/src/glusterd-utils.c:7708]: (error) Possible null pointer dereference: this [xlators/mount/fuse/src/fuse-bridge.c:4687]: (error) Uninitialized variable: finh [xlators/mount/fuse/src/fuse-bridge.c:3080]: (error) Possible null pointer dereference: state [xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it). [xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf Rerunning cppcheck after fixing the above: As before, test program: [extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments. [tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments. As before, false positive: [geo-replication/src/gsyncd.c:100]: (error) Memory leak: str [geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv [xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde [xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr False positive after fix: [xlators/performance/quick-read/src/quick-read.c:584]: (error) Possible null pointer dereference: iobuf Change-Id: I20e0e3ac1d600b2f2120b8d8536cd6d9e17023e8 BUG: 1109180 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/8064 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Use F_CLOSEM if availableEmmanuel Dreyfus2014-10-301-0/+7
| | | | | | | | | | | | | Use F_CLOSEM to close all file descriptors if available. BUG: 764655 Change-Id: Ib3c682825b89c163ebb152848f2533b3cb62cdce Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8379 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Lazy umount emulation: deal with stopped volumesEmmanuel Dreyfus2014-10-301-5/+11
| | | | | | | | | | | | | | | | On non Linux systems, lazy umount is emulated using contrib/umountd. It first check that the path given to unmount exists, but it should not give up on ENOTCONN as it is what happens when a volume is mounted but stopped. This lets NetBSD pass tests/bugs/bug-1049323.t BUG: 1129939 Change-Id: I3451362453607a0fd82b095a9e5aa6f63bfe869a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8991 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Do not hardcode umount(8) path, emulate lazy umountEmmanuel Dreyfus2014-10-032-0/+257
| | | | | | | | | | | | | | | | | | 1) Use a system-dependent macro for umount(8) location instead of relying on $PATH to find it, for security and portability sake. 2) Introduce gf_umount_lazy() to replace umount -l (-l for lazy) invocations, which is only supported on Linux; On Linux behavior in unchanged. On other systems, we fork an external process (umountd) that will take care of periodically attempt to unmount, and optionally rmdir. BUG: 1129939 Change-Id: Ia91167c0652f8ddab85136324b08f87c5ac1e51d Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8649 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Csaba Henk <csaba@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* porting: Remove unnecessary code from mount_darwin.cHarshavardhana2014-08-291-279/+180
| | | | | | | | | | | | | - Cleanup mount_darwin.c to make it cleaner - Restructure the code to be more readable - Avoid unnecessary delays invoking `mount_osxfusefs` Change-Id: I7f28875b0ec872a08bf8e77dfc8ebe5eca750d0e BUG: 1135348 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8564 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* osx: Enable qemu feature build on DarwinHarshavardhana2014-08-112-3/+7
| | | | | | | | | Change-Id: Ie5d437aa6c52c180fd8d54680c5f882e75c0bf7e BUG: 1089172 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8448 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-20 13:45:42 +0000