summaryrefslogtreecommitdiffstats
path: root/xlators/features/changelog
Commit message (Collapse)AuthorAgeFilesLines
* python: remove shebangs of non-main scripts and make others executableNiels de Vos2018-10-021-0/+0
| | | | | | | | | | | | | | | Some of the scripts that have a #!/usr/bin/python3 shebang do not have a main() like function. These scripts will not get executed but only imported. They do not need the shebang. A few others are not installed with 'make install', but do have a main() like function. These scripts are expected to be used by developers for different tasks (mostly code generation). Marking these scripts executable to make it easier to identify them. Change-Id: I73541471deb7e0830766b804786244e73dfe4221 Updates: #411 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-1218-7832/+7531
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Land clang-format changesGluster Ant2018-09-1213-891/+859
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* core: python3Kaleb S. KEITHLEY2018-09-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* libgfchangelog: Fix changelog history APIKotresh HR2018-08-311-5/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: If requested start time and end time doesn't fall into first HTIME file, then history API fails even though continuous changelogs are avaiable for the requested range in other HTIME files. This is induced by changelog disable and enable which creates fresh HTIME index file. Cause and Analysis: Each HTIME index file represents the availability of continuous changelogs. If changelog is disabled and enabled, a new HTIME index file is created represents non availability of continuous changelogs. So as long as the requested start and end falls into single HTIME index file and not across, history API should succeed. But History API checks for the changelogs only in first HTIME index file and errors out if not available. Fix: Check in all HTIME index files for availability of continuous changelogs for requested change. fixes: bz#1622549 Change-Id: I80eeceb5afbd1b89f86a9dc4c320e161907d3559 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* changelog xlator: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-314-37/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | xlators/features/changelog/lib/src/gf-changelog-journal-handler.c xlators/features/changelog/lib/src/gf-changelog.c xlators/features/changelog/src/changelog-helpers.c xlators/features/changelog/src/changelog-misc.h strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Where possible, ensure there's no truncation of the output. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN (sizeof() ) for const strings. - switch a strncpy to a memcpy. Compile-tested only! Change-Id: Ia7a52bce0b243613ad910192ec163c93d944e077 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* features/changelog: Fix a file handle leak in htime_create()Vijay Bellur2018-08-231-2/+7
| | | | | | | | | In a few error scenarios, ht_file_hd was not being cleaned up. Addresses CID: 1325549 Change-Id: If9b4388aa700303c1eebbf1410dc35d18c4637df updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: close htime_fd in fini()Vijay Bellur2018-08-171-0/+5
| | | | | | | | Addresses CID: 1325549 Change-Id: Ib041c7c288db6810b2e13a05a19ee894a47c9b05 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: Fix missing unlocksVijay Bellur2018-08-172-4/+19
| | | | | | | | Addresses CID 1210981 Change-Id: Icd325588ae0639e09d924fdde171931dedd06ca6 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: Fix a resource leakVijay Bellur2018-08-151-0/+1
| | | | | | | | Fixes CID 1382359 Change-Id: Iaafbdb9a45496091327e3dc9092e09148fa9a5c5 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* build: rename event.h to gf-event.hNiels de Vos2018-07-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Newer FreeBSD versions (noticed with 10.3-RELEASE) provide a event.h file that on occasion gets included instead of the libglusterfs file. When this happens, 'struct event_pool' will not be defined and building will fail with errors like: autoscale-threads.c:18:55: error: incomplete definition of type 'struct event_pool' int thread_count = pool->eventthreadcount; ~~~~^ autoscale-threads.c:17:16: note: forward declaration of 'struct event_pool' struct event_pool *pool = ctx->event_pool; ^ This problem is caused by 'pkg-config --cflags uuid' that adds /usr/local/include to the GF_CPPFLAGS. The use of libuuid is preferred so that the contrib/uuid/ directory can be removed. By renaming event.h to gf-event.h there is no conflict between the different event.h files anymore and compiling on FreeBSD works without issues. Change-Id: Ie69f6b8a4f8f8e9630d39a86693eb74674f0f763 Updates: bz#1607319 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* core (named threads): flood of -Wformat-truncation warnings with gcc-7.1Kaleb S. KEITHLEY2018-07-232-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Starting in Fedora 26 which has gcc-7.1.x, -Wformat-trunction is enabled with -Wformat, resulting in a flood of new warnings. This many warnings is a concern because it makes it hard(er) to see other warnings that should be addressed. An example is at https://kojipkgs.fedoraproject.org//packages/glusterfs/3.12.0/1.fc28/data/logs/x86_64/build.log For more info see https://review.gluster.org/#/c/18267/ I can't find much (or good) documentation on the heuristics the compiler uses for this warning. In the case of printing integer types it appears it looks at the available space in the destination and the range of values for the variable and/or its type. To address the specific question about why 0x3ff versus 0xfff to mask the value, either would suffice to hint to the compiler that the printed value will fit in three characters. But the loop is from 0...1023 (or 0...0x3ff if you prefer) so I chose that as a more "accurate" mask to use as it exactly matches the range of values of the loop. Fixes: bz#1492847 Change-Id: I6e309ba42159841131d8241bfc0566ef09e00aa9
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-226-11/+11
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* Fix compile warningsXavi Hernandez2018-07-101-8/+31
| | | | | | | | | | | This patch fixes compile warnings that appear with newer compilers. The solution applied is only to remove the warnings, but it doesn't always solve the problem in the best way. It assumes that the problem will never happen, as the previous code assumed. Change-Id: I6e8470d6c2e2dbd3bd7d324b5fd2f92ffdc3d6ec updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* rpc/clnt: Don't let consumers manage "connected" stateRaghavendra G2018-06-042-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The state management of "connected" in rpc is ad-hoc as far as the responsibility goes. Note that there is nothing wrong with functionality itself. rpc layer manages this state in disconnect codepath and has exposed an api to manage this one from consumers. Note that rpc layer never sets "connected" to true by itself, which forces the consumers to use this api to get a working rpc connection. The situation is best captured from a comment in code from Jeff Darcy in glusterfsd/src/gf-attach.c: -/* - * In a sane world, the generic RPC layer would be capable of tracking - * connection status by itself, with no help from us. It might invoke our - * callback if we had registered one, but only to provide information. Sadly, - * we don't live in that world. Instead, the callback *must* exist and *must* - * call rpc_clnt_{set,unset}_connected, because that's the only way those - * fields get set (with RPC both above and below us on the stack). If we don't - * do that, then rpc_clnt_submit doesn't think we're connected even when we - * are. It calls the socket code to reconnect, but the socket code tracks this - * stuff in a sane way so it knows we're connected and returns EINPROGRESS. - * Then we're stuck, connected but unable to use the connection. To make it - * work, we define and register this trivial callback. - */ Also, consumers of rpc know about state of connection only through the notifications sent by rpc-clnt. So, consumers don't have any extra information to manage the state and hence letting them manage the state is counter intuitive. This patch cleans that up and instead moves the responsibility of state management of rpc layer into itself. Change-Id: I31e641a60795fc480ca753917f4b2579f1e05094 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Fixes: bz#1585585
* libgfchangelog: Remove duplicate includedir definition for changelog.hAnoop C S2018-06-011-1/+0
| | | | | | | | | | includedir for changelog.h is already defined in Makefile.am under libglusterfs/src since it was moved from xlators/features/changelog/lib/src. Therefore removing the duplicate definition. Change-Id: Iaff2e02fca45715820caa35b41efc2f6b656203a updates: bz#1193929 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* changelog: fix br-state-check.t failure for brick_muxMohit Agrawal2018-05-252-1/+39
| | | | | | | | | | | | | | | Problem: Sometime br-state-check.t crash while runnning for brick multiplex and command in test case is taking 2 minutes for detach a brick Solution: Update code in changelog xlator specific to wait on all connection before cleanup rpc threads and cleanup rpc object only in non brick mux scenario BUG: 1577672 Change-Id: I16e257c1e127744a815000b87bd8b7b8d9c51e1b fixes: bz#1577672 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* build: Disallow unresolved symbol referencesPrashanth Pai2018-05-181-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | In the past, it was often[1] forgotten for xlators to be linked against the symbols they refer to. This often caused glusterd2 to fail while loading xlator's shared object (.so) file. This change adds "--no-undefined" as a linker flag which causes the linker to treat unresolved symbol references as an error and hence fail linking. [1]: https://review.gluster.org/#/c/19912/ https://review.gluster.org/#/c/19664/ https://review.gluster.org/#/c/19056/ https://review.gluster.org/#/c/17659/ https://bugzilla.redhat.com/show_bug.cgi?id=1532238 Bonus: Added cloudsync and utime xlator's generated source files to .gitignore Updates: bz#1193929 Change-Id: I9604a4a87b7313a5fa43bda5fdb37dfa7ef8facd Signed-off-by: Prashanth Pai <ppai@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-021-5/+6
| | | | | | | | | | see https://review.gluster.org/#/c/19788/ use print fn from __future__ Change-Id: If5075d8d9ca9641058fbc71df8a52aa35804cda4 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* gluster: Sometimes Brick process is crashed at the time of stopping brickMohit Agrawal2018-04-192-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | Problem: Sometimes brick process is getting crashed at the time of stop brick while brick mux is enabled. Solution: Brick process was getting crashed because of rpc connection was not cleaning properly while brick mux is enabled.In this patch after sending GF_EVENT_CLEANUP notification to xlator(server) waits for all rpc client connection destroy for specific xlator.Once rpc connections are destroyed in server_rpc_notify for all associated client for that brick then call xlator_mem_cleanup for for brick xlator as well as all child xlators.To avoid races at the time of cleanup introduce two new flags at each xlator cleanup_starting, call_cleanup. BUG: 1544090 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Note: Run all test-cases in separate build (https://review.gluster.org/#/c/19700/) with same patch after enable brick mux forcefully, all test cases are passed. Change-Id: Ic4ab9c128df282d146cf1135640281fcb31997bf updates: bz#1544090
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* features/changelog: Update option levelsAravinda VK2018-04-051-0/+7
| | | | | | | | Options levels for Changelog Xlator Change-Id: Idd246717e38096c44258a990a0939f82e5fc9654 Updates: #430 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* libgfchangelog: Correct the log messageNiklas Hambüchen2018-03-241-1/+1
| | | | | | | | | | | | Provide correct error message for changelog end time check Updated error message to print "wrong result for end". Original patch by Keith Schincke <kschinck@redhat.com> from https://review.gluster.org/#/c/8121/ Change-Id: Ia3458cbac7784bfc71c05da67391a3f8259f18f0 BUG: 1559126 Signed-off-by: Niklas Hambüchen <mail@nh2.me>
* python: Remove all uses of find_library. Fixes #1450593Niklas Hambüchen2018-03-241-2/+1
| | | | | | | | `find_library()` doesn't consider LD_LIBRARY_PATH on Python < 3.6. Change-Id: Iee26085cb5d14061001f19f032c2664d69a378a8 BUG: 1450593 Signed-off-by: Niklas Hambüchen <mail@nh2.me>
* glusterfsd: Memleak in glusterfsd process while brick mux is onMohit Agrawal2018-02-273-11/+13
| | | | | | | | | | | | | | | | | | Problem: At the time of stopping the volume while brick multiplex is enabled memory is not cleanup from all server side xlators. Solution: To cleanup memory for all server side xlators call fini in glusterfs_handle_terminate after send GF_EVENT_CLEANUP notification to top xlator. BUG: 1544090 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Note: Run all test-cases in separate build (https://review.gluster.org/19574) with same patch after enable brick mux forcefully, all test cases are passed. Change-Id: Ia10dc7f2605aa50f2b90b3fe4eb380ba9299e2fc
* Revert "glusterfsd: Memleak in glusterfsd process while brick mux is on"Mohit Agrawal2018-02-192-5/+8
| | | | | | | | | | | There are still remain some code paths where cleanup is required while brick mux is on.I will upload a new patch after resolve all code paths. This reverts commit b313d97faa766443a7f8128b6e19f3d2f1b267dd. BUG: 1544090 Change-Id: I26ef1d29061092bd9a409c8933d5488e968ed90e Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* glusterfsd: Memleak in glusterfsd process while brick mux is onMohit Agrawal2018-02-152-8/+5
| | | | | | | | | | | | | Problem: At the time of stopping the volume while brick multiplex is enabled memory is not cleanup from all server side xlators. Solution: To cleanup memory for all server side xlators call fini in glusterfs_handle_terminate after send GF_EVENT_CLEANUP notification to top xlator. BUG: 1544090 Change-Id: Ifa1525e25b697371276158705026b421b4f81140 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* core: fix some of the dict_{get,set} with proper APIsAmar Tumballi2018-01-171-12/+4
| | | | | | | updates #220 Change-Id: I6e25dbb69b2c7021e00073e8f025d212db7de0be Signed-off-by: Amar Tumballi <amarts@redhat.com>
* fips: Replace md5sum usage to enable fips supportKotresh HR2017-12-191-10/+10
| | | | | | | | | | | | md5sum is not fips compliant. Using xxhash64 instead of md5sum for socket file generation in glusterd and changelog to enable fips support. NOTE: md5sum is 128 bit hash. xxhash used is 64 bit. Updates: #230 Change-Id: I1bf2ea05905b9151cd29fa951f903685ab0dc84c Signed-off-by: Kotresh HR <khiremat@redhat.com>
* all: Simplify component message id's definitionXavier Hernandez2017-12-142-706/+108
| | | | | | | | | This patch creates a new way of defining message id's that is easier and less error prone because it doesn't require so many manual changes each time a new component is defined or a new message created. Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* features/changelog: changelog threads are not cleaned properly after receive ↵Mohit Agrawal2017-12-133-12/+10
| | | | | | | | | | | | | | GF_EVENT_CLEANUP Problem: changelog threads are not cleaned properly after receive GF_EVENT_CLEANUP while brick mux is enabled Solution: change changelog notify code path to cleanup threads properly BUG: 1524816 Change-Id: I007f4e14e3d557707df738faf222b78c793d5ab5 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* glusterfs: Use gcc builtin ATOMIC operator to increase/decreate refcount.Mohit Agrawal2017-12-122-15/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: In glusterfs code base we call mutex_lock/unlock to take reference/dereference for a object.Sometime it could be reason for lock contention also. Solution: There is no need to use mutex to increase/decrease ref counter, instead of using mutex use gcc builtin ATOMIC operation. Test: I have not observed yet how much performance gain after apply this patch specific to glusterfs but i have tested same with below small program(mutex and atomic both) and get good difference. static int numOuterLoops; static void * threadFunc(void *arg) { int j; for (j = 0; j < numOuterLoops; j++) { __atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL); } return NULL; } int main(int argc, char *argv[]) { int opt, s, j; int numThreads; pthread_t *thread; int verbose; int64_t n = 0; if (argc < 2 ) { printf(" Please provide 2 args Num of threads && Outer Loop\n"); exit (-1); } numThreads = atoi(argv[1]); numOuterLoops = atoi (argv[2]); if (1) { printf("\tthreads: %d; outer loops: %d;\n", numThreads, numOuterLoops); } thread = calloc(numThreads, sizeof(pthread_t)); if (thread == NULL) { printf ("calloc error so exit\n"); exit (-1); } __atomic_store (&glob, &n, __ATOMIC_RELEASE); for (j = 0; j < numThreads; j++) { s = pthread_create(&thread[j], NULL, threadFunc, NULL); if (s != 0) { printf ("pthread_create failed so exit\n"); exit (-1); } } for (j = 0; j < numThreads; j++) { s = pthread_join(thread[j], NULL); if (s != 0) { printf ("pthread_join failed so exit\n"); exit (-1); } } printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED)); exit(0); } time ./thr_count 800 800000 threads: 800; outer loops: 800000; glob value is 640000000 real 1m10.288s user 0m57.269s sys 3m31.565s time ./thr_count_atomic 800 800000 threads: 800; outer loops: 800000; glob value is 640000000 real 0m20.313s user 1m20.558s sys 0m0.028 Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* libglusterfs: specify ctx in gf_log_set_loglevelZhang Huan2017-12-061-1/+1
| | | | | | | | | specify ctx in gf_log_set_loglevel, instead of getting it from a thread specific variable. Change-Id: I498f826e8e32231235a6b0005026a27c327727fd BUG: 1521213 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* features/changelog: Add default values for optionsKaushal M2017-11-291-0/+2
| | | | | | | Updates #302 Change-Id: I806e42b658114b242b787491400332299dbdbf77 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* features/changelog: xlators opts improvement for GD2Aravinda VK2017-11-171-9/+34
| | | | | | | Updates: #302 Change-Id: Ibbf0f99d4b81a5e9a5ccee1889214b74f083a7db Signed-off-by: Aravinda VK <avishwan@redhat.com>
* features/changelog: Structured logging supportAravinda VK2017-11-154-168/+230
| | | | | | | | | | | Structured logging infra is available in logging library with issue #240. Log messages with dynamic content are identified and converted to structured logging format(`gf_msg` to `gf_smsg` and `gf_log` to `gf_slog`) BUG: 1501054 Change-Id: I5fccc354730c07cb9ae444d0b959d1d72bd9be49 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* features/libgfchangelog: Fix coverity issuesAravinda VK2017-11-132-35/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Error: DEADCODE: xlators/features/changelog/lib/src/gf-changelog-journal-handler.c:524: dead_error_begin: Execution cannot reach this statement: "default:". Error: DEADCODE: xlators/features/changelog/lib/src/gf-history-changelog.c:984: dead_error_line: Execution cannot reach the expression "to" inside this statement: "if (!from || !to) ret = -1;". Error: REVERSE_INULL: xlators/features/changelog/lib/src/gf-changelog-journal-handler.c:678: check_after_deref: Null-checking "entry" suggests that it may be null, but it has already been dereferenced on all paths leading to the check. Error: STACK_USE: xlators/features/changelog/lib/src/gf-changelog-journal-handler.c:275: stack_use_local_overflow: Local variable "ascii" uses 12288 bytes of stack space, which exceeds the maximum single use of 10000 bytes. Error: STACK_USE: xlators/features/changelog/lib/src/gf-changelog-journal-handler.c:167: stack_use_local_overflow: Local variable "ascii" uses 12288 bytes of stack space, which exceeds the maximum single use of 10000 bytes. Error: STRING_NULL: xlators/features/changelog/lib/src/gf-changelog-journal-handler.c:589:9: string_null_sink_parm_call: Passing parameter "from_path" to "open" which expects a null-terminated string. Error: UNUSED_VALUE: xlators/features/changelog/lib/src/gf-history-changelog.c:628: assigned_value: Assigning value "-1" to "ret" here, but that stored value is overwritten before it can be used. Error: STRING_NULL: xlators/features/changelog/lib/src/gf-history-changelog.c:518: string_null_argument: Function "sys_pread" does not terminate string "*ccd->changelog". BUG: 789278 Change-Id: I1414ec6d4a118ce0c14df4dbe50995c85fe3ecf7 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* features/libgfchangelog: Structured logging supportAravinda VK2017-11-095-104/+150
| | | | | | | | | | | Structured logging infra is available in logging library with issue #240. Log messages with dynamic content are identified and converted to structured logging format(`gf_msg` to `gf_smsg` and `gf_log` to `gf_slog`) BUG: 1501054 Change-Id: I99b35396455a634f5267eb1379d481ea981e5494 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* features/changelog: Coverity Error Check ReturnVishal Pandey2017-11-081-3/+4
| | | | | | | | If condition to check error in lseek wrapper sys_lseek. Change-Id: Ic96ddbb0898146bbfae11acc86818d6604e5eb98 BUG: 789278 Signed-off-by: Vishal Pandey <vishpandey2014@gmail.com>
* rpc: optimize fop program lookupMilind Changire2017-11-061-1/+1
| | | | | | | | | | Ensure that the fop program is the first in the program list so that there's minimum amount of time spent to search the program for the most frequently needed use case. Change-Id: I45c3dcdbf39ec90ba39d914432d13a2ace00a5ee BUG: 1509647 Signed-off-by: Milind Changire <mchangir@redhat.com>
* libgfchangelog: Fix possible null pointer dereferenceKotresh HR2017-10-231-6/+6
| | | | | | | | | | If pthread_attr_init fails, gf_msg uses this->name where 'this' is not initialized yet. This patch fixes the same. Change-Id: Ie004cbe1015a0d62fc3b5512e8954c5606eeeb5f Signed-off-by: Kotresh HR <khiremat@redhat.com> BUG: 1505325
* changelog: add mem-pool initializationNiels de Vos2017-07-291-1/+9
| | | | | | | | | | | | | | | | | | | With recent changes to the mem-pool initialization, mem_pools_init_early() and mem_pools_init_late() need to be called before mem_get() is usable. This change has been tested manually with the included test from xlators/features/changelog/lib/examples/c/get-changes.c. Change-Id: I139563c4ff78a566cef2ff7e3da2ee10306def92 BUG: 1475255 Reported-by: Kotresh HR <khiremat@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17900 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* features/libgfchangelog: Fix encoding to encode only space and newlineAravinda VK2017-07-214-11/+11
| | | | | | | | | | | | | | | | | | libgfchangelog was encoding path using spec rfc3986, but encoding only required for SPACE and NEWLINE chars since the NEWLINE char is used as record separator and SPACE as field separator in the parsed changelogs output. Changed the encoding function to encode only SPACE and NEWLINE. BUG: 1451724 Change-Id: I4305459aab9e710517dd3eb065f0024503064b77 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: https://review.gluster.org/17674 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* libglusterfs: Name threads on creationRaghavendra Talur2017-07-195-15/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set names to threads on creation for easier debugging. Output of top -H -p <PID-OF-GLUSTERFSD> Before: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd After: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustertimer 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustermemsweep 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc0 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc1 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll0 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteridxwrker 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteriotwr0 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrssign 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrswrker 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterclogecon 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd0 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd1 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd2 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixjan 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixfsy 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll1 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll2 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixhc Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703 BUG: 1254002 Updates: #271 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: https://review.gluster.org/11926 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* feature/changelog: Fix buffer overflow crashKotresh HR2017-06-211-2/+4
| | | | | | | | | | | | | | | | The buffer used to hold the basename was hard coded to the size of NAME_MAX(255). It might lead to buffer overflow crashes when the basename which is sent is more than NAME_MAX length. Fixed the same. Change-Id: I6c1cad3ccaeb8c55549b1d3c5f96a198f65ba2b7 BUG: 1463178 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17579 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* glusterfs: Not able to mount running volume after enable brick mux and ↵Mohit Agrawal2017-05-312-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | stopped any volume Problem: After enabled brick mux if any volume has down and then try ot run mount with running volume , mount command is hung. Solution: After enable brick mux server has shared one data structure server_conf for all associated subvolumes.After down any subvolume in some ungraceful manner (remove brick directory) posix xlator sends GF_EVENT_CHILD_DOWN event to parent xlatros and server notify updates the child_up to false in server_conf.When client is trying to communicate with server through mount it checks conf->child_up and it is FALSE so it throws message "translator are not yet ready". From this patch updated structure server_conf to save child_up status for xlator wise. Another improtant correction from this patch is cleanup threads from server side xlators after stop the volume. BUG: 1453977 Change-Id: Ic54da3f01881b7c9429ce92cc569236eb1d43e0d Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17356 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* features/changelog: Add xattr as special type in changelog entryMohammed Rafi KC2017-05-102-11/+13
| | | | | | | | | | | | | | | | | When both SETATTR and SETXATTR fops are happening on gfid within the rollover time then, SETXATTR were not logged. In Which case we will miss the xattr fop in slave. This patch will be fix the same Change-Id: Ia75538ad1fd2797dbcf90d20dfa89f756009243d BUG: 1448914 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17205 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* Halo Replication feature for AFR translatorKevin Vigor2017-05-022-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Halo Geo-replication is a feature which allows Gluster or NFS clients to write locally to their region (as defined by a latency "halo" or threshold if you like), and have their writes asynchronously propagate from their origin to the rest of the cluster. Clients can also write synchronously to the cluster simply by specifying a halo-latency which is very large (e.g. 10seconds) which will include all bricks. In other words, it allows clients to decide at mount time if they desire synchronous or asynchronous IO into a cluster and the cluster can support both of these modes to any number of clients simultaneously. There are a few new volume options due to this feature: halo-shd-latency: The threshold below which self-heal daemons will consider children (bricks) connected. halo-nfsd-latency: The threshold below which NFS daemons will consider children (bricks) connected. halo-latency: The threshold below which all other clients will consider children (bricks) connected. halo-min-replicas: The minimum number of replicas which are to be enforced regardless of latency specified in the above 3 options. If the number of children falls below this threshold the next best (chosen by latency) shall be swapped in. New FUSE mount options: halo-latency & halo-min-replicas: As descripted above. This feature combined with multi-threaded SHD support (D1271745) results in some pretty cool geo-replication possibilities. Operational Notes: - Global consistency is gaurenteed for synchronous clients, this is provided by the existing entry-locking mechanism. - Asynchronous clients on the other hand and merely consistent to their region. Writes & deletes will be protected via entry-locks as usual preventing concurrent writes into files which are undergoing replication. Read operations on the other hand should never block. - Writes are allowed from _any_ region and propagated from the origin to all other regions. The take away from this is care should be taken to ensure multiple writers do not write the same files resulting in a gfid split-brain which will require resolution via split-brain policies (majority, mtime & size). Recommended method for preventing this is using the nfs-auth feature to define which region for each share has RW permissions, tiers not in the origin region should have RO perms. TODO: - Synchronous clients (including the SHD) should choose clients from their own region as preferred sources for reads. Most of the plumbing is in place for this via the child_latency array. - Better GFID split brain handling & better dent type split brain handling (i.e. create a trash can and move the offending files into it). - Tagging in addition to latency as a means of defining which children you wish to synchronously write to Test Plan: - The usual suspects, clang, gcc w/ address sanitizer & valgrind - Prove tests Reviewers: jackl, dph, cjh, meyering Reviewed By: meyering Subscribers: ethanr Differential Revision: https://phabricator.fb.com/D1272053 Tasks: 4117827 Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1 BUG: 1428061 Signed-off-by: Kevin Vigor <kvigor@fb.com> Reviewed-on: http://review.gluster.org/16099 Reviewed-on: https://review.gluster.org/16177 Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* core: run many bricks within one glusterfsd processJeff Darcy2017-01-303-4/+8
| | | | | | | | | | | | | | | | | | | | | | | This patch adds support for multiple brick translator stacks running in a single brick server process. This reduces our per-brick memory usage by approximately 3x, and our appetite for TCP ports even more. It also creates potential to avoid process/thread thrashing, and to improve QoS by scheduling more carefully across the bricks, but realizing that potential will require further work. Multiplexing is controlled by the "cluster.brick-multiplex" global option. By default it's off, and bricks are started in separate processes as before. If multiplexing is enabled, then *compatible* bricks (mostly those with the same transport options) will be started in the same process. Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb BUG: 1385758 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/14763 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: fix a bug in changelog_rpc_server_destroyRyan Ding2017-01-301-0/+1
| | | | | | | | | | | | | | | | | the programe unregister loop never get moved forward, it's a dead loop. to resolve it, add progs++ in the loop. Change-Id: Ib25ded10b3ce808e2fb57b13d767833f24cf01a7 BUG: 1369393 Signed-off-by: Ryan Ding <ryan.ding@open-fs.com> Reviewed-on: https://review.gluster.org/15292 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
generated by cgit (git 2.34.1) at 2024-06-09 14:29:58 +0000