summaryrefslogtreecommitdiffstats
path: root/api
Commit message (Collapse)AuthorAgeFilesLines
* cluster/afr: Delay post-op for fsyncPranith Kumar K2020-06-081-1/+14
| | | | | | | | | | | | | | | | | | Problem: AFR doesn't delay post-op for fsync fop. For fsync heavy workloads this leads to un-necessary fxattrop/finodelk for every fsync leading to bad performance. Fix: Have delayed post-op for fsync. Add special flag in xdata to indicate that afr shouldn't delay post-op in cases where either the process will terminate or graph-switch would happen. Otherwise it leads to un-necessary heals when the graph-switch/process-termination happens before delayed-post-op completes. Fixes: #1253 Change-Id: I531940d13269a111c49e0510d49514dc169f4577 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* syncop: improve scaling and implement more toolsXavi Hernandez2020-05-131-1/+1
| | | | | | | | | | | | | | | | | | | | The current scaling of the syncop thread pool is not working properly and can leave some tasks in the run queue more time than necessary when the maximum number of threads is not reached. This patch provides a better scaling condition to react faster to pending work. Condition variables and sleep in the context of a synctask have also been implemented. Their purpose is to replace regular condition variables and sleeps that block synctask threads and prevent other tasks to be executed. The new features have been applied to several places in glusterd. Change-Id: Ic50b7c73c104f9e41f08101a357d30b95efccfbf Fixes: #1116 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* gfapi: Suspend synctasks instead of blocking themSoumya Koduri2020-04-073-2/+50
| | | | | | | | | | | | | | | | | | | | | | | | There are certain conditions which blocks the current execution thread (like waiting on mutex lock or condition variable or I/O response). In such cases, if it is a synctask thread, we should suspend the task instead of blocking it (like done in SYNCOP using synctask_yield) This is to avoid deadlock like the one mentioned below - 1) synctaskA sets fs->migration_in_progress to 1 and does I/O (LOOKUP) 2) Other synctask threads wait for fs->migration_in_progress to be reset to 0 by synctaskA and hence blocked 3) but synctaskA cannot resume as all synctask threads are blocked on (2). Note: this same approach is already used by few other components like syncbarrier etc. Change-Id: If90f870d663bb242c702a5b86ac52eeda67c6f0d Fixes: #1146 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* libgfapi: fix resource leakSunny Kumar2020-02-031-0/+1
| | | | | | | | | | This patch fixes resource leak in glfs-fops.c. CID : 1403873 : Resource leak updates: bz#789278 Change-Id: I589d4d65d2b074801f7b4b0afd7dc4607d79420e Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* multiple xlators: reduce key lengthYaniv Kaul2020-01-141-3/+1
| | | | | | | | | | | | | | | In many cases, we were freely allocating long keys with no need. Smaller char arrays are just fine almost anywhere, so just went ahead and looked where they we can use smaller ones. In some cases, annotated the functions as static and the prefixes passed as const as it was easier to read and understand. Where relevant, converted the dict functions to use known key length. Change-Id: I882ab33ea20d90b63278336cd1370c09ffdab7f2 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* api-glfs-fops: structure loggingyatipadia2020-01-147-286/+330
| | | | | | | | Convert gf_msg() to gf_smsg() Change-Id: Iceb40d60a75106c7b4b0a2487b611f7376bccc46 Updates: #657 Signed-off-by: yatipadia <ypadia@redhat.com>
* api: Fix documentation path in READMENiklas Hambüchen2019-11-131-1/+1
| | | | | | | Ref: #84 Updates: bz#1193929 Change-Id: If346dbe421d3e301ae035165b2e01ee0a7176d36 Signed-off-by: Amar Tumballi <amarts@gmail.com>
* rpc: align structsYaniv Kaul2019-10-171-8/+8
| | | | | | | | | | | | | squash tens of warnings on padding of structs in afr structures. The warnings were found by manually added '-Wpadded' to the GCC command line. Also made relevant structs and definitions static, where it was applicable. Change-Id: Ib71a7e9c6179378f072d796d11172d086c343e53 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* libgfapi : Dereference after null check - coverity fixPurna Pavan Chandra Aekkaladevi2019-09-301-1/+6
| | | | | | | | | | | | This patch fixes coverity issue with CID 1405581 A pointer 'volname' is being deferenced (by passing it as an argument to strncmp), even when it points to NULL, leading to segmentation fault. fix : Adding a NULL check condition beforehand and returning NULL Change-Id: I18685fa1cb251253996051b83b6790a5d5f0aff9 Updates: bz#789278 Signed-off-by: Purna Pavan Chandra Aekkaladevi <paekkala@redhat.com>
* gfapi: Update the version of 'glfs_h_creat_open'Soumya Koduri2019-09-274-8/+8
| | | | | | Change-Id: I43eaa53a7b8d876640d225e011b56192bba57eed Fixes: bz#1753569 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: 'glfs_h_creat_open' - new API to create handle and open fdSoumya Koduri2019-09-254-0/+147
| | | | | | | | | | | | | | | | | Right now we have two separate APIs, one - 'glfs_h_creat_handle' to create handle & another - 'glfs_h_open' to create a glfd to return to application Having two separate routines can result in access errors while trying to create and write into a read-only file. Since a fd is opened even during file/directory creation, introducing a new API to make these two operations atomic i.e, which can create both handle & fd and pass them to application Change-Id: Ibf513fcfcdad175f4d7eb6fa7a61b8feec6d33b5 Fixes: bz#1753569 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* libgfapi: return correct errno on invalid volume nameSheetal Pamecha2019-09-121-1/+14
| | | | | | | | | | | | | glfs_init when called with volume name prefixed by '/' sets errno to 0. Setting errno to EINVAL to resolve the issue. Also volname is a parameter to glfs_new. Thus, validating volname in glfs_new itself and returning EINVAL from that function fixes: bz#1507896 Change-Id: I0d4d2423e26cc07644d50ec8cce788ecc639203d Signed-off-by: Sheetal Pamecha <spamecha@redhat.com>
* api: fixing a coverity issueBarak Sason2019-08-201-2/+4
| | | | | | | | | | Added a null pointer check to avoid seg-fault CID:1403874 Updates: bz#789278 Change-Id: Ia8ff992c9f06d79647089f296afe77519fb5af2a Signed-off-by: Barak Sason <bsasonro@redhat.com>
* gfapi: provide version for statedump pathAmar Tumballi2019-08-134-4/+4
| | | | | | Updates: bz#1689097 Change-Id: I2e3c2a3aef17fda67d04fa9e604ea3f8335a2b0b Signed-off-by: Amar Tumballi <amarts@redhat.com>
* gfapi: Fix deadlock while processing upcallSoumya Koduri2019-08-021-33/+131
| | | | | | | | | | | | | | | As mentioned in bug1733166, there could be potential deadlock while processing upcalls depending on how each xlator choose to act on it. The right way of fixing such issues is to change rpc callback communication process. - https://github.com/gluster/glusterfs/issues/697 Till then, making changes in gfapi layer to avoid any I/O processing. Change-Id: I2079e95339e5d761d5060707f4555cfacab95c83 fixes: bz#1733166 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: increase function-coverageAmar Tumballi2019-07-311-0/+2
| | | | | | | | | | | | | | * Add few more mgmt functions to the coverage * While testing mgmt function, found an issue, where if the 'glfs_set_volfile_server()' is not called before calling 'glfs_unset_volfile_server()', unset would cause a crash. Null check of few variables fixes the issue, which is handled in this patch itself. * Added a test for volfile API Updates: bz#1693692 Change-Id: Iba151f8da1b64107e2f436ddbfef9da45b1c1588 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* event: rename event_XXX with gf_ prefixedXiubo Li2019-07-291-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I hit one crash issue when using the libgfapi. In the libgfapi it will call glfs_poller() --> event_dispatch() in file api/src/glfs.c:721, and the event_dispatch() is defined by libgluster locally, the problem is the name of event_dispatch() is the extremly the same with the one from libevent package form the OS. For example, if a executable program Foo, which will also use and link the libevent and the libgfapi at the same time, I can hit the crash, like: kernel: glfs_glfspoll[68486]: segfault at 1c0 ip 00007fef006fd2b8 sp 00007feeeaffce30 error 4 in libevent-2.0.so.5.1.9[7fef006ed000+46000] The link for Foo is: lib_foo_LADD = -levent $(GFAPI_LIBS) It will crash. This is because the glfs_poller() is calling the event_dispatch() from the libevent, not the libglsuter. The gfapi link info : GFAPI_LIBS = -lacl -lgfapi -lglusterfs -lgfrpc -lgfxdr -luuid If I link Foo like: lib_foo_LADD = $(GFAPI_LIBS) -levent It will works well without any problem. And if Foo call one private lib, such as handler_glfs.so, and the handler_glfs.so will link the GFAPI_LIBS directly, while the Foo won't and it will dlopen(handler_glfs.so), then the crash will be hit everytime. The link info will be: foo_LADD = -levent libhandler_glfs_LIBADD = $(GFAPI_LIBS) I can avoid the crash temporarily by linking the GFAPI_LIBS in Foo too like: foo_LADD = $(GFAPI_LIBS) -levent libhandler_glfs_LIBADD = $(GFAPI_LIBS) But this is ugly since the Foo won't use any APIs from the GFAPI_LIBS. And in some cases when the --as-needed link option is added(on many dists it is added as default), then the crash is back again, the above workaround won't work. Fixes: #699 Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa Signed-off-by: Xiubo Li <xiubli@redhat.com>
* cli: defer create_frame() (and dict creation) to later stages.Yaniv Kaul2019-07-161-4/+7
| | | | | | | | | Where possible, defer create_frame() - whenever possible, after command line verification, for example. Change-Id: Id6606e90e7ea6190f30b225c4733b229c519bb2f updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* Replace usleep() with nanosleep()Vijay Bellur2019-07-111-1/+1
| | | | | | | | | | | | | | | | | | | As usleep has been obsoleted, changed all invocations of usleep to nanosleep. From man 3 usleep: "4.3BSD, POSIX.1-2001. POSIX.1-2001 declares this function obsolete; use nanosleep(2) instead. POSIX.1-2008 removes the specification of usleep()." Added a helper function gf_nanosleep() to have a single place for handling edge cases that might arise from the conversion of usleep to nanosleep and allow the sleep to resume with right remaining value upon being interrupted. Fixes: bz#1721686 Change-Id: Ia39ab82c9e0f4669d2c00d4cdf25e38d94ef9f62 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* gfapi: set right pargfid according to parent's inodeKinglong Mee2019-06-251-0/+3
| | | | | | | | | | | EC updates parent's inode from loc, when pargfid is not the parent's inode's, a warnning message is print, and an error is returned to up xlator. [ec-helpers.c:400:ec_loc_gfid_check] 0-openfs1-disperse-0: Mismatching GFID's in loc Change-Id: Ie8036be91f039083a1e3fe17ec235a222df84b97 fixes: bz#1723280 Signed-off-by: Kinglong Mee <mijinlong@horiscale.com>
* glfs: add syscall.h after header cleanupAmar Tumballi2019-06-141-0/+1
| | | | | | | | | | in one of the recent patches, we cleaned-up the unneccesary header file includes. In the order of merging the patches, there cropped up an compile error. updates: bz#1193929 Change-Id: I2ad52aa918f9c698d5273bb293838de6dd50ac31 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* multiple files: another attempt to remove includesYaniv Kaul2019-06-142-13/+0
| | | | | | | | | | | | | | | | | | There are many include statements that are not needed. A previous more ambitious attempt failed because of *BSD plafrom (see https://review.gluster.org/#/c/glusterfs/+/21929/ ) Now trying a more conservative reduction. It does not solve all circular deps that we have, but it does reduce some of them. There is just too much to handle reasonably (dht-common.h includes dht-lock.h which includes dht-common.h ...), but it does reduce the overall number of lines of include we need to look at in the future to understand and fix the mess later one. Change-Id: I550cd001bdefb8be0fe67632f783c0ef6bee3f9f updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* gfapi: provide an api for setting statedump pathAmar Tumballi2019-06-144-0/+98
| | | | | | | | | | | | | | | Currently for an application using glfsapi to use glusterfs, when a statedump is taken, it uses /var/run/gluster dir to dump info. There can be concerns as this directory may be owned by some other user, and hence it may fail taking statedump. Such applications should have an option to use different path. This patch provides an API to do so. Updates: bz#1689097 Change-Id: I8918e002bc823d83614c972b6c738baa04681b23 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* gfapi: fix incorrect initialization of upcall syncop argumentsSoumya Koduri2019-06-141-37/+72
| | | | | | | | | | While sending upcall notifications via synctasks, the argument used to carry relevant data for these tasks is not initialized properly. This patch is to fix the same. Change-Id: I9fa8f841e71d3c37d3819fbd430382928c07176c fixes: bz#1718316 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* across: clang-scan: fix NULL dereferencing warningsAmar Tumballi2019-06-041-1/+1
| | | | | | | | | All these checks are done after analyzing clang-scan report produced by the CI job @ https://build.gluster.org/job/clang-scan updates: bz#1622665 Change-Id: I590305af4ceb779be952974b2a36066ffc4865ca Signed-off-by: Amar Tumballi <amarts@redhat.com>
* api/glfsxmp.c: minor fixesSheetal Pamecha2019-05-261-63/+236
| | | | | | | | | | | * add more fops: f{get,set,list,remove}xattr(), access(), fstat(), fsetattr(), getxattr(), lgetxattr(), llistxattr(), lsetxattr(), fgetxattr() * handle some error cases (like volume not found) Updates: #655 Change-Id: I3334bdf3090eafd83a54e1be12036ea01b181089 Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Sheetal Pamecha <spamecha@redhat.com>
* core: avoid dynamic TLS allocation when possibleXavi Hernandez2019-04-241-2/+1
| | | | | | | | | | | | | | | | | | | Some interdependencies between logging and memory management functions make it impossible to use the logging framework before initializing memory subsystem because they both depend on Thread Local Storage allocated through pthread_key_create() during initialization. This causes a crash when we try to log something very early in the initialization phase. To prevent this, several dynamically allocated TLS structures have been replaced by static TLS reserved at compile time using '__thread' keyword. This also reduces the number of error sources, making initialization simpler. Updates: bz#1193929 Change-Id: I8ea2e072411e30790d50084b6b7e909c7bb01d50 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* gfapi: Unblock epoll thread for upcall processingSoumya Koduri2019-03-291-8/+34
| | | | | | | | | | | | | | | | | | | | | With commit#ad35193,we have made changes to offload processing upcall notifications to synctask so as not to block epoll threads. However seems like the issue wasnt fully addressed. In "glfs_cbk_upcall_data" -> "synctask_new1" after creating synctask if there is no callback defined, the thread waits on synctask_join till the syncfn is finished. So that way even with those changes, epoll threads are blocked till the upcalls are processed. Hence the right fix now is to define a callback function for that synctask "glfs_cbk_upcall_syncop" so as to unblock epoll/notify threads completely and the upcall processing can happen in parallel by synctask threads. Change-Id: I4d8645e3588fab2c3ca534e0112773aaab68a5dd fixes: bz#1693575 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: add function to set client-pidRavishankar N2019-03-264-0/+26
| | | | | | | | | | | | This api offers the ability to set the pid of a client to a particular value, identical to how gluster fuse clients provide the --client-pid option. This is an internal API to be used by gluster processes only. See https://lists.gluster.org/pipermail/gluster-devel/2019-March/055925.html for more details. Currently glfsheal is the only proposed consumer. updates: bz#1689250 Change-Id: I0620be2127d79d69cdd57cffb29bba44e6e5da1f Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* rpc/transport: Missing a ref on dict while creating transport objectMohammed Rafi KC2019-03-201-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | while creating rpc_tranpsort object, we store a dictionary without taking a ref on dict but it does an unref during the cleaning of the transport object. So the rpc layer expect the caller to take a ref on the dictionary before passing dict to rpc layer. This leads to a lot of confusion across the code base and leads to ref leaks. Semantically, this is not correct. It is the rpc layer responsibility to take a ref when storing it, and free during the cleanup. I'm listing down the total issues or leaks across the code base because of this confusion. These issues are currently present in the upstream master. 1) changelog_rpc_client_init 2) quota_enforcer_init 3) rpcsvc_create_listeners : when there are two transport, like tcp,rdma. 4) quotad_aggregator_init 5) glusterd: init 6) nfs3_init_state 7) server: init 8) client:init This patch does the cleanup according to the semantics. Change-Id: I46373af9630373eb375ee6de0e6f2bbe2a677425 updates: bz#1659708 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
* Updating the glfs_setattr and glfs_fsetattr public APIs to use glfs_statArjun Sharma2019-02-272-61/+84
| | | | | | Change-Id: I0e86c8222c88b4c71087ec287ba81f8353d70822 updates: #389 Signed-off-by: Arjun <arjsharm@redhat.com>
* api: Update all future API versions to rel-6ShyamsundarR2019-02-075-75/+73
| | | | | | | | | | As release 6 is branched, all future APIs now become 6.0 This change implements the same. Change-Id: I6db368b4dc8585278ec11d4a411adcd04635de53 Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* api: bad GFAPI_4.1.6 blockKaleb S. KEITHLEY2019-01-301-2/+3
| | | | | | | | missing global: line, tabs not spaces Change-Id: Icdbc23b4e4cd608da1d764e81757201c4b1269a6 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* rpc: use address-family option from vol fileMilind Changire2019-01-221-4/+7
| | | | | | | | | | | | | | | | | This patch helps enable IPv6 connections in the cluster. The default address-family is IPv4 without using this option explicitly. When address-family is set to "inet6" in the /etc/glusterfs/glusterd.vol file, the mount command-line also needs to have -o xlator-option="transport.address-family=inet6" added to it. This option also gets added to the brick command-line. Snapshot and gfapi use-cases should also use this option to pass in the inet6 address-family. Change-Id: I97db91021af27bacb6d7578e33ea4817f66d7270 fixes: bz#1635863 Signed-off-by: Milind Changire <mchangir@redhat.com>
* gfapi : making glfs_setattr/glfs_fsetattr as publicArjun2019-01-167-64/+135
| | | | | | | | | | Initially glfs_fsetattr and glfs_setattr, both functions accepted iatt as arguements but now they accept stat and later in the function the stat is being converted to iatt so that it can be passed to syncop_fsetattr/syncop_setattr. Change-Id: I41a9e0124785a32ca19ef4d492c5ed5002e66ede updates: #389 Signed-off-by: Arjun Sharma <arjsharm@redhat.com>
* core: Resolve dict_leak at the time of destroying graphMohit Agrawal2019-01-141-2/+2
| | | | | | | | | | | | Problem: In gluster code some of the places it call's get_new_dict to create a dictionary without taking reference so at the time of dict_unref it has become a leak Solution: To resolve the same call dict_new instead of get_new_dict updates bz#1650403 Change-Id: I3ccbbf5af07079a4fa09aad2cd0458c8625b2f06 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* gfapi: update returned/callback pre/post attributes to glfs_statShyamsundarR2019-01-075-50/+159
| | | | | | | Change-Id: Ie0fe971e694101aa011d66aa496d0644669c2c5a Updates: #389 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: new api glfs_statx as linux's statxShyamsundarR2019-01-076-3/+273
| | | | | | | Change-Id: I44dd6ceef0954ae7fc13f920e84d81bbd3f6a774 Updates: #389 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Signed-off-by: ShyamsundarR <srangana@redhat.com>
* glfs-fops.c: fix the bad string length for snprintfKinglong Mee2019-01-071-2/+2
| | | | | | | | | Snprintf reserves one byte for the ending '\0'. A NAME_MAX bytes d_name is truncated to NAME_MAX - 1 bytes. Change-Id: Ic884d18cee24360e55ddb896dc587b0b74ef97fe updates: bz#1193929 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
* gfapi: nit cleanup related to releasing fs->mutex lockSoumya Koduri2018-12-311-1/+1
| | | | | | | | | | | This is follow-up patch to the comment received for - https://review.gluster.org/#/c/glusterfs/+/21882 We need not hold the fs->mutex lock to log error message. Change-Id: I29d2ea2e6cfecc3dd94982bd48f4bc9f11cc3aac fixes: bz#1660577 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* libgfapi : fix coverity issueSunny Kumar2018-12-311-2/+4
| | | | | | | | | | | | This patch fixes coverity issue in api/src/glfs-fops.c. CID: 1389247, 1389296, 1389369, 1389392. All coverity defects are of type Mixing enum types (MIXED_ENUMS). updates: bz#789278 Change-Id: I007bb317ed5f0b8ddaf94a93b3a4d02b1e74cb8d Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* gfapi: Access fs->oldvolfile under mutex lockSoumya Koduri2018-12-251-0/+6
| | | | | | | | | | | In some cases (for eg., when there are multiple RPC_CLNT_CONNECT notifications), multiple threads may fetch volfile and try to update it in 'fs' object simultaneously. Hence protect those variables' access under fs->mutex lock. Change-Id: Idaee9548560db32d83f4c04ebb1f375fee7864a9 fixes: bz#1660577 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* clang: Fix various missing checks for empty listShyamsundarR2018-12-141-16/+16
| | | | | | | | | | | | | | | | | | | | When using list_for_each_entry(_safe) functions, care needs to be taken that the list passed in are not empty, as these functions are not empty list safe. clag scan reported various points where this this pattern could be caught, and this patch fixes the same. Additionally the following changes are present in this patch, - Added an explicit op_ret setting in error case in the macro MAKE_INODE_HANDLE to address another clang issue reported - Minor refactoring of some functions in quota code, to address possible allocation failures in certain functions (which in turn cause possible empty lists to be passed around) Change-Id: I1e761a8d218708f714effb56fa643df2a3ea2cc7 Updates: bz#1622665 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* xlator: make 'xlator_api' mandatoryAmar Tumballi2018-12-131-3/+18
| | | | | | | | | | | | | | * Remove the options to load old symbol. * keep only 'xlator_api' symbol from being exported using xlator.sym * add xlator_api to all the xlators where its missing NOTE: This covers all the xlators which has at least a test case to validate its loading. If there is a translator, which doesn't have any test, then we should probably remove that from codebase. fixes: #164 Change-Id: Ibcdc8c9844cda6b4463d907a15813745d14c1ebb Signed-off-by: Amar Tumballi <amarts@redhat.com>
* copy_file_range support in GlusterFSRaghavendra Bhat2018-12-125-1/+197
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * libglusterfs changes to add new fop * Fuse changes: - Changes in fuse bridge xlator to receive and send responses * posix changes to perform the op on the backend filesystem * protocol and rpc changes for sending and receiving the fop * gfapi changes for performing the fop * tools: glfs-copy-file-range tool for testing copy_file_range fop - Although, copy_file_range support has been added to the upstream fuse kernel module, no release has been made yet of a kernel which contains the support. It is expected to come in the upcoming release of linux-4.20 So, as of now, executing copy_file_range fop on a fused based filesystem results in fuse kernel module sending read on the source fd and write on the destination fd. Therefore a small gfapi based tool has been written to be able test the copy_file_range fop. This tool is similar (in functionality) to the example program given in copy_file_range man page. So, running regular copy_file_range on a fuse mount point and running gfapi based glfs-copy-file-range tool gives some idea about how fast, the copy_file_range (or reflink) can be. On the local machine this was the result obtained. mount -t glusterfs workstation:new /mnt/glusterfs [root@workstation ~]# cd /mnt/glusterfs/ [root@workstation glusterfs]# ls file [root@workstation glusterfs]# cd [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new real 0m6.495s user 0m0.000s sys 0m1.439s [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr OPEN_SRC: opening /file is success OPEN_DST: opening /rrr is success FSTAT_SRC: fstat on /rrr is success copy_file_range successful real 0m0.309s user 0m0.039s sys 0m0.017s This tool needs following arguments 1) hostname 2) volume name 3) log file path 4) source file path (relative to the gluster volume root) 5) destination file path (relative to the gluster volume root) "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>" - Added a testcase as well to run glfs-copy-file-range tool * io-stats changes to capture the fop for profiling * NOTE: - Added conditional check to see whether the copy_file_range syscall is available or not. If not, then return ENOSYS. - Added conditional check for kernel minor version in fuse_kernel.h and fuse-bridge while referring to copy_file_range. And the kernel minor version is kept as it is. i.e. 24. Increment it in future when there is a kernel release which contains the support for copy_file_range fop in fuse kernel module. * The document which contains a writeup on this enhancement can be found at https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367 updates: #536 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-059-36/+36
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: Offload callback notifications to synctaskSoumya Koduri2018-11-262-10/+45
| | | | | | | | | | | | | | | | | | | | | | Upcall notifications are received from server via epoll and same thread is used to forward these notifications to the application. This may lead to deadlock and hang in the following scenario. Consider if as part of handling these callbacks, application has to do some operations which involve sending I/Os to gfapi stack which inturn have to wait for epoll threads to receive repsonse. Thus this may lead to deadlock if all the epoll threads are waiting to complete these callback notifications. To address it, instead of using epoll thread itself, make use of synctask to send those notificaitons to the application. Change-Id: If614e0d09246e4279b9d1f40d883a32a39c8fd90 updates: bz#1648768 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: Send fop_attr dict as part of syncop_openSoumya Koduri2018-11-171-1/+1
| | | | | | | | | | Leaseid (stored in thread locals) is sent to server via xdata. This dict variable is set but not passed as argument in glfs_h_open(). Fixed the same. Change-Id: Idd2f8a0ec184b4b6b1ad1e6e5d75df551c36a96d updates: bz#1648768 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: fix bad dict setting of lease-idKinglong Mee2018-11-081-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lease_id is a 16 bits opaque data, copying it by gf_strdup is wrong. Invalid read of size 2 at 0x483FA2F: memmove (vg_replace_strmem.c:1270) by 0xE2EF6FB: ??? (in /usr/lib64/libtirpc.so.3.0.0) by 0xE2EE047: xdr_opaque (in /usr/lib64/libtirpc.so.3.0.0) by 0x107A97DC: xdr_gfx_value (glusterfs4-xdr.c:207) by 0x107A98C0: xdr_gfx_dict_pair (glusterfs4-xdr.c:321) by 0xE2EF35E: xdr_array (in /usr/lib64/libtirpc.so.3.0.0) by 0x107A9A89: xdr_gfx_dict (glusterfs4-xdr.c:335) by 0x107AA97B: xdr_gfx_write_req (glusterfs4-xdr.c:897) by 0x107A181E: xdr_serialize_generic (xdr-generic.c:25) by 0x231044A2: client_submit_request (client.c:205) by 0x2314D3C1: client4_0_writev (client-rpc-fops_v2.c:3863) by 0x230FD5FA: client_writev (client.c:956) Address 0xad659e18 is 72 bytes inside a block of size 73 alloc'd at 0x483880B: malloc (vg_replace_malloc.c:299) by 0x106BA7EC: __gf_malloc (mem-pool.c:136) by 0x1064521E: gf_strndup (mem-pool.h:166) by 0x1064521E: gf_strdup (mem-pool.h:183) by 0x1064521E: get_fop_attr_thrd_key (glfs.c:627) by 0x1064D8E9: glfs_pwritev@@GFAPI_3.4.0 (glfs-fops.c:1154) by 0x10610C0C: glusterfs_write2 (handle.c:2092) by 0x54D30C: mdcache_write2 (mdcache_file.c:647) by 0x48A3FC: nfs4_write (nfs4_op_write.c:459) by 0x48A44D: nfs4_op_write (nfs4_op_write.c:487) by 0x4634F5: nfs4_Compound (nfs4_Compound.c:947) by 0x460155: nfs_rpc_process_request (nfs_worker_thread.c:1329) by 0x4608A3: nfs_rpc_valid_NFS (nfs_worker_thread.c:1539) by 0x488F12F: svc_vc_decode (svc_vc.c:825) Updates bz#1647651 Change-Id: Ib9fff55c897bc43c15036a869888e763df133757 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
* api: fill out attribute information if not validRaghavendra Gowdappa2018-10-171-2/+10
| | | | | | | | | | | | | | | | | translators like readdir-ahead selectively retain entry information of iatt (gfid and type) when rest of the iatt is invalidated (for write invalidating ia_size, (m)(c)times etc). Fuse-bridge uses this information and sends only entry information in readdirplus response. However such option doesn't exist in gfapi. This patch modifies gfapi to populate the stat by forcing an extra lookup. Thanks to Shyamsundar Ranganathan <srangana@redhat.com> and Prashanth Pai <ppai@redhat.com> for tests. Change-Id: Ieb5f8fc76359c327627b7d8420aaf20810e53000 Fixes: bz#1630804 Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com> Signed-off-by: Soumya Koduri <skoduri@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-19 11:49:37 +0000