summaryrefslogtreecommitdiffstats
path: root/api/src/glfs-internal.h
Commit message (Collapse)AuthorAgeFilesLines
* api: libgfapi symbol versions break LTO in Fedora rawhide/f33Kaleb S. KEITHLEY2020-07-031-6/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The way symbol versions are implemented is incompatible with gcc-10 and LTO. Fedora provenpackager Jeff Law (law [at] redhat.com) writes in the Fedora dist-git glusterfs.spec: This package uses top level ASM constructs which are incompatible with LTO. Top level ASMs are often used to implement symbol versioning. gcc-10 introduces a new mechanism for symbol versioning which works with LTO. Converting packages to use that mechanism instead of toplevel ASMs is recommended. In particular, note that the version of gluster in Fedora rawhide/f33 is glusterfs-8.0RC0. Once this fix is merged it will be necessary to backport it to the release-8 branch. At the time that gfapi symbol versions were first implemented we copied the GNU libc (glibc) symbol version implementation following Uli Drepper's symbol versioning HOWTO. Now gcc-10 has a symver attribute that can be used instead. (Maybe it has been there all along?) Both the original implemenation and this implemenation yield the same symbol versions. This can be seen by running `nm -D --with-symbol-versions libgfapi.so` on the libgfapi.so built before and after applying this fix. Change-Id: I05fda580afacfff1bfc07be810dd1afc08a92fb8 Fixes: #1352 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* syncop: improve scaling and implement more toolsXavi Hernandez2020-05-131-1/+1
| | | | | | | | | | | | | | | | | | | | The current scaling of the syncop thread pool is not working properly and can leave some tasks in the run queue more time than necessary when the maximum number of threads is not reached. This patch provides a better scaling condition to react faster to pending work. Condition variables and sleep in the context of a synctask have also been implemented. Their purpose is to replace regular condition variables and sleeps that block synctask threads and prevent other tasks to be executed. The new features have been applied to several places in glusterd. Change-Id: Ic50b7c73c104f9e41f08101a357d30b95efccfbf Fixes: #1116 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* gfapi: Suspend synctasks instead of blocking themSoumya Koduri2020-04-071-2/+32
| | | | | | | | | | | | | | | | | | | | | | | | There are certain conditions which blocks the current execution thread (like waiting on mutex lock or condition variable or I/O response). In such cases, if it is a synctask thread, we should suspend the task instead of blocking it (like done in SYNCOP using synctask_yield) This is to avoid deadlock like the one mentioned below - 1) synctaskA sets fs->migration_in_progress to 1 and does I/O (LOOKUP) 2) Other synctask threads wait for fs->migration_in_progress to be reset to 0 by synctaskA and hence blocked 3) but synctaskA cannot resume as all synctask threads are blocked on (2). Note: this same approach is already used by few other components like syncbarrier etc. Change-Id: If90f870d663bb242c702a5b86ac52eeda67c6f0d Fixes: #1146 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: add function to set client-pidRavishankar N2019-03-261-0/+6
| | | | | | | | | | | | This api offers the ability to set the pid of a client to a particular value, identical to how gluster fuse clients provide the --client-pid option. This is an internal API to be used by gluster processes only. See https://lists.gluster.org/pipermail/gluster-devel/2019-March/055925.html for more details. Currently glfsheal is the only proposed consumer. updates: bz#1689250 Change-Id: I0620be2127d79d69cdd57cffb29bba44e6e5da1f Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* api: Update all future API versions to rel-6ShyamsundarR2019-02-071-2/+2
| | | | | | | | | | As release 6 is branched, all future APIs now become 6.0 This change implements the same. Change-Id: I6db368b4dc8585278ec11d4a411adcd04635de53 Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi : making glfs_setattr/glfs_fsetattr as publicArjun2019-01-161-0/+3
| | | | | | | | | | Initially glfs_fsetattr and glfs_setattr, both functions accepted iatt as arguements but now they accept stat and later in the function the stat is being converted to iatt so that it can be passed to syncop_fsetattr/syncop_setattr. Change-Id: I41a9e0124785a32ca19ef4d492c5ed5002e66ede updates: #389 Signed-off-by: Arjun Sharma <arjsharm@redhat.com>
* gfapi: update returned/callback pre/post attributes to glfs_statShyamsundarR2019-01-071-0/+4
| | | | | | | Change-Id: Ie0fe971e694101aa011d66aa496d0644669c2c5a Updates: #389 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: new api glfs_statx as linux's statxShyamsundarR2019-01-071-0/+28
| | | | | | | Change-Id: I44dd6ceef0954ae7fc13f920e84d81bbd3f6a774 Updates: #389 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Signed-off-by: ShyamsundarR <srangana@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-051-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: Cleanup alias fileShyamsundarR2018-09-171-4/+0
| | | | | | | | | 2 extra symbols were in the alias file, that are not exported this is cleaned up with this patch. Change-Id: I1ab54b9fb6b0d455884fbbfd89820c60bb861e6f Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* Land clang-format changesGluster Ant2018-09-121-375/+397
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-221-5/+6
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* api: cleanup headersKaleb S. KEITHLEY2018-06-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | fix inconsistent use in headers of: struct glfs and glfs_t struct glfs_fd and glfs_fd_t struct glfs_object and glfs_object_t add typedefs to headers for various struct glfs_foos, and use them, e.g.: struct glfs_upcall and glfs_upcall_t fix inconsistent type naming for 'enum glfs_lock_mode_t'. (All other enum decls are 'enum glfs_foo' not 'enum_glfs_foo_t'), and add typedefs for all enums, and use them. style, replace tabs with spaces and follow return value style, i.e.: int pub_glfs_foo() instead of: int pub_glfs_foo() Update copyright dates, substantial new work has been done since 2012 Change-Id: I0a9654486ea132efde7977b655799fb37887b1d5 Updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* gfapi: various broken symbol versionsKaleb S. KEITHLEY2018-05-101-3/+12
| | | | | | | | | | | | | | | | | | | | | | lots of breakage in symbol versions: symbols added in 4.1 incorrectly, and symbols added in 4.1 but labeled 4.0.0, and symbols added in 3.13 incorrectly Not noticed until someone tried to build 3.13.2 on FreeBSD 11.1, despite the fact that we build on FreeBSD 10.3 IIRC, somehow the 3.13 errors aren't a build error there. Note: in rereading the Ulrich Drepper write-up I noticed that when a symbol version is changed, you are supposed to leave the old symbol in its original section in addition to adding it to its new section. Adding back those symbols to their original sections. Reported-by: Roman Serbski <mefystofel@gmail.com> Change-Id: I9a883546d08e0847f7228d8ea5943bc54275b319 fixes: bz#1575716 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* gfapi : RECALL_LEASE implementationSoumya Koduri2018-05-041-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Right now there are two types of upcalls * poll method * registering callback But callback can be registered per fs and same callback fn shall be used for any lease recall with object handle as argument as done for cache invalidation. TODO: RECALL LEASE for each glfd (for future reference) (may be needed fo Samba as they do not deal with object handles. In case of RECALL_LEASE, we could associate separate cbk function for each glfd either by - extending pub_glfs_lease to accept new args (recall_cbk_fn, cookie) - or by defining new API "glfs_register_recall_cbk_fn (glfd, recall_cbk_fn, cookie) . In such cases, flag it and instead of calling below upcall functions, define a new one to go through the glfd list and invoke each of theirs recall_cbk_fn. Plus added following as well * passed lease id to dict in required arguments * added flag check in pub_glfs_open Updates: #350 Change-Id: I07a971f0f26ec6aae0b9f9a5613504317dee153b Signed-off-by: Soumya Koduri <skoduri@redhat.com> Signed-off-by: Poornima G <pgurusid@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* gfapi : New APIs have been added to use lease feature in glusterSoumya Koduri2018-01-261-5/+8
| | | | | | | | | | | Following APIs glfs_h_lease(), glfs_lease() added, so that gfapi applications can set and get lease which enables more efficient client side caching. Updates: #350 Change-Id: Iede85be9af1d4df969b890d0937ed0afa4ca6596 Signed-off-by: Poornima G <pgurusid@redhat.com> Signed-off-by: Soumya Koduri <skoduri@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* gfapi: Register/Unregister Upcall events' callbackSoumya Koduri2017-10-311-0/+7
| | | | | | | | | | | | | | | | | Polling continuously for upcall events is not optimal. Hence new APIs have been added to allow applications to register and unregister upcall events it is interested in along with callback function to be invoked in case of any such upcalls sent by backend server. @TODO: Make changes in upcall xlator so that events are sent to only those clients which either registered callbacks or started polling. Shall be addressed in separate patch. Updates: #315 Change-Id: I40473fd5cf689172ff2d7bb2869756b7fd5bc761 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: set lkowner in glfdSoumya Koduri2017-10-121-0/+1
| | | | | | | | | | | | | | We need a provision to be able to set lkowner (which is used to distinguish locks maintained by server) in gfapi. Since the same lk_owner need to be used to be able to flush the lock while closing the fd, store the lkowner in the glfd structure itself. A new API has been added to be able to set lkowner in glfd. Change-Id: I67591d6b9a89c20b9617d52616513ff9e6c06b47 BUG: 1499641 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* gfapi: adds a glfs_mem_header for exported memoryKinglong Mee2017-09-011-0/+108
| | | | | | | | | | | | | | | | | | glfs_free releases different types of data depends on memory type. Drop the depends of memory type of memory accounting, new macro GLFS_CALLOC/GLFS_MALLOC/GLFS_REALLOC/GLFS_FREE are added to support assign release function dynamically, it adds a separate memory header named glfs_mem_header for gfapi. Updates: #312 Change-Id: Ie608e5227cbaa05d3f4681a515e83a50d5b17c3f Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/18092 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: Duplicate the buffer sent in setxattr callsPoornima G2017-08-071-1/+0
| | | | | | | | | | | | | | | | | | | | | Issue: The caller of glfs_setxattr sends a buffer to set as the value. We create a dict in which the pointer to the value is set. Underlying layers like md-cache take a ref on this dict to store the value for a longer time. But the moment setxattr is complete, the caller of glfs_setxattr can free the value memory. Solution: memcpy the setxattr value to the gluster buffer. Change-Id: I58753fe702e8b7d0f6c4f058714c65d0ad5d7a0a BUG: 1477488 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/17967 Reviewed-by: soumya k <skoduri@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* gfapi: mark glfs_ipc() for internal use onlyNiels de Vos2017-07-261-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | The 3.7 version of glfs_ipc() has never been used by external applications. There is little use for internal xlator communication that is triggered from outside of core GlusterFS executables. This function has now been removed from libgfapi.so. For Gluster 4.0 a new variation for glfs_ipc() has been added. The function expects dict_t parameters, which are currently not available for external applications. There is no sense in providing glfs_ipc() for non-core GlusterFS executables. Therefore, glfs_ipc() has been marked as private, and the declaration is now in the glfs-internal.h header. The Python test case (tests/features/ipctest.py) is not correct and will be re-written in C to prevent portability issues. This test is currently disabled (commit d26f0bac149d495fa93710c3f7b6b63c36cb8387). Change-Id: Idbfe35570d34d45ce8b6b43084627a552ac21f59 Fixes: #269 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17854 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* multiple: fix struct/typedef inconsistenciesJeff Darcy2017-06-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | The most common pattern, both in our code and elsewhere, is this: struct _xyz { ... }; typedef struct _xyz xyz_t; These exceptions - especially call_frame/call_stack - have been slowing down code navigation for years. By converging on a single pattern, navigating from xyz_t in code to the actual definition of struct _xyz (i.e. without having to visit the typedef first) might even be automatable. Change-Id: I0e5dd1f51f98e000173c62ef4ddc5b21d9ec44ed Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17650 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* gfapi: change root lookup from nameless to named lookupMohammed Rafi KC2017-06-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Problem: During component resolve we do lookup on root based on root gfid. If we are doing nameless lookup, then md-cache won't be able to serve those lookup. Solution: With this patch, we covert nameless lookup to named lookup on root. Credits: Poornima G <pgurusid@redhat.com> Change-Id: Ie64141936fd294d8969c39e3bd4dbc73ee375c6b BUG: 1458768 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17465 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* gfapi/handleops: Introducing glfs_xreaddirplus_r() fop for handleopsSoumya Koduri2017-05-021-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Its known that readdirplus operation fetches stat as well for each of the dirents. But often applications may need extra information, like for eg., NFS-Ganesha which operates on handles needs handles for each of those dirents returned. So this would require extra calls to the backend, in this case LOOKUP (which is very expensive operation) resulting in very low readdir performance. To address that introducing this new API using which applications can make request for any extra information to be returned as part of readdirplus response. Currently this new api returns stat and handles as demanded by application. The synopsis of the API is noted in glfs.h. @todo: * Enhance test script using this new API Below were the perf results on single brick volume with and without these changes - Dataset used - 10*100 directories and each directory containing 100 empty files. I used NFS-Ganesha application to test these changes - >for i in {1..5}; do systemctl restart nfs-ganesha; sleep 10; mount -t nfs -o vers=4 localhost:/brick_vol /mnt; cd /mnt; echo "ITERATION$i"; date; find . > tmp-nfs.log; date; cd /; umount /mnt; sleep 2; done; Without these changes - ITERATION1 Mon Mar 20 17:22:26 IST 2017 Mon Mar 20 17:23:18 IST 2017 ITERATION2 Mon Mar 20 17:23:39 IST 2017 Mon Mar 20 17:24:28 IST 2017 ITERATION3 Mon Mar 20 17:24:49 IST 2017 Mon Mar 20 17:25:36 IST 2017 ITERATION4 Mon Mar 20 17:30:57 IST 2017 Mon Mar 20 17:31:37 IST 2017 ITERATION5 Mon Mar 20 17:31:57 IST 2017 Mon Mar 20 17:32:40 IST 2017 [root@dhcp35-197 /]# On an average ~46.2 sec With these changes applied - ITERATION1 Mon Mar 20 17:35:03 IST 2017 Mon Mar 20 17:35:15 IST 2017 ITERATION2 Mon Mar 20 17:35:36 IST 2017 Mon Mar 20 17:35:46 IST 2017 ITERATION3 Mon Mar 20 17:36:06 IST 2017 Mon Mar 20 17:36:17 IST 2017 ITERATION4 Mon Mar 20 17:41:38 IST 2017 Mon Mar 20 17:41:49 IST 2017 ITERATION5 Mon Mar 20 17:42:10 IST 2017 Mon Mar 20 17:42:20 IST 2017 On an average ~10.8 sec Updates #174 BUG: 1442950 Change-Id: I0f74f74dc62085ca4c4a23c38e3edc84bd850876 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: https://review.gluster.org/15663 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: glfs_subvol_done should NOT wait for graph migration.Rajesh Joseph2016-11-291-3/+12
| | | | | | | | | | | | | | | | | | | | | | | In graph_setup function glfs_subvol_done is called which is executed in an epoll thread. glfs_lock waits on other thread to finish graph migration. This can lead to dead lock if we consume all the epoll threads. In general any call-back function executed in epoll thread should not call any blocking call which waits on a network reply either directly or indirectly, e.g. syncop functions should not be called in these threads. As a fix we should not wait for migration in the call-back path. Change-Id: If96d0689fe1b4d74631e383048cdc30b01690dc2 BUG: 1397754 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/15913 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: redesign the public interface for upcall consumersNiels de Vos2016-09-281-1/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The glfs_callback_arg and glfs_callback_inode_arg were allocated by gfapi, and expected to be free()'d by the application. However it is not reasonable to expect that applications use the same memory allocator to as the compiled libgfapi.so. For instance, it is possible that gfapi uses glibc malloc/free, and an application like NFS-Ganesha the versions from jemalloc. Mismatching of the malloc() and free() functions causes segmentation faults at best. In order to prevent problems like this in the future, the API for applications that consume upcalls has been remodeled. Any of the structures that gfapi allocates, should be free'd with glfs_free(). The members of the structures can not be accessed directly anymore, each has its own function to access now. Correcting the naming of the functions, structures and constants is a continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These new improvements not only have correct prefixes for the functions and structures, the naming also reflects more to the upcall framework and does not use "callback" anymore. Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93 BUG: 1344714 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/14701 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* md-cache: Register the list of xattrs with cache-invalidationPoornima G2016-08-301-22/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Issue: md-cache caches a specified list of xattrs, and when cache invalidation is enabled, it makes sense to recieve invalidation only when those xattrs are modified by other clients. But the current implementation of upcall is that, it will send invalidation when any of the on-disk xattrs is modified. Solution: md-cache sends a list of xattrs that it is interested in, to upcall by issuing an ipc(). The challenge here is to make sure everytime a brick goes offline and comes back up, the ipc() needs to be issued to the bricks. Hence ipc() is sent from md-cache every time there is a CHILD_UP/CHILD_MODIFIED event. TODO: There will be patches following, in cluster xlators, to implement ipc fop. Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d BUG: 1211863 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/15002 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* gfapi: do not cache upcalls if the application is not interestedNiels de Vos2016-08-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the volume option 'features.cache-invalidation' is enabled, upcall events are sent from the brick process to the client. Even if the client is not interested in upcall events itself, md-cache or other xlators may benefit from them. By adding a new 'cache_upcalls' boolean in the 'struct glfs', we can enable the caching of upcalls when the application called glfs_h_poll_upcall(). NFS-Ganesha sets up a thread for handling upcalls in the initialization phase, and calls glfs_h_poll_upcall() before any NFS-client accesses the NFS-export. In the future there will be a more flexible registration API for enabling certain kind of upcall events. Until that is available, this should work just fine. Verificatio of this change is not trivial within our current regression test framework. The bug report contains a description on how to reliably reproduce the problem with the glusterfs-coreutils. Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2 BUG: 1368842 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15191 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* gfapi: Fix IO error caused when there is consecutive graph switchesPoornima G2016-08-101-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is part 2 of the fix, the part 1 can be found at: http://review.gluster.org/#/c/14656/ Problem: ======= Consider a race between, __glfs_active_subvol() and graph_setup(). Lets say @TIME T1: fs->active_subvol = A fs->next_subvol = B __glfs_active_subvol() //under lock fs->mutex { .... new_subvol = fs->next_subvol //which is B .... //Start migration from A to B __glfs_first_lookup(){ .... unlock fs->mutex //@TIME T2 network fop lock fs->mutex .... } .... //migration continue on B fs->active_subvol = fs->next_subvol //which is C (explained below) .... } @Time T2, lets say in another thread, graph_setup() is called with C, note that at T2, fs->mutex is unlocked. graph_stup(C...) { lock fs->mutex .... if (fs->next_subvol) // which is B destroy subvol (fs->next_subvol) .... fs->next_subvol = C .... unlock fs->mutex } Thus at the end of this, fs->old_subvol = A; fs->active_subvol = C; fs->next_subvol = NULL; which is wrong, as B completed migration, but was destroyed by graph_setup, and C never was migrated. Solution: ========= Any new graph can be in one of the 2 states: - Picked for migration, migration in progress (fs->mip_subvol) - Not picked so far for migration (fs->next_subvol) graph_setup() updates fs->next_subvol only, __glfs_active_subvol() moves fs->next_subvol to fs->mip_subvol and fs->next_subvol = NULL atomically, and then once the migration is complete, make that the fs->active_subvol Change-Id: Ib6ff0565105c5eedb912a43da4017cd413243612 BUG: 1343038 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/14722 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* libgfapi/upcall : prepend "glfs_" to callback_arg, callback_inode_argJiffin Tony Thottan2016-06-101-1/+1
| | | | | | | | | | | | Change-Id: I371525775db4f6a4d69beb94baaa53d17b16fb41 BUG: 1344714 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: http://review.gluster.org/14702 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* libgfapi: glfd close is not correctly handled for async fopRajesh Joseph2016-02-101-4/+14
| | | | | | | | | | | | | | | | | | | | | | | There is chance that before the async fop is complete client can send a close. libgfapi destroys glfd on close. Therefore it can lead to crash or unexpected behaviour when the pening fop reaches libgfapi layer. Currently we don't provide any api to cancel these outstanding fops neither we check if the glfd is already closed or not. Therefore as a fix provided refcount for glfd. Each fop (sync or async) will take a ref and once the fop is complete it will unref the refcount. We should not call the registered callback function if glfd is already closed. To achieve this we maintain state of glfd so that we can safely take a call if the fd is closed or not. Change-Id: Ibe71b2225312db3f1be66b244fcf8826c70c357d BUG: 1303995 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/13340 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* api: Fix errno being set to EINVAL even on successPrashanth Pai2016-01-051-0/+2
| | | | | | | | | | | | BUG: 1289068 Change-Id: I7905ac70a537f23e1844c097a24eaa6cb762fb82 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/12909 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* libgfapi: non-default symbol version macros are incorrectKaleb S. KEITHLEY2015-08-191-2/+2
| | | | | | | | | | | | | | | default symbol versions are in form glfs_h_lookupat@@GFAPI_2.7.4, versus old, non-default versions are in the form glfs_h_lookup@GFAPI_2.4.2 I.e. "@@" versus "@" Change-Id: I88a6b129558c0b3a6064de7620b3b20425e80bc9 BUG: 1254863 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/11955 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* rpc: add owner xlator argument to rpc_clnt_newKrishnan Parthasarathi2015-08-121-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The @owner argument tells RPC layer the xlator that owns the connection and to which xlator THIS needs be set during network notifications like CONNECT and DISCONNECT. Code paths that originate from the head of a (volume) graph and use STACK_WIND ensure that the RPC local endpoint has the right xlator saved in the frame of the call (callback pair). This guarantees that the callback is executed in the right xlator context. The client handshake process which includes fetching of brick ports from glusterd, setting lk-version on the brick for the session, don't have the correct xlator set in their frames. The problem lies with RPC notifications. It doesn't have the provision to set THIS with the xlator that is registered with the corresponding RPC programs. e.g, RPC_CLNT_CONNECT event received by protocol/client doesn't have THIS set to its xlator. This implies, call(-callbacks) originating from this thread don't have the right xlator set too. The fix would be to save the xlator registered with the RPC connection during rpc_clnt_new. e.g, protocol/client's xlator would be saved with the RPC connection that it 'owns'. RPC notifications such as CONNECT, DISCONNECT, etc inherit THIS from the RPC connection's xlator. Change-Id: I9dea2c35378c511d800ef58f7fa2ea5552f2c409 BUG: 1235582 Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/11436 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* libgfapi: Gracefully exit when glfd is invalidPrashanth Pai2015-07-071-2/+2
| | | | | | | | | | | | | | | When glfs_* methods operating on glfd are invoked after calling glfs_close(), the program segfaults inside __GLFS_ENTRY_VALIDATE_FD trying to deference glfd->fd->inode which is invalid. Also, returning EBADF seemed more specific than EINVAL. BUG: 1221008 Change-Id: I13a92dca52da9a300252b69e026581b3a9e931fd Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/10759 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* build: Mac OS X build issues, gfapi symbol versionsKaleb S. KEITHLEY2015-07-041-1/+1
| | | | | | | | | | | | fix broken symbol versions Change-Id: I153b87d80895ddd4718d0a9e51b4c0712b178bd4 BUG: 1238791 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/11514 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Revert "Upcall/gfapi: Return ENOTSUP when upcall feature is disabled"Niels de Vos2015-06-281-1/+0
| | | | | | | | | | | | | | This reverts commit b68f671b2b8a0aafef8f98145aee7044edaa907d from http://review.gluster.org/11196 . The change depends on modifications to the cluster xlators, but these are still partially under review. Dropping this change now, it causes regression tests to fail. Change-Id: If5ae4a519c9c6312cdb2e2a31acce4b1901f9442 BUG: 1231132 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11452
* gfapi : symlink resolution for glfs_objectJiffin Tony Thottan2015-06-281-0/+4
| | | | | | | | | | | | | | | Generally posix expects symlink should be resolved, before performing an acl related operation. This patch introduces a new api glfs_h_resolve_symlink() which will do the same. Change-Id: Ieee645154455a732edfb2c28834021bab4248810 BUG: 1209735 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: http://review.gluster.org/11410 Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Upcall/gfapi: Return ENOTSUP when upcall feature is disabledSoumya Koduri2015-06-271-0/+1
| | | | | | | | | | | | | | Changes to detect the list of upcall events enabled using GF_FOP_IPC and return ENOTSUP to applications in case if they poll for any of the events disabled. Change-Id: Icc748054ef903598288119dbe99b1e337174662a BUG: 1231132 Signed-off-by: Niels de Vos <ndevos@redhat.com> Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/11196 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* libgfapi: introduce bit flags for pthread mutex and cond variablesHumble Devassy Chirammal2015-05-281-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | At present stage, the initialization and destroy of conditional and mutex variables of glfs struct happen in different places and there is no uniform way of destroying these variables incase there is a failure when initializing these. The fs mutex and conditional variables are getting destroyed from glfs_free_from_ctx(). However this destroy is not done by a conditional check inside destroy function. By introducing bit flags in glfs object, we can make use of the same (if there is a failure in initialization) in glfs_fini() which is evolving as one and only function to free fs and ctx resources. This patch introduce the flags field and set the flags according to the initialization flow of the mutex and conditional variables of struct glfs members. Without this patch we are compelled to use the goto lables and other hacks in init functions to make sure the resources are freed if there is an error path. Change-Id: I86e2719fb7ce437419a05699b4f06c14b02d0e69 BUG: 1208482 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/10120 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* libgfapi: Store and restore THIS in every API exposed by libgfapiPoornima G2015-05-081-12/+26
| | | | | | | | | | | | | | | | | | | | | | Storing and restoring THIS: When the APIs exposed by libgfapi are called by other xlators like snapview server etc. the THIS value is overwritten to contain the THIS of libgfapi(viz libgfapi master xlator). Hence using 'THIS' in any xlator after calling libgfapi API will lead to issues. One such issue was uncovered in snapview and the patch http://review.gluster.org/#/c/9469/ was sent to workaround this issue. Hence, storing and restoring THIS, at the entry and exit of every API exposed by libgfapi. Change-Id: I6f330dde25e7700fb26339d667a7ccd193ec6ba0 BUG: 1210934 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/9797 Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* libgfapi : anonymous fd supportJiffin Tony Thottan2015-05-051-0/+8
| | | | | | | | | | | | | | | | Anonymous fd's are floating fd assigned to a glusterfs client without a explicit file open. Here either it will create a new anonymous fd or existing anonymous fd in the client stack for requested file.The anonymous fd's are mainly used for IO's. This patch introduces two api's glfs_h_anonymous_read and glfs_h_anonymous_write which performs read and write respectively Change-Id: Id646f2220e8387b2f8bb244c848dc1db6761444f BUG: 1204651 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/9971 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* Upcall: Send stat as part of cache_invalidation notificationsSoumya Koduri2015-05-051-0/+6
| | | | | | | | | | | | | | | | Have added support to send attributes of both entries and its parent (include oldparent in case of RENAME fop) in the same notification request to avoid multiple rpc requests. Also, made changes in gfapi to send parent object and its attributes changed in a single upcall event. Change-Id: I92833da3bcec38d65216921c2ce4d10367c32ef1 BUG: 1200262 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/10460 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* Upcall: Process each of the upcall events separatelySoumya Koduri2015-04-301-14/+4
| | | | | | | | | | | | | | | | As suggested during the code-review of Bug1200262, have modified GF_CBK_UPCALL to be exlusively GF_CBK_CACHE_INVALIDATION. Thus, for any new upcall event, a new CBK procedure will be added. Also made changes to store upcall data separately based on the upcall event type received. BUG: 1200262 Change-Id: I0f5e53d6f5ece16aecb514a0a426dca40fa1c755 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/10049 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* gfapi: APIs to store and process upcall notifications receivedSoumya Koduri2015-03-171-0/+27
| | | | | | | | | | | | | | | | | | | | | | | In case of any upcall cbk events received by the protocol/client, gfapi will be notified which queues them up in a list (<gfapi_cbk_upcall>). Applicatons are responsible to provide APIs to process & notify them in case of any such upcall events queued. Added a new API which will be used by Ganesha to repeatedly poll for any such upcall event notified (<glfs_h_poll_upcall>). A new test-file has been added to test the cache_invalidation upcall events. Below link has a writeup which explains the code changes done - URL: https://soumyakoduri.wordpress.com/2015/02/25/glusterfs-understanding-upcall-infrastructure-and-cache-invalidation-support/ Change-Id: Iafc6880000c865fd4da22d0cfc388ec135b5a1c5 BUG: 1200262 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/9536 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* gfapi: improve source comments and error messages.Humble Devassy Chirammal2015-03-161-3/+3
| | | | | | | | | Change-Id: I0bfa44eb5b5f21e381af3e71c26ea863e4adc46f BUG:1202274 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/9878 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* api: versioned symbols for glfs_resolve()Ravishankar N2015-01-151-1/+1
| | | | | | | | | | | | | | | A minor fix not addressed in http://review.gluster.org/#/c/9143/. Needed by http://review.gluster.org/#/c/9377/5/ to resolve a merge conflict and for a sucessful compile without warnings about implicit declarations. Change-Id: I4aeb3b53887687096b69516c42a0888c6e137bf6 BUG: 1160709 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/9439 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* api: versioned symbols in libgfapi.so for compatibilityKaleb S. KEITHLEY2015-01-121-17/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use versioned symbols to keep libgfapi at libgfapi.so.0.0.0 Revisited to address broken build on Mac OS X See http://review.gluster.org/9036 Rebased to include http://review.gluster.org/#/c/9376/ (glfs_resolve()) but note that gerrit's "Rebase Change" couldn't do it. N.B. noticed that glfs_get_volumeid() decl in glfs.h was missing the __THROW, added it. On systems using ELF and the GNU toolchain, symbol versions are created with a .symver asm operand in the .c source file. Clang is claimed to be compatible with gcc, so we'll pretend for now that this also works with clang. On Mac OS X, aliases are created with __asm "magic" in the .h header file. In the normal case, when both the decl and defn match, that's all that's needed. In our case though the decl and defn don't match --- we have, e.g. a defn such as 'int glfs_foo(...)' and the corresponding decl is 'int pub_glfs_foo(...)'. To make this work we create the necessary aliases in the library at link time with the -alias_list link option. Note that this results in there being pairs of symbols in the .dylib, e.g. _pub_glfs_foo and _glfs_foo$GFAPI_3.4.0. We could use another link option, -unexported_symbols_list to elide the _pub_glfs_* symbols. (And we probably should.) Linux symbol versioning was essentially copied from Solaris; in general I would expect this to "just work" on Solaris, but until someone tries we don't really know. Change-Id: Icb96a3c2d80be7b6d7a6849bb9168f03a947f47c BUG: 1160709 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/9143 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* api: export glfs_resolve()Ravishankar N2015-01-061-2/+2
| | | | | | | | | | | | | | | | | glfs_resolve() is an internal function in libgfapi. Export it so that it can be used by glfs-heal.c. The change is done in line with the approach followed in commit 7e497871d11a3a527e2ce192e4274322631f27d0 (http://review.gluster.org/9036) This patch [2/3] is required as a part of afr automated split-brain resolution implementation. Change-Id: If9057f2a037fd25a0dfa231683bfbaa72d3cb1b2 BUG: 1136769 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/9376 Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Tested-by: Shyamsundar Ranganathan <srangana@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-10 06:48:11 +0000