summaryrefslogtreecommitdiffstats
path: root/rpc/rpc-transport/rdma
Commit message (Collapse)AuthorAgeFilesLines
* Revert "iobuf: Get rid of pre allocated iobuf_pool and use per thread mem pool"Amar Tumballi2019-01-082-4/+248
| | | | | | | | | This reverts commit b87c397091bac6a4a6dec4e45a7671fad4a11770. There seems to be some performance regression with the patch and hence recommended to have it reverted. Updates: #325 Change-Id: Id85d6203173a44fad6cf51d39b3e96f37afcec09
* rdma: fix possible buffer overflowRinku Kothiya2018-12-191-1/+6
| | | | | | | | | | | | | used snprintf instead of sprintf and if the source string is bigger than destination then logged a warning message. clang warning: ā€˜%sā€™ directive writing up to 1024 bytes into a region of size 108. updates: bz#1622665 Change-Id: Ia5e7c53d35d8178dd2c75708698599fe8bded5de Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
* iobuf: Get rid of pre allocated iobuf_pool and use per thread mem poolPoornima G2018-12-182-248/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current implementation of iobuf_pool has two problems: - prealloc of 12.5MB memory, this limits the scale factor of the gluster processes due to RAM requirements - lock contention, as the current implementation has one global iobuf_pool lock. Credits for debugging and addressing the same goes to Krutika Dhananjay <kdhananj@redhat.com>. Issue: #410 Hence changing the iobuf implementation to use per thread mem pool. This may theoritically appear to cause perf dip as there is no preallocation. But per thread mem pool will not have significant perf impact as the last allocated memory is kept alive for subsequent allocs, for some time. The worst case would be if iobufs requested are of random sizes each time. The best case is, if we get iobuf request of the same size. From the perf tests, this patch did not seem to cause any perf decrease. Note that, with this patch, the rdma performance is going to degrade drastically. In one of the previous patchsets we had fixes to not degrade rdma perf, but rdma is not supported and also not tested [1]. Hence the decision was to not have code in rdma that is not tested and not supported. [1] https://lists.gluster.org/pipermail/gluster-users.old/2018-July/034400.html Updates: #325 Change-Id: Ic2ef3bd498f9250dea25f25ba0c01fde19584b27 Signed-off-by: Poornima G <pgurusid@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-055-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* rpc *.h fles: align structsYaniv Kaul2018-12-031-6/+2
| | | | | | | | Make an effort to slightly better align the structures. Change-Id: I6f80a451f2ffbf15adfb986cedc24c2799787b49 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* libglusterfs multiple files: remove dead initilizationYaniv Kaul2018-11-111-8/+2
| | | | | | | | | | | | | Per newer GCC releases and clang-scan, some trivial dead initialization (values that were set but were never read) were removed. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ia9959b2ff87d2e9cb46864e68ffe7dccb984db34
* rdma: coverity fixesBhumika Goyal2018-10-211-6/+7
| | | | | | | | Fixes CID: 1382442 1382415 1382379 1382355 Change-Id: Ia712e37cb5a6db452d3178386394f87f83b85d38 updates: bz#789278 Signed-off-by: Bhumika Goyal <bgoyal@redhat.com>
* rpc: make binding to port 0 as the default if no option is providedAmar Tumballi2018-10-022-14/+14
| | | | | | | | | | | | | | | | | | | | | | | Right now, if no option is provided, the default port is assumed, which is 24007. Ideally, for 'glusterfsd' processes, it is better to not assume there are any ports given, so it can start listening on any port which is available. This helps us to cleanup the dependencies on glusterd from glusterfsd at the moment. No changes would be done to glusterd code, but making the right defaults helps to make glusterfsd more independent process later. NOTE: This patch is a reduced version of below set of patches: * https://review.gluster.org/14613/ & * https://review.gluster.org/14670/ & * https://review.gluster.org/14671/ Credits: Prasanna Kumar Kalever <pkalever@redhat.com> updates: bz#1343926 Change-Id: Ib874e10505e7366dc56ba754458252b67052e653 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-122-4652/+4441
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Land clang-format changesGluster Ant2018-09-123-347/+297
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* All: remove memset() before sprintf()Yaniv Kaul2018-08-141-1/+0
| | | | | | | | | | | | It's not needed. There's a good chance the compiler is smart enough to remove it anyway, but it can't hurt - I hope. Compile-tested only! Change-Id: Id7c054e146ba630227affa591007803f3046416b updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* build: rename event.h to gf-event.hNiels de Vos2018-07-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Newer FreeBSD versions (noticed with 10.3-RELEASE) provide a event.h file that on occasion gets included instead of the libglusterfs file. When this happens, 'struct event_pool' will not be defined and building will fail with errors like: autoscale-threads.c:18:55: error: incomplete definition of type 'struct event_pool' int thread_count = pool->eventthreadcount; ~~~~^ autoscale-threads.c:17:16: note: forward declaration of 'struct event_pool' struct event_pool *pool = ctx->event_pool; ^ This problem is caused by 'pkg-config --cflags uuid' that adds /usr/local/include to the GF_CPPFLAGS. The use of libuuid is preferred so that the contrib/uuid/ directory can be removed. By renaming event.h to gf-event.h there is no conflict between the different event.h files anymore and compiling on FreeBSD works without issues. Change-Id: Ie69f6b8a4f8f8e9630d39a86693eb74674f0f763 Updates: bz#1607319 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-222-2/+2
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* build: Link libgfrpc within rpc-transport shared librariesAnoop C S2018-01-111-2/+4
| | | | | | | | | | | | | | | | | Now with https://review.gluster.org/#/c/19056/ merged, we perform dlopen() with RTLD_LOCAL so that every shared library is expected to be explicitly linked against other required libraries. "undefined symbol ..." was seen while trying to access GlusterFS volumes via NFS-Ganesha/Samba using libgfapi resulting in an unusable state. This is a follow-up patch to https://review.gluster.org/#/c/17659/ to make sure that we link libgfrpc too while making socket and rdma rpc-transport shared libraries. Change-Id: I9943cdc449c257ded3cb9f9f2becdd5784d1d82d BUG: 1532238 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* rpc-transport/rdma: Fix coverity issues in rdma transportYi Wang2017-12-281-32/+34
| | | | | | | | | | | | | | | | Issues: [1] https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-12-25-0bc22bef/cov-errors.txt Resolved: [1] rdma.c:128 var_deref_op: Dereferencing null pointer "post". [2] rdma.c:677: Potentially overflowing expression [3] rdma.c:4250: freed_arg: "rpc_transport_unref" frees "peer->trans". [4] rdma.c:4644: var_deref_op: Dereferencing null pointer "rdma_ctx". [5] rdma.c:4945: cond_false: Condition "rdma_ctx != NULL", taking false branch. Change-Id: Iec38f118d645df4131739da412a6c741ebbd2f85 BUG: 789278 Signed-off-by: Yi Wang <wangyi@storswift.com>
* all: Simplify component message id's definitionXavier Hernandez2017-12-141-192/+87
| | | | | | | | | This patch creates a new way of defining message id's that is easier and less error prone because it doesn't require so many manual changes each time a new component is defined or a new message created. Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) accessYi Wang2017-12-122-38/+72
| | | | | | | | | | | | | | | Problem: gf_rdma_device_t->all_mr is a __gf_rdma_arena_mr(includes MR content) kind of list in the rdma rpc-transport. The rdma rpc-transport will add/delete items to the list when MRs register, deregister, and free. Because gf_rdma_device_t->all_mr is used by different threads and it is not mutex protected, rdma transport maybe access obsolete items in it. Solution: Add a mutex protection for the gf_rdma_device_t->all_mr. Change-Id: I2b7de0f7aa516b90bb6f3c6aae3aadd23b243900 BUG: 1522651 Signed-off-by: Yi Wang <wangyi@storswift.com>
* rpc/rpc-transport: dead code coverity fixKartik_Burmee2017-11-211-3/+0
| | | | | | | | | | | | issue: Execution cannot reach this statement: "iobuf_unref(iobuf);" function: gf_rdma_do_reads fix: removed the statement and the corresponding 'if' block Change-Id: Ia21b872996dbc8e0a66c8c28dd90b367d3c9aadd BUG: 789278 Signed-off-by: Kartik_Burmee <kburmee@redhat.com>
* rpc/rpc-transport/: MISSING_BREAK in name.cGirjesh Rajoria2017-11-081-2/+2
| | | | | | | | | | | | | Coverity ID: 286, 287, 288, 289, 290 Issue: Event unterminated_case: The case for value "27" is not terminated by a 'break' statement. Added Fall through in comment between switch cases. Change-Id: I9e3f205c05ac22a85a87423ef690fc18891eaece BUG: 789278 Signed-off-by: Girjesh Rajoria <grajoria@redhat.com>
* rpc/rpc-transport/rdma/: Coverity Issue MISSING_BREAKGirjesh Rajoria2017-11-061-2/+2
| | | | | | | | | | | | | Issue: Event unterminated_case: The case for value "27" is not terminated by a 'break' statement. Function: gf_rdma_get_transport_identifiers, gf_rdma_server_get_local_sockaddr Added Fall through in comment after case AF_INET_SDP. Change-Id: Ia30d0143a63a63af2b4b1d9b02551b9260444f35 BUG: 789278 Signed-off-by: Girjesh Rajoria <grajoria@redhat.com>
* Fix use-after-free in gf_rdma_do_readsMichael Scherer2017-09-151-0/+1
| | | | | | | | | | | | | | | | If iobref_new can't allocate a ref, we free the iobuf, and then go to the cleanup part of the function, that will run iobuf_unref a 2nd time, which trigger a warning with coverity. Change-Id: Ie9cf7a5d5f98244a390e44e1403c614199eb650c BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/18245 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* libglusterfs: Name threads on creationRaghavendra Talur2017-07-191-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set names to threads on creation for easier debugging. Output of top -H -p <PID-OF-GLUSTERFSD> Before: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd After: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustertimer 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustermemsweep 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc0 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc1 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll0 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteridxwrker 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteriotwr0 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrssign 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrswrker 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterclogecon 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd0 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd1 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd2 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixjan 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixfsy 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll1 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll2 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixhc Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703 BUG: 1254002 Updates: #271 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: https://review.gluster.org/11926 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* protocol/server: make listen backlog value as configurableMohammed Rafi KC2017-06-082-1/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | problem: When we call listen from protocol/server, we are giving a hard coded valie of 10 if it is not manually given. With multiplexing, especially when glusterd restarts all clients may try to connect to the server at a time. Which will result in overflowing the queue, and kernel will complain about the errors. Solution: This patch will introduce a volume set command to make backlog value as a configurable. This patch also changes the default values for backlog from 10 to 128. This changes is only applicable for sockets listening from protocol. Example: gluster volume set <volname> transport.listen-backlog 1024 Note: 1 Brick has to be restarted to get this value in effect 2 This changes won't be reflected in glusterd, or other xlators which calls listen. If you need, you have to add this option to the volfile. Change-Id: I0c5a2bbf28b5db612f9979e7560e05dd82b41477 BUG: 1456405 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17411 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* rpc: fix a routine to destory RDMA qp(queue-pair)Ji-Hyeon Gim2017-05-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Problem: If an error has occured with rdma_create_id() in gf_rdma_connect(), process will jump to the 'unlock' label and then call gf_rdma_teardown() which call __gf_rdma_teardown(). Presently, __gf_rdma_teardown() checks InifiniBand QP with peer->cm_id->qp! Unfortunately, cm_id is not allocated and will be crushed in this situation :) Solution: If 'this->private->peer->cm_id' member is null, do not check 'this->private->peer->cm_id->qp'. Change-Id: Ie321b8cf175ef4f1bdd9733d73840f03ddff8c3b BUG: 1449495 Signed-off-by: Ji-Hyeon Gim <potatogim@potatogim.net> Reviewed-on: https://review.gluster.org/17249 Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Ji-Hyeon Gim CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* transport: allow OS to assign us a portKevin Vigor2017-03-121-0/+5
| | | | | | | | | | | | | | | | Replace complex and slow port selection code with bind(0) which already respects privileged ports. Change-Id: I408a8528e58e1aafcd32eba6a8f1a759e0bf274e BUG: 1405628 Reviewed-on-release-3.8-fb: http://review.gluster.org/16150 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16178 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Return ret as soon as `this` is freedNigel Babu2017-03-011-0/+1
| | | | | | | | | | | | | | | Omitting this return causes a confusing error message in the next block, which is also a case of use after free. This bug was found by Coverity scan. BUG: 789278 Change-Id: Ifd4932de437e8ae875ff191033ea43cff81b701d Signed-off-by: Nigel Babu <nigelb@redhat.com> Reviewed-on: https://review.gluster.org/16790 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* libglusterfs+transport+io-threads: fix 256KB stack abuseJeff Darcy2017-02-011-6/+6
| | | | | | | | | | | | | | | | | Some functions were allocating 64K booleans, which are (crazily) mapped to 4-byte ints, for a total of 256KB per call. Changed to use bitfields instead, so usage is now only 8KB per call. This was the impediment to changing the io-threads stack size, so that has been adjusted too. Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653 BUG: 1418095 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/15745 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* socket: socket disconnect should wait for poller thread exitRajesh Joseph2016-12-211-9/+10
| | | | | | | | | | | | | | | | | | | | | When SSL is enabled or if "transport.socket.own-thread" option is set then socket_poller is run as different thread. Currently during disconnect or PARENT_DOWN scenario we don't wait for this thread to terminate. PARENT_DOWN will disconnect the socket layer and cleanup resources used by socket_poller. Therefore before disconnect we should wait for poller thread to exit. Change-Id: I71f984b47d260ffd979102f180a99a0bed29f0d6 BUG: 1404181 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/16141 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* build: out-of-tree builds generates files in the wrong directoryKaleb S KEITHLEY2016-09-181-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | And minor cleanup of a few of the Makefile.am files while we're at it. Rewrite the make rules to do what xdrgen does. Now we can get rid of xdrgen. Note 1. netbsd6's sed doesn't do -i. Why are we still running smoke tests on netbsd6 and not netbsd7? We barely support netbsd7 as it is. Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with libglusterfs? A cut-and-paste mistake? It has no references to symbols in libglusterfs. Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_ regex that matches the same lines as the _extended_ regex "/#(ifndef|define|endif)/". To match the extended regex sed needs to be run with -r on Linux; with -E on *BSD. However NetBSD's and FreeBSD's sed helpfully also provide -r for compatibility. Using a basic regex avoids having to use a kludge in order to run sed with the correct option on OS X. Note 4. Not copying the bit of xdrgen that inserts copyright/license boilerplate. AFAIK it's silly to pretend that machine generated files like these can be copyrighted or need license boilerplate. The XDR source files have their own copyright and license; and their copyrights are bound to be more up to date than old boilerplate inserted by a script. From what I've seen of other Open Source projects -- e.g. gcc and its C parser files generated by yacc and lex -- IIRC they don't bother to add copyright/license boilerplate to their generated files. It appears that it's a long-standing feature of make (SysV, BSD, gnu) for out-of-tree builds to helpfully pretend that the source files it can find in the VPATH "exist" as if they are in the $cwd. rpcgen doesn't work well in this situation and generates files with "bad" #include directives. E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`, you get an #include directive in the generated .c file like this: ... #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h" ... which (obviously) results in compile errors on out-of-tree build because the (generated) header file doesn't exist at that location. Compared to `rpcgen ./glusterfs3-xdr.x` where you get: ... #include "glusterfs3-xdr.h" ... Which is what we need. We have to resort to some Stupid Make Tricks like the addition of various .PHONY targets to work around the VPATH "help". Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/... looks exactly like -I$(top_srcdir)/rpc/xdr/... Don't be fooled though. And don't delete the -I$(top_builddir)/rpc/xdr/... bits Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e BUG: 1330604 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14085 Tested-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* libglusterfs (timer): race conditions, illegal mem access, mem leakKaleb S KEITHLEY2016-06-011-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | While investigating gfapi memory consumption with valgrind, valgrind reported several memory access issues. Also see the timer 'registry' being recreated (shortly) after being freed during teardown due to the way it's currently written. Passing ctx as data to gf_timer_proc() is prone to memory access issues if ctx is freed before gf_timer_proc() terminates. (And in fact this does happen, at least in valgrind.) gf_timer_proc() doesn't need ctx for anything, it only needs ctx->timer, so just pass that. Nothing ever calls gf_timer_registry_init(). Nothing outside of timer.c that is. Making it and gf_timer_proc() static. Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a BUG: 1333925 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14247 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* rpc: change client insecure port ceiling from 65535 to 49151Prasanna Kumar Kalever2016-05-181-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | current port allocation to various processes (clumsy): 1023 - 1 -> client ports range if bind secure is turned on 49151 - 1024 -> fall back to this, if in above case ports exhaust 65535 - 1024 -> client port range if bind insecure is on 49152 - 65535 -> brick port range now, we have segregated port ranges 0 - 65535 to below 3 ranges 1023 - 1 -> client ports range if bind secure is turned on 49151 - 1024 -> client port range if bind insecure is on (fall back to this, if in above case ports exhaust) 49152 - 65535 -> brick port range so now we have a clean segregation of port mapping Change-Id: Ie3b4e7703e0bbeabbe0adbdd6c60d9ef78ef7c65 BUG: 1335776 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/14326 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* rpc: define client port rangePrasanna Kumar Kalever2016-05-031-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: when bind-insecure is 'off', all the clients bind to secure ports, if incase all the secure ports exhaust the client will no more bind to secure ports and tries gets a random port which is obviously insecure. we have seen the client obtaining a port number in the range 49152-65535 which are actually reserved as part of glusterd's pmap_registry for bricks, hence this will lead to port clashes between client and brick processes. Solution: If we can define different port ranges for clients incase where secure ports exhaust, we can avoid the maximum port clashes with in gluster processes. Still we are prone to have clashes with other non-gluster processes, but the chances being very low in the rhgs Env, but that's a different story on its own, which will be handled in upcoming patches. Change-Id: Ib5ce05991aa1290ccb17f6f04ffd65caf411feaf BUG: 1322805 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13998 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* rpc: fix gf_process_reserved_portsPrasanna Kumar Kalever2016-04-271-7/+1
| | | | | | | | | | | | | | | this patch also does minor code cleanups. Change-Id: I0d005bd0f9baaaae498aa1df4faa6fcb65fa7a6e BUG: 1198849 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13997 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* rpc: assign port only if it is unreservedPrasanna Kumar Kalever2016-04-061-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current order: assign port; check for port; if reserved { port--; continue to i; } bind(); basically, we are assigning port first then checking if it is reserved Fix: get unreserved port; assign port; bind(); from now, we get unreserved port first and then assign it Change-Id: I004580c5215e7c9cae3594af6405b20fcd9fa4ad BUG: 1323659 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/13900 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* build: install and package header files more conventionallyKaleb S. KEITHLEY2015-11-161-2/+0
| | | | | | | | | | | | | | The current way we install and package header files for the -devel package is a hack. This patch uses more conventional autoconf, libtool, and rpmbuild idioms to package -devel headers and libraries. Change-Id: I63ffb3460f5c12b6b355493bd00824ac9e5354c5 BUG: 1271907 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12360 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* all: reduce "inline" usageJeff Darcy2015-09-011-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | There are three kinds of inline functions: plain inline, extern inline, and static inline. All three have been removed from .c files, except those in "contrib" which aren't our problem. Inlines in .h files, which are overwhelmingly "static inline" already, have generally been left alone. Over time we should be able to "lower" these into .c files, but that has to be done in a case-by-case fashion requiring more manual effort. This part was easy to do automatically without (as far as I can tell) any ill effect. In the process, several pieces of dead code were flagged by the compiler, and were removed. Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155 BUG: 1245331 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/11769 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* rdma:restore device linked list structure in case of failureMohammed Rafi KC2015-08-131-3/+5
| | | | | | | | | | | | | | | We maintain a linked list strcture to store device information, and maintain head in a ctx variable. A new device will be added at the beginning of list. But if the device creation failed, then we need to adjust the linked list to proper state. Change-Id: I07cefd3b808d8973a915728b3ba7f2955d29c92a BUG: 1250297 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/11829 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* rpc: fix binding brick issue while bind-insecure is enabledPrasanna Kumar Kalever2015-07-261-29/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | problem: When bind-insecure is turned on (which is the default now), it may happen that brick is not able to bind to port assigned by Glusterd for example 49192-49195... It seems to occur because the rpc_clnt connections are binding to ports in the same range. so brick fails to bind to a port which is already used by someone else solution: fix for now is to make rpc_clnt to get port numbers from 65535 in a descending order, as a result port clash is minimized other fixes: previously rdma binds to port >= 1024 if it cannot find a free port < 1024, even when bind insecure was turned off(ref to commit '0e3fd04e'), this patch add's a check for bind-insecure in gf_rdma_client_bind function This patch also re-enable bind-insecure and allow insecure by default which was reverted (ref: commit cef1720) previously Change-Id: Ia1cfa93c5454e2ae0ff57813689b75de282ebd07 BUG: 1238661 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/11512 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* rdma : porting missing gf_log to gf_msgManikandan Selvaganesh2015-07-122-11/+15
| | | | | | | | | Change-Id: I036b43007fbcd0e528faab8d44e1a7fc820eaf1f BUG: 1242333 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/11607 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* rdma : removing duplicate printing of error string in gf_msgManikandan Selvaganesh2015-06-272-35/+27
| | | | | | | | | | Change-Id: I206cf80392d430f35558cc11699402c4cef12527 BUG: 1194640 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/11426 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* build: do not #include "config.h" in each fileNiels de Vos2015-05-293-15/+0
| | | | | | | | | | | | | | | | | | Instead of including config.h in each file, and have the additional config.h included from the compiler commandline (-include option). When a .c file tests for a certain #define, and config.h was not included, incorrect assumtions were made. With this change, it can not happen again. BUG: 1222319 Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10808 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* rdma:properly handle iobuf_pool when rdma transport is unloadedMohammed Rafi KC2015-05-052-20/+62
| | | | | | | | | | | | | | | | | We are registering iobuf_pool with rdma. When rdma transport is unloaded, we need to deregister all the buffers registered with rdma. Otherwise iobuf_arena destroy will fail. Also if rdma.so is loaded again, then register iobuf_pool with rdma Change-Id: Ic197721a44ba11dce41e03058e0a73901248c541 BUG: 1200704 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9854 Tested-by: NetBSD Build System Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
* rdma: porting rdma to a new message id logging formatHumble Devassy Chirammal2015-04-294-371/+695
| | | | | | | | | | Change-Id: I71e940817ae0a9378e82332d5a8569114fc13482 BUG: 1194640 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/9868 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* rdma:changing list iteration to safe modeMohammed Rafi KC2015-03-151-5/+10
| | | | | | | | | | | Change-Id: I2299378f02a5577a8bf2874664ba79e92c3811b5 BUG: 1201621 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9872 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* rdma: Free resources related to iobuf in finiMohammed Rafi KC2015-03-102-0/+19
| | | | | | | | | | | | | | | | If rdma transport is destroyed because of any reason, then rdma.so will be unloaded. But we are not setting iobuf registeration function to null. After this, if an iobuf request is came, then we will try to call a function which is not loaded. Change-Id: I3293f9974e16d8e865131785ee697ea02be8cdfc BUG: 1187456 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9697 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* rdma:enhance logging when a connection error occurMohammed Rafi KC2015-03-101-1/+3
| | | | | | | | | | | Change-Id: I6146307949a3d852d3af5f8b273004ad6b27451b BUG: 1196584 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9756 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* rdma: return proper data type.Humble Devassy Chirammal2015-03-091-1/+1
| | | | | | | | | | Change-Id: I9bb0898af96cfcfaba0f0c976a7808bc6ea08e6a Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/9838 Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* rdma: 'list','wr' and 'new' memory has to be verified.Humble Devassy Chirammal2015-03-091-1/+22
| | | | | | | | | | | Change-Id: I29a8825107b8f4cefe4f4c59296e98fe675ee943 BUG: 1199053 Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com> Reviewed-on: http://review.gluster.org/9811 Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* rdma:setting wrong remote memory.Mohammed Rafi KC2015-03-041-2/+2
| | | | | | | | | | | | | when we send more than one work request in a single call, the remote addr is always setting as the first address of the vector. Change-Id: I55aea7bd6542abe22916719a139f7c8f73334d26 BUG: 1197548 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9794 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* rdma: segfault trying to call ibv_dealloc_pd on a null pointer Mark Lipscombe2015-03-031-1/+3
| | | | | | | | | | | | | | if ibv_alloc_pd failed If creating an ib protection domain fails, during the cleanup a segfault will occur because trav->pd is null. Bug: 1197260 Change-Id: I21b867c204c4049496b1bf11ec47e4139610266a Signed-off-by: Mark Lipscombe <mlipscombe@gmail.com> Reviewed-on: http://review.gluster.org/9774 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Vijay Bellur <vbellur@redhat.com>