summaryrefslogtreecommitdiffstats
path: root/rpc
Commit message (Collapse)AuthorAgeFilesLines
* rpc: Cleanup SSL specific data at the time of freeing rpc objectl17zhou2020-07-131-2/+20
| | | | | | | | | | | | | | | | | Problem: At the time of cleanup rpc object ssl specific data is not freeing so it has become a leak. Solution: To avoid the leak cleanup ssl specific data at the time of cleanup rpc object > Credits: l17zhou <cynthia.zhou@nokia-sbell.com.cn> > Fixes: bz#1768407 > Change-Id: I37f598673ae2d7a33c75f39eb8843ccc6dffaaf0 > (cherry picked from commit 54ed71dba174385ab0d8fa415e09262f6250430c) Change-Id: I37f598673ae2d7a33c75f39eb8843ccc6dffaaf0 Fixes: #1016 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* socket/ssl: fix crl handlingMilind Changire2020-07-102-19/+94
| | | | | | | | | | | | | | | | | | | | | | Problem: Just setting the path to the CRL directory in socket_init() wasn't working. Solution: Need to use special API to retrieve and set X509_VERIFY_PARAM and set the CRL checking flags explicitly. Also, setting the CRL checking flags is a big pain, since the connection is declared as failed if any CRL isn't found in the designated file or directory. A comment has been added to the code appropriately. > Change-Id: I8a8ed2ddaf4b5eb974387d2f7b1a85c1ca39fe79 > fixes: bz#1687326 > Signed-off-by: Milind Changire <mchangir@redhat.com> > (Cherry pick from commit 06fa261207f0f0625c52fa977b96e5875e9a91e0) > (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22334) Change-Id: I8a8ed2ddaf4b5eb974387d2f7b1a85c1ca39fe79 Fixes: #1362 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* socket: fix error handlingXavi Hernandez2019-12-131-84/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When __socket_proto_state_machine() detected a problem in the size of the request or it couldn't allocate an iobuf of the requested size, it returned -ENOMEM (-12). However the caller was expecting only -1 in case of error. For this reason the error passes undetected initially, adding back the socket to the epoll object. On further processing, however, the error is finally detected and the connection terminated. Meanwhile, another thread could receive a poll_in event from the same connection, which could cause races with the connection destruction. When this happened, the process crashed. To fix this, all error detection conditions have been hardened to be more strict on what is valid and what not. Also, we don't return -ENOMEM anymore. We always return -1 in case of error. An additional change has been done to prevent destruction of the transport object while it may still be needed. Backport of: > Change-Id: I6e59cd81cbf670f7adfdde942625d4e6c3fbc82d > Fixes: bz#1782495 > Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> Change-Id: I6e59cd81cbf670f7adfdde942625d4e6c3fbc82d Fixes: bz#1749625 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* rpc: Update address family if it is not provide in cmd-line argumentsMohit Agrawal2019-09-091-1/+12
| | | | | | | | | | | | | | | | | | | | Problem: After enabling transport-type to inet6 and passed ipv6 transport.socket.bind-address in glusterd.vol clients are not started. Solution: Need to update address-family based on remote-address for all gluster client process > Change-Id: Iaa3588cd87cebc45231bfd675745c1a457dc9b31 > Fixes: bz#1747746 > Credits: Amgad Saleh <amgad.saleh@nokia.com> > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit 80b8cfe3f1386606bada97a76a0cad7acdf6b877) Change-Id: Iaa3588cd87cebc45231bfd675745c1a457dc9b31 Fixes: bz#1739320 Credits: Amgad Saleh <amgad.saleh@nokia.com> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* event: rename event_XXX with gf_ prefixedXiubo Li2019-08-282-26/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I hit one crash issue when using the libgfapi. In the libgfapi it will call glfs_poller() --> event_dispatch() in file api/src/glfs.c:721, and the event_dispatch() is defined by libgluster locally, the problem is the name of event_dispatch() is the extremly the same with the one from libevent package form the OS. For example, if a executable program Foo, which will also use and link the libevent and the libgfapi at the same time, I can hit the crash, like: kernel: glfs_glfspoll[68486]: segfault at 1c0 ip 00007fef006fd2b8 sp 00007feeeaffce30 error 4 in libevent-2.0.so.5.1.9[7fef006ed000+46000] The link for Foo is: lib_foo_LADD = -levent $(GFAPI_LIBS) It will crash. This is because the glfs_poller() is calling the event_dispatch() from the libevent, not the libglsuter. The gfapi link info : GFAPI_LIBS = -lacl -lgfapi -lglusterfs -lgfrpc -lgfxdr -luuid If I link Foo like: lib_foo_LADD = $(GFAPI_LIBS) -levent It will works well without any problem. And if Foo call one private lib, such as handler_glfs.so, and the handler_glfs.so will link the GFAPI_LIBS directly, while the Foo won't and it will dlopen(handler_glfs.so), then the crash will be hit everytime. The link info will be: foo_LADD = -levent libhandler_glfs_LIBADD = $(GFAPI_LIBS) I can avoid the crash temporarily by linking the GFAPI_LIBS in Foo too like: foo_LADD = $(GFAPI_LIBS) -levent libhandler_glfs_LIBADD = $(GFAPI_LIBS) But this is ugly since the Foo won't use any APIs from the GFAPI_LIBS. And in some cases when the --as-needed link option is added(on many dists it is added as default), then the crash is back again, the above workaround won't work. Backport of: > https://review.gluster.org/#/c/glusterfs/+/23110/ > Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa > Fixes: #699 > Signed-off-by: Xiubo Li <xiubli@redhat.com> Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa updates: bz#1740525 Signed-off-by: Xiubo Li <xiubli@redhat.com> (cherry picked from commit 799edc73c3d4f694c365c6a7c27c9ab8eed5f260)
* rpc: glusterd start is failed and throwing an error Address already in useMohit Agrawal2019-08-191-7/+37
| | | | | | | | | | | | | | | | | | | | Problem: Some of the .t are failed due to bind is throwing an error EADDRINUSE Solution: After killing all gluster processes .t is trying to start glusterd but somehow if kernel has not cleaned up resources(socket) then glusterd startup is failed due to bind system call failure.To avoid the issue retries to call bind 10 times to execute system call succesfully > Change-Id: Ia5fd6b788f7b211c1508c1b7304fc08a32266629 > Fixes: bz#1743020 > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit c370c70f77079339e2cfb7f284f3a2fb13fd2f97) Change-Id: Ia5fd6b788f7b211c1508c1b7304fc08a32266629 Fixes: bz#1743219 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* ctime: Set mdata xattr on legacy filesKotresh HR2019-08-064-1/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The files which were created before ctime enabled would not have "trusted.glusterfs.mdata"(stores time attributes) xattr. Upon fops which modifies either ctime or mtime, the xattr gets created with latest ctime, mtime and atime, which is incorrect. It should update only the corresponding time attribute and rest from backend Solution: Creating xattr with values from brick is not possible as each brick of replica set would have different times. So create the xattr upon successful lookup if the xattr is not created Note To Reviewers: The time attributes used to set xattr is got from successful lookup. Instead of sending the whole iatt over the wire via setxattr, a structure called mdata_iatt is sent. The mdata_iatt contains only time attributes. Backport of: > Patch: https://review.gluster.org/22936 > Change-Id: I5e535631ddef04195361ae0364336410a2895dd4 > BUG: 1593542 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I5e535631ddef04195361ae0364336410a2895dd4 updates: bz#1733885 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* If bind-address is IPv6 return it successfullyAmgad Saleh2019-05-291-6/+11
| | | | | | | | | | >Change-Id: Ibd37b6ea82b781a1a266b95f7596874134f30079 >fixes: bz#1713730 >Signed-off-by: Amgad Saleh <amgad.saleh@nokia.com> fixes: bz#1715012 Change-Id: I9a9877b6c32a1ccbd119301a4225808ddaa02de4 Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
* transport/socket: log shutdown msg occasionallyRaghavendra G2019-04-162-2/+3
| | | | | | | Change-Id: If3fc0884e7e2f45de2d278b98693b7a473220a5e Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Fixes: bz#1679904 (cherry picked from commit ec1b84300fe267dd12c1e42e7e91905db935f1e2)
* rpclib: slow floating point math and libmKaleb S. KEITHLEY2019-04-161-8/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In release-6 rpc/rpc-lib (libgfrpc) added the function get_rightmost_set_bit() which calls log2(3), a call that takes a floating point parameter and returns a floating point. It's used thusly: right_most_unset_bit = get_rightmost_set_bit(...); (So is it really the right-most unset bit, or the right-most set bit?) It's unclear to me whether this is in the data path or not. If it is, it's rather scary to think about integer-to-float and float-to-integer conversions and slow calls to libm functions in the data path. gcc and clang have __builtin_ctz() which returns the same result as get_rightmost_set_bit(), and does it substantially faster. Approx 20M iterations of get_rightmost_set_bit() took ~33sec of wall clock time on my devel machine, while 20M iterations of __builtin_ctz() took < 9sec; get_rightmost_set_bit() is 3x slower than __builtin_ctz(). And as a side benefit, we can again eliminate the need to link libgfrpc with libm. Change-Id: If9e7e80874577c52223f8125b385fc930de20699 fixes: bz#1692957 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* dict: handle STR_OLD data type in xdr conversionsAmar Tumballi2019-03-082-0/+3
| | | | | | | | | | | | | | | | | Currently a dict conversion on wire for 3.x protocol happens using `dict_unserialize()`, which sets the type of data as STR_OLD. But the new protocol doesn't send it over the wire as its not considered as a valid format in new processes. But considering we deal with old and new protocol when we do a rolling upgrade, it will allow us to get all the information properly with new protocol. Credits: Krutika Dhananjay Fixes: bz#1686364 Change-Id: I165c0021fb195b399790b9cf14a7416ae75ec84f Signed-off-by: Amar Tumballi <amarts@redhat.com>
* socket: socket event handlers now return voidMilind Changire2019-03-021-7/+7
| | | | | | | | | | | | | | | | | | | | | Problem: Returning any value from socket event handlers to the event sub-system doesn't make sense since event sub-system cannot handle socket sub-system errors. Solution: Change return type of all socket event handlers to 'void' mainline: > Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f > Fixes: bz#1651246 > Signed-off-by: Milind Changire <mchangir@redhat.com> > Reviewed-on: https://review.gluster.org/c/glusterfs/+/22221 Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f Fixes: bz#1683900 Signed-off-by: Milind Changire <mchangir@redhat.com> (cherry picked from commit 776ba851c6ee6c265253d44cf1d6e4e3d4a21772)
* Multiple files: reduce work while under lock.Yaniv Kaul2019-01-291-17/+13
| | | | | | | | | | | | | | | | | Mostly, unlock before logging. In some cases, moved different code that was not needed to be under lock (for example, taking time, or malloc'ing) to be executed before taking the lock. Note: logging might be slightly less accurate in order, since it may not be done now under the lock, so order of logs is racy. I think it's a reasonable compromise. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I2438710016afc9f4f62a176ef1a0d3ed793b4f89
* core: heketi-cli is throwing error "target is busy"Mohit Agrawal2019-01-242-0/+2
| | | | | | | | | | | | | | | Problem: At the time of deleting block hosting volume through heketi-cli , it is throwing an error "target is busy". cli is throwing an error because brick is not detached successfully and brick is not detached due to race condition to cleanp xprt associated with detached brick Solution: To avoid xprt specifc race condition introduce an atomic flag on rpc_transport Change-Id: Id4ff1fe8375a63be71fb3343f455190a1b8bb6d4 fixes: bz#1668190 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* core: move logs which are only developer relevant to DEBUG levelAmar Tumballi2019-01-231-1/+1
| | | | | | | | | | | We had only changed the log level to DEBUG in release branch earlier. But considering 90%+ of our deployments happen in same env, we can look at these specific logs on need basis. With this change, the master branch will be easier to debug with lesser logs. Change-Id: I4157a7ec7d5ec9c2948b2bbc1e4cb8317f28d6b8 Updates: bz#1666833 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* rpc: use address-family option from vol fileMilind Changire2019-01-222-3/+3
| | | | | | | | | | | | | | | | | This patch helps enable IPv6 connections in the cluster. The default address-family is IPv4 without using this option explicitly. When address-family is set to "inet6" in the /etc/glusterfs/glusterd.vol file, the mount command-line also needs to have -o xlator-option="transport.address-family=inet6" added to it. This option also gets added to the brick command-line. Snapshot and gfapi use-cases should also use this option to pass in the inet6 address-family. Change-Id: I97db91021af27bacb6d7578e33ea4817f66d7270 fixes: bz#1635863 Signed-off-by: Milind Changire <mchangir@redhat.com>
* socket: don't pass return value from protocol handler to event handlerZhang Huan2019-01-221-2/+2
| | | | | | | | | | | Event handler handles socket level error only, while protocol handler handles in protocol level error. If protocol handler decides to disconnect on error in any case, it should call disconnect instead of return an error back to event handler. Change-Id: I9375be98cc52cb969085333f3c7229a91207d1bd updates: bz#1666143 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* socket: fix issue when socket read return with EAGAINZhang Huan2019-01-221-7/+11
| | | | | | | | | | | | | | | | | | | | In the case socket read returns EAGAIN, positive value about remaining vector to send is returned. This return value will be passed all the way back to event handler, making it complains. [2018-12-29 08:02:25.603199] T [socket.c:1640:__socket_read_simple_payload] 0-test-client-0-extra.0: partial read on non-blocking socket. [2018-12-29 08:02:25.603201] T [rpc-clnt.c:654:rpc_clnt_reply_init] 0-test-client-2-extra.1: received rpc message (RPC XID: 0xfa6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 12) from rpc-transport (test-client-2-extra.1) [2018-12-29 08:02:25.603207] T [socket.c:3129:socket_event_handler] 0-test-client-0-extra.0: (sock:32) socket_event_poll_in returned 1 Formerly, in socket_proto_state_machine, return value of socket_readv is used to check if message is all read-in. In this commit, it is checked whether size of bytes indicated in header are all read in. In this way, only 0 and -1 will be returned from socket_proto_state_machine(), indicating whether there is error in the underlying socket. Change-Id: I8be0d178b049f0720d738a03aec41c4b375d2972 updates: bz#1666143 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* socket: fix issue when socket write return with EAGAINZhang Huan2019-01-171-0/+2
| | | | | | | | | | | | | In the case socket write return with EAGAIN, the remaining vector count is return all way back to event handler, making followup pollin event to skip handling and dispatch loop complains about failure. Even thought temporary write failure is not an error. [2018-12-29 07:31:41.772310] E [MSGID: 101191] [event-epoll.c:674:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler Change-Id: Idf03d120b5f7619eda19720a583cbcc3e7da2504 updates: bz#1666143 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* socket: fix counting of socket total_bytes_read and total_bytes_writeZhang Huan2019-01-171-4/+4
| | | | | | Change-Id: If35d0dbae963facf00ab6bcf07c6e4d1706ed982 updates: bz#1666143 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* Revert "iobuf: Get rid of pre allocated iobuf_pool and use per thread mem pool"Amar Tumballi2019-01-082-4/+248
| | | | | | | | | This reverts commit b87c397091bac6a4a6dec4e45a7671fad4a11770. There seems to be some performance regression with the patch and hence recommended to have it reverted. Updates: #325 Change-Id: Id85d6203173a44fad6cf51d39b3e96f37afcec09
* rpc-clnt: reduce transport connect log for EINPROGRESSKinglong Mee2019-01-071-1/+2
| | | | | | | | | | quotad and ganesha.nfsd prints many logs as, [rpc-clnt.c:1739:rpc_clnt_submit ] 0-<VOLUME_NAME>-quota: error returned while attempting to connect to host: (null), port 0 Change-Id: Ic0c815400619e4a87a772a51b19822920228c1ef Updates: bz#1596787 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
* rpcsvc: Don't expect dictionary values to be availablePranith Kumar K2019-01-071-2/+2
| | | | | | | | | | | When reconfigure happens, string values from one dictionary are directly set in another dictionary. This can lead to invalid memory when the first dictionary is freed up. So do dict_set_dynstr_with_alloc instead of dict_set_str updates bz#1650403 Change-Id: Id53236467521cfdeb07e7178d87ba6cf88d17003 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* rpc/rpc-lib: fix coverity issueSheetal Pamecha2018-12-281-4/+0
| | | | | | | | | | | Defect: Code can never be reached because of the condition queue_index > 1024 cannot be true. CID: 1398471 Logically dead code updates: bz#789278 Change-Id: I367cda7e734f6d774900a58d8664cffcab69126f Signed-off-by: Sheetal Pamecha <sheetal.pamecha08@gmail.com>
* rpc : fix coverity in rpc/rpc-lib/src/rpcsvc.cSunny Kumar2018-12-281-2/+2
| | | | | | | | | | This patch fixes newly introduced coverity. CID: 1398472: Dereference before null check. updates: bz#789278 Change-Id: Ie9b13084097de8f24b138acd7608c3e15b3bba9c Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* socket: Remove redundant in_lock in incoming message handlingKrutika Dhananjay2018-12-262-37/+4
| | | | | | | | | | | | | A given epoll thread can handle only one incoming (POLLIN) request. And until the socket is rearmed for listening, it is guaranteed that there won't be any new incoming requests. As a result, the priv->in_lock which guards the socket proto state machine seems redundant. This patch removes priv->in_lock. Change-Id: I26b6ddd852aba8c10385833b85ffd2e53e46cb8c updates: bz#1467614 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* rpc: Use adaptive mutex in rpcsvc_program_registerMohit Agrawal2018-12-201-2/+8
| | | | | | | | | | Adaptive mutexes are used to protect critical/shared data items that are held for short periods.It provides a balance between spin locks and traditional mutex.We have observed after use adaptive mutex in rpcsvc_program_register got some improvement. Change-Id: I7905744b32516ac4e4ca3c83c2e8e5e306093add fixes: bz#1660701
* rdma: fix possible buffer overflowRinku Kothiya2018-12-191-1/+6
| | | | | | | | | | | | | used snprintf instead of sprintf and if the source string is bigger than destination then logged a warning message. clang warning: ‘%s’ directive writing up to 1024 bytes into a region of size 108. updates: bz#1622665 Change-Id: Ia5e7c53d35d8178dd2c75708698599fe8bded5de Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
* iobuf: Get rid of pre allocated iobuf_pool and use per thread mem poolPoornima G2018-12-182-248/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current implementation of iobuf_pool has two problems: - prealloc of 12.5MB memory, this limits the scale factor of the gluster processes due to RAM requirements - lock contention, as the current implementation has one global iobuf_pool lock. Credits for debugging and addressing the same goes to Krutika Dhananjay <kdhananj@redhat.com>. Issue: #410 Hence changing the iobuf implementation to use per thread mem pool. This may theoritically appear to cause perf dip as there is no preallocation. But per thread mem pool will not have significant perf impact as the last allocated memory is kept alive for subsequent allocs, for some time. The worst case would be if iobufs requested are of random sizes each time. The best case is, if we get iobuf request of the same size. From the perf tests, this patch did not seem to cause any perf decrease. Note that, with this patch, the rdma performance is going to degrade drastically. In one of the previous patchsets we had fixes to not degrade rdma perf, but rdma is not supported and also not tested [1]. Hence the decision was to not have code in rdma that is not tested and not supported. [1] https://lists.gluster.org/pipermail/gluster-users.old/2018-July/034400.html Updates: #325 Change-Id: Ic2ef3bd498f9250dea25f25ba0c01fde19584b27 Signed-off-by: Poornima G <pgurusid@redhat.com>
* clang: Fix various missing checks for empty listShyamsundarR2018-12-141-3/+1
| | | | | | | | | | | | | | | | | | | | When using list_for_each_entry(_safe) functions, care needs to be taken that the list passed in are not empty, as these functions are not empty list safe. clag scan reported various points where this this pattern could be caught, and this patch fixes the same. Additionally the following changes are present in this patch, - Added an explicit op_ret setting in error case in the macro MAKE_INODE_HANDLE to address another clang issue reported - Minor refactoring of some functions in quota code, to address possible allocation failures in certain functions (which in turn cause possible empty lists to be passed around) Change-Id: I1e761a8d218708f714effb56fa643df2a3ea2cc7 Updates: bz#1622665 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* [geo-rep]: Worker still ACTIVE after killing bricksMohit Agrawal2018-12-131-0/+6
| | | | | | | | | | | | | | | | | | | | | | | Problem: In changelog xlator after destroying listener it call's unlink to delete changelog socket file but socket file reference is not cleaned up from process memory Solution: 1) To cleanup reference completely from process memory serialize transport cleanup for changelog and then unlink socket file 2) Brick xlator will notify GF_EVENT_PARENT_DOWN to next xlator only after cleanup all xprts Test: To test the same run below steps 1) Setup some volume and enable brick mux 2) kill anyone brick with gf_attach 3) check changelog socket for specific to killed brick in lsof, it should cleanup completely fixes: bz#1600145 Change-Id: Iba06cbf77d8a87b34a60fce50f6d8c0d427fa491 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* rpc: Resolve memory leak in mgmt_pmap_signout_cbkMohit Agrawal2018-12-121-0/+6
| | | | | | | | | | | | Problem: At the time of submit signout request to mgmt rpc_clnt_mgmt_pmap_signout create a frame but in cbk frame is not destroyed Solution: cleanup frame in mgmt_pmap_signout_cbk to avoid leak Change-Id: I9961cacb2e02c8023c4c99e22e299b8729c2b09f fixes: bz#1658045 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* copy_file_range support in GlusterFSRaghavendra Bhat2018-12-124-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * libglusterfs changes to add new fop * Fuse changes: - Changes in fuse bridge xlator to receive and send responses * posix changes to perform the op on the backend filesystem * protocol and rpc changes for sending and receiving the fop * gfapi changes for performing the fop * tools: glfs-copy-file-range tool for testing copy_file_range fop - Although, copy_file_range support has been added to the upstream fuse kernel module, no release has been made yet of a kernel which contains the support. It is expected to come in the upcoming release of linux-4.20 So, as of now, executing copy_file_range fop on a fused based filesystem results in fuse kernel module sending read on the source fd and write on the destination fd. Therefore a small gfapi based tool has been written to be able test the copy_file_range fop. This tool is similar (in functionality) to the example program given in copy_file_range man page. So, running regular copy_file_range on a fuse mount point and running gfapi based glfs-copy-file-range tool gives some idea about how fast, the copy_file_range (or reflink) can be. On the local machine this was the result obtained. mount -t glusterfs workstation:new /mnt/glusterfs [root@workstation ~]# cd /mnt/glusterfs/ [root@workstation glusterfs]# ls file [root@workstation glusterfs]# cd [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new real 0m6.495s user 0m0.000s sys 0m1.439s [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr OPEN_SRC: opening /file is success OPEN_DST: opening /rrr is success FSTAT_SRC: fstat on /rrr is success copy_file_range successful real 0m0.309s user 0m0.039s sys 0m0.017s This tool needs following arguments 1) hostname 2) volume name 3) log file path 4) source file path (relative to the gluster volume root) 5) destination file path (relative to the gluster volume root) "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>" - Added a testcase as well to run glfs-copy-file-range tool * io-stats changes to capture the fop for profiling * NOTE: - Added conditional check to see whether the copy_file_range syscall is available or not. If not, then return ENOSYS. - Added conditional check for kernel minor version in fuse_kernel.h and fuse-bridge while referring to copy_file_range. And the kernel minor version is kept as it is. i.e. 24. Increment it in future when there is a kernel release which contains the support for copy_file_range fop in fuse kernel module. * The document which contains a writeup on this enhancement can be found at https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367 updates: #536 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-0544-106/+106
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* protocol/server: support server.all-squashXie Changlong2018-12-053-0/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | We still use gnfs on our side, so do a little work to support server.all-squash. Just like server.root-squash, it's also a volume wide option. Also see bz#1285126 $ gluster volume set <VOLNAME> server.all-squash on Note: If you enable server.root-squash and server.all-squash at the same time, only server.all-squash works. Please refer to following table +---------------+-----------------+---------------------------+ | |all_squash | no_all_squash | +-------------------------------------------------------------+ | | |anonuid/anongid for root | |root_squash |anonuid/anongid |useruid/usergid for no-root| +-------------------------------------------------------------+ |no_root_squash |anonuid/anongid |useruid/usergid | +-------------------------------------------------------------+ Updates bz#1285126 Signed-off-by: Xie Changlong <xiechanglong@cmss.chinamobile.com> Signed-off-by: Xue Chuanyu <xuechuanyu@cmss.chinamobile.com> Change-Id: Iea043318fe6e9a75fa92b396737985062a26b47e
* rpc-transport/socket: NULL pointer dereferencing clang fixSheetal Pamecha2018-12-051-1/+3
| | | | | | | | | | Problem: res->ai_addr could be NULL Added a check to address this issue Change-Id: Iac88a8d6dc1f009836554448afbc228df93decd6 Updates: bz#1622665 Signed-off-by: Sheetal Pamecha <sheetal.pamecha08@gmail.com>
* rpc: check if fini is there before calling itRaghavendra Bhat2018-12-041-1/+3
| | | | | | | | | | | | | | | The rpc_transport_t structure is allocated and filled in the rpc_transport_load function. If filling the fileds of the rpc structure fails, then in the failure handling the structure is freed by rpc_transport_cleanup. There, it unconditionally calls fini. But, if the failure handling was invoked because of any failure in between the allocation of rpc_transport_t and filling the transport->fini (including the failure to fill fini ()), then rpc_transport_cleanup can lead to a segfault. Change-Id: I8be9b84cd6b19933c559c9736198a6e440373f68 fixes: bz#1654917 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* server: Resolve memory leak path in server_initMohit Agrawal2018-12-035-9/+66
| | | | | | | | | | | | | | Problem: 1) server_init does not cleanup allocate resources while it is failed before return error 2) dict leak at the time of graph destroying Solution: 1) free resources in case of server_init is failed 2) Take dict_ref of graph xlator before destroying the graph to avoid leak Change-Id: I9e31e156b9ed6bebe622745a8be0e470774e3d15 fixes: bz#1654917 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* rpc *.h fles: align structsYaniv Kaul2018-12-038-106/+100
| | | | | | | | Make an effort to slightly better align the structures. Change-Id: I6f80a451f2ffbf15adfb986cedc24c2799787b49 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* Multiple xlator .h files: remove unused private gf_* memory types.Yaniv Kaul2018-11-301-1/+0
| | | | | | | | | | | | | It seems there were quite a few unused enums (that in turn cause unndeeded memory allocation) in some xlators. I've removed them, hopefully not causing any damage. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I8252bd763dc1506e2d922496d896cd2fc0886ea7
* rpcsvc: provide each request handler thread its own queueRaghavendra Gowdappa2018-11-298-141/+348
| | | | | | | | | | | | | | | | | | | | | | | | | | | | A single global per program queue is contended by all request handler threads and event threads. This can lead to high contention. So, reduce the contention by providing each request handler thread its own private queue. Thanks to "Manoj Pillai"<mpillai@redhat.com> for the idea of pairing a single queue with a fixed request-handler-thread and event-thread, which brought down the performance regression due to overhead of queuing significantly. Thanks to "Xavi Hernandez"<xhernandez@redhat.com> for discussion on how to communicate the event-thread death to request-handler-thread. Thanks to "Karan Sandha"<ksandha@redhat.com> for voluntarily running the perf benchmarks to qualify that performance regression introduced by ping-timer-fixes is fixed with this patch and patiently running many iterations of regression tests while RCAing the issue. Thanks to "Milind Changire"<mchangir@redhat.com> for patiently running the many iterations of perf benchmarking tests while RCAing the regression caused by ping-timer-expiry fixes. Change-Id: I578c3fc67713f4234bd3abbec5d3fbba19059ea5 Fixes: bz#1644629 Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
* libglusterfs: rename macros roof and floor to not conflict with math.hRaghavendra Gowdappa2018-11-281-4/+4
| | | | | | Change-Id: I666eeb63ebd000711b3f793b948d4e0c04b1a242 Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com> Updates: bz#1644629
* core: create a constant for default network timeoutXavi Hernandez2018-11-231-3/+3
| | | | | | | | | | A new constant named GF_NETWORK_TIMEOUT has been defined and all references to the hard-coded timeout of 42 seconds have been replaced with this constant. Change-Id: Id30f5ce4f1230f9288d9e300538624bcf1a6da27 fixes: bz#1652852 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* rpc: stop log flooding about ENODATAMilind Changire2018-11-231-3/+3
| | | | | | | | | | | | | | Problem: Logs are being flooded with ENODATA errors. This log was introduced via https://review.gluster.org/c/glusterfs/+/21481 Solution: Add a flag to remember that ENODATA error was logged for a socket/transport Change-Id: I54c10b87e46c2592339cc8b966333b8d08331750 fixes: bz#1650389 Signed-off-by: Milind Changire <mchangir@redhat.com>
* core: fix strncpy warningsKaleb S. KEITHLE2018-11-151-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since gcc-8.2.x (fedora-28 or so) gcc has been emitting warnings about buggy use of strncpy. Most uses that gcc warns about in our sources are exactly backwards; the 'limit' or len is the strlen/size of the _source param_, giving exactly zero protection against overruns. (Which was, after all, one of the points of using strncpy in the first place.) IOW, many warnings are about uses that look approximately like this: ... char dest[8]; char src[] = "this is a string longer than eight chars"; ... strncpy (dest, src, sizeof(src)); /* boom */ ... The len/limit should be sizeof(dest). Note: the above example has a definite over-run. In our source the overrun is typically only theoretical (but possibly exploitable.) Also strncpy doesn't null-terminate on truncation; snprintf does; prefer snprintf over strncpy. Mildly surprising that coverity doesn't warn/isn't warning about this. Change-Id: I022d5c6346a751e181ad44d9a099531c1172626e updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLE <kkeithle@redhat.com>
* rpc/rpc-lib/src/rpc-clnt.c: unlock sooner, if we fail to connect.Yaniv Kaul2018-11-151-15/+11
| | | | | | | | | | | | Previously, we did not go to unlock the mutex if we failed to connect. This patch fixes it. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I0fcca066a2601dba6bc3e9eb8b3c9fc757ffe4db
* glusterfsd: Make each multiplexed brick sign inPrashanth Pai2018-11-121-0/+1
| | | | | | | | | | | | | | | | | | | | | | | NOTE: This change will be consumed by brick mux implementation of glusterd2 only. No corresponsing change in glusterd1 has been made. When a multiplexed brick process is shutting down, it sends sign out requests to glusterd for all bricks that it contains. However, sign in request is only sent for a single brick. Consequently, glusterd has to use some tricky means to repopulate pmap registry with information of multiplexed bricks during glusterd restart. This change makes each multiplexed brick send a sign in request to glusterd2 which ensures that glusterd2 can easily repopulate pmap registry with port information. As a bonus, sign in request will now also contain PID of the brick sending the request so that glusterd2 can rely on this instead of having to read/manage brick pidfiles. Change-Id: I409501515bd9a28ee7a960faca080e97cabe5858 updates: bz#1193929 Signed-off-by: Prashanth Pai <ppai@redhat.com>
* rpc-clnt*: several code changes to reduce conn lock timesYaniv Kaul2018-11-123-55/+25
| | | | | | | | | | | | | | Assorted code refactoring to reduce lock contention. Also, took the opportunity to reorder structs more properly. Removed dead code. Hopefully, no functional changes. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I5de6124ad071fd5e2c31832364d602b5f6d6fe28
* libglusterfs multiple files: remove dead initilizationYaniv Kaul2018-11-111-8/+2
| | | | | | | | | | | | | Per newer GCC releases and clang-scan, some trivial dead initialization (values that were set but were never read) were removed. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ia9959b2ff87d2e9cb46864e68ffe7dccb984db34
* all: fix the format string exceptionsAmar Tumballi2018-11-051-2/+2
| | | | | | | | | | | | | | | | Currently, there are possibilities in few places, where a user-controlled (like filename, program parameter etc) string can be passed as 'fmt' for printf(), which can lead to segfault, if the user's string contains '%s', '%d' in it. While fixing it, makes sense to make the explicit check for such issues across the codebase, by making the format call properly. Fixes: CVE-2018-14661 Fixes: bz#1644763 Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4 Signed-off-by: Amar Tumballi <amarts@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-25 07:05:42 +0000