summaryrefslogtreecommitdiffstats
path: root/xlators/features
Commit message (Collapse)AuthorAgeFilesLines
* features/shard: Ref shard inode while adding to fsync listKrutika Dhananjay2019-02-041-8/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROBLEM: Lot of the earlier changes in the management of shards in lru, fsync lists assumed that if a given shard exists in fsync list, it must be part of lru list as well. This was found to be not true. Consider this - a file is FALLOCATE'd to a size which would make the number of participant shards to be greater than the lru list size. In this case, some of the resolved shards that are to participate in this fop will be evicted from lru list to give way to the rest of the shards. And once FALLOCATE completes, these shards are added to fsync list but without a ref. After the fop completes, these shard inodes are unref'd and destroyed while their inode ctxs are still part of fsync list. Now when an FSYNC is called on the base file and the fsync-list traversed, the client crashes due to illegal memory access. FIX: Hold a ref on the shard inode when adding to fsync list as well. And unref under following conditions: 1. when the shard is evicted from lru list 2. when the base file is fsync'd 3. when the shards are deleted. Change-Id: Iab460667d091b8388322f59b6cb27ce69299b1b2 fixes: bz#1669382 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> (cherry picked from commit 72922c1fd69191b220f79905a23395c3a87f86ce)
* features/shard: Fix launch of multiple synctasks for background deletionKrutika Dhananjay2019-01-152-71/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROBLEM: When multiple sharded files are deleted in quick succession, multiple issues were observed: 1. misleading logs corresponding to a sharded file where while one log message said the shards corresponding to the file were deleted successfully, this was followed by multiple logs suggesting the very same operation failed. This was because of multiple synctasks attempting to clean up shards of the same file and only one of them succeeding (the one that gets ENTRYLK successfully), and the rest of them logging failure. 2. multiple synctasks to do background deletion would be launched, one for each deleted file but all of them could readdir entries from .remove_me at the same time could potentially contend for ENTRYLK on .shard for each of the entry names. This is undesirable and wasteful. FIX: Background deletion will now follow a state machine. In the event that there are multiple attempts to launch synctask for background deletion, one for each file deleted, only the first task is launched. And if while this task is doing the cleanup, more attempts are made to delete other files, the state of the synctask is adjusted so that it restarts the crawl even after reaching end-of-directory to pick up any files it may have missed in the previous iteration. This patch also fixes uninitialized lk-owner during syncop_entrylk() which was leading to multiple background deletion synctasks entering the critical section at the same time and leading to illegal memory access of base inode in the second syntcask after it was destroyed post shard deletion by the first synctask. Change-Id: Ib33773d27fb4be463c7a8a5a6a4b63689705324e updates: bz#1665803 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> (cherry picked from commit c0c2022e7d7097e96270a74f37813eda0c4e6339)
* features/shard: Assign fop id during background deletion to prevent ↵Krutika Dhananjay2019-01-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | excessive logging ... of the kind "[2018-12-26 05:22:44.195019] E [MSGID: 133010] [shard.c:2253:shard_common_lookup_shards_cbk] 0-volume1-shard: Lookup on shard 785 failed. Base file gfid = cd938e64-bf06-476f-a5d4-d580a0d37416 [No such file or directory]" shard_common_lookup_shards_cbk() has a specific check to ignore ENOENT error without logging them during specific fops. But because background deletion is done in a new frame (with local->fop being GF_FOP_NULL), the ENOENT check is skipped and the absence of shards gets logged everytime. To fix this, local->fop is initialized to GF_FOP_UNLINK during background deletion. Change-Id: I0ca8d3b3bfbcd354b4a555eee520eb0479bcda35 updates: bz#1665803 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> (cherry picked from commit aa28fe32364e39981981d18c784e7f396d56153f)
* leases: Reset lease_ctx->timer post deletionSoumya Koduri2019-01-091-0/+1
| | | | | | | | | | To avoid use_after_free, reset lease_ctx->timer back to NULL after the structure has been freed. Change-Id: Icd213ec809b8af934afdb519c335a4680a1d6cdc updates: bz#1651323 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit a9b0003c717087ff168bc143c70559162e53e0d5)
* shard: prevent segfault in shard_unlink_block_inode()Sunny Kumar2018-12-201-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | gluster-blockd sometimes segfaults with the following backtrace: Core was generated by `/usr/sbin/gluster-blockd --glfs-lru-count 5 --log-level INFO'. Program terminated with signal 11, Segmentation fault. #0 0x00007fbb9cd639b9 in shard_unlink_block_inode (local=local@entry=0x7fbb80000a78, shard_block_num=<optimized out>) at shard.c:2929 2929 base_ictx->fsync_count--; (gdb) bt #0 0x00007fbb9cd639b9 in shard_unlink_block_inode (local=local@entry=0x7fbb80000a78, shard_block_num=<optimized out>) at shard.c:2929 #1 0x00007fbb9cd64311 in shard_unlink_shards_do_cbk (frame=frame@entry=0x7fbb9010a768, cookie=<optimized out>, this=<optimized out>, op_ret=<optimized out>, op_errno=<optimized out>, preparent=preparent@entry=0x7fbb7470dcf8, postparent=postparent@entry=0x7fbb7470dd90, xdata=xdata@entry=0x0) at shard.c:2987 A fix for this has already been provided through a Converity report. Backport of: > Change-Id: Ic5d302a5e32d375acf8adc412763ab94e6dabc3d > Signed-off-by: Sunny Kumar <sunkumar@redhat.com> > (cherry picked from commit 145e180517054626d07892219fdee689b703c218) Change-Id: I699a039e9c5115eb3376190dd8014427d12a293b Updates: bz#1659563 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* features/bitrot: compare the signature with proper lengthRaghavendra Bhat2018-12-121-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | * The scrubber was comparing the checksum of the file that it calculated (by reading the file) with the on disk signature (stored via xattr) wrongly. It was using strlen to calculate the signature, while the actual length of the signature is given by the brick. Just use the actual length that the brick provides instead of trying to calculate the signature length via strlen API. * In posix, gfid2path was using the same string that contains the list of all the xattrs of file to save the value of the gfid2path xattr as well. This causes confusion when gfid2path xattr is queried by scrubber for getting the actual path of a corrupted file. Use separate string to fetch the value of the xattr instead of the string that contains the list of xattrs. Backport of: > Patch: https://review.gluster.org/21752/ > BUG: 1654805 > Change-Id: I2d664ab524d2b312233476cb35863dde3122e9a9 (cherry picked from commit f77fb6d568616592ab25501c402c140d15235ca9) Change-Id: I2d664ab524d2b312233476cb35863dde3122e9a9 fixes: bz#1654370 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* leases: Do not conflict with internal fopsSoumya Koduri2018-12-121-0/+11
| | | | | | | | | | | | | | | Internal fops (with frame->root->pid < 0) are used to heal or move data and maintains data integrity. That is they do not modify client data which holds the lease. Hence no need to recall Lease for such fops. Note: Like for locks, we would need rebalance and self-heal daemon process to heal lease state as well. Change-Id: I8988693fef8d00e17c19dcc842e2238f9eb5ab48 updates: bz#1651323 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit 080aa5b9e9d998552e23f7c33aed3afb0ca93c34)
* lease: Treat unlk request as noop if lease not foundSoumya Koduri2018-12-121-2/+14
| | | | | | | | | | | | | | | | | When the glusterfs server recalls the lease, it expects client to flush data and unlock the lease. If not it sets a timer (starting from the time it sends RECALL request) and post timeout, it revokes it. Here we could have a race where in client did send UNLK lease request but because of network delay it may have reached after server revokes it. To handle such situations, treat such requests as noop and return sucesss. Change-Id: I166402d10273f4f115ff04030ecbc14676a01663 updates: bz#1651323 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit c2e758b54d8a3f778e3e63db0000bb8b63de9b25)
* leases: Fix incorrect inode_ref/unrefsSoumya Koduri2018-11-292-3/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | From testing & code-reading, found couple of places where we incorrectly unref the inode resulting in use_after_free crash or ref leaks. This patch addresses couple of them. a) When we try to grant the very first lease for a inode, inode_ref is taken in __add_lease. This ref should be active till all the leases granted to that inode are released (i.e, till lease_cnt > 0). In addition even after lease_cnt becomes '0', the inode should be active till all the blocked fops are resumed. Hence release this ref, after resuming all those fops. To avoid granting new leases while resuming those fops, defined a new boolean (blocked_fops_resuming) to flag it in the lease_ctx. b) 'new_lease_inode' which creates new lease_inode_entry and takes ref on inode, is used while adding that entry to client_list and recall_list. Use its counter function '__destroy_lease_inode' which does unref while removing those entries from those lists. c) inode ref is also taken when added to timer->data. Unref the same after processing timer->data. Change-Id: Ie77c78ff4a971e0d9a66178597fb34faf39205fb updates: bz#1651323 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit b7aec05aa965202ab73120acf0da4c32fe0cf16c)
* all: fix the format string exceptionsAmar Tumballi2018-11-095-11/+11
| | | | | | | | | | | | | | | | Currently, there are possibilities in few places, where a user-controlled (like filename, program parameter etc) string can be passed as 'fmt' for printf(), which can lead to segfault, if the user's string contains '%s', '%d' in it. While fixing it, makes sense to make the explicit check for such issues across the codebase, by making the format call properly. Fixes: CVE-2018-14661 Fixes: bz#1647666 Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* features/locks:Use pthread_mutex_unlock() instead of pthread_mutex_lock()Vijay Bellur2018-11-091-1/+1
| | | | | | | | | Fixes CID 1396581 Change-Id: Ic04091b5783a75d8e1e605a9c1c28b77fea048d3 updates: bz#1647962 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Susant Palai <spalai@redhat.com>
* lock: Do not allow meta-lock count to be more than oneSusant Palai2018-11-091-1/+34
| | | | | | | | | | | | | | | | | | | | In the current scheme of glusterfs where lock migration is experimental, (ideally) the rebalance process which is migrating the file should request for a metalock. Hence, the metalock count should not be more than one for an inode. In future, if there is a need for meta-lock from other clients, this patch can be reverted. Since pl_metalk is called as part of setxattr operation, any client process(non-rebalance) residing outside trusted network can exhaust memory of the server node by issuing setxattr repetitively on the metalock key. The current patch makes sure that more than one metalock cannot be granted on an inode. Fixes CVE-2018-14660 updates: bz#1647962 Change-Id: Ie1e697766388718804a9551bc58351808fe71069 Signed-off-by: Susant Palai <spalai@redhat.com>
* index: prevent arbitrary file creation outside entry-changes folderRavishankar N2018-11-051-0/+17
| | | | | | | | | | | | | | | | | | | | | Patch in master: https://review.gluster.org/#/c/glusterfs/+/21534/ Problem: A compromised client can set arbitrary values for the GF_XATTROP_ENTRY_IN_KEY and GF_XATTROP_ENTRY_OUT_KEY during xattrop fop. These values are consumed by index as a filename to be created/deleted according to the key. Thus it is possible to create/delete random files even outside the gluster volume boundary. Fix: Index expects the filename to be a basename, i.e. it must not contain any pathname components like "/" or "../". Enforce this. Fixes: CVE-2018-14654 Fixes: bz#1646204 Change-Id: I35f2a39257b5917d17283d0a4f575b92f783f143 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* leases:Mark the fop conflicting if lease_id not setSoumya Koduri2018-10-251-5/+8
| | | | | | | | | | | | | | | | | Glusterfs leases expects lease_id to be set and sent for each fop to determine conflict resolution with the existing lease. Incase if not set (most likely if there is an older client in a mixed cluster), it makes sense to consider it as conflicitng fop and recall the lease. Also fixed the return status check for __remove_lease(), wherein non-negative value is considered as success case. Change-Id: I5bcfba4f7c71a5af7cdedeb03436d0b818e85783 updates: #350 Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit cf5b13896d65b6916634976a3a5f61ddeefbc19c)
* features/shard: Hold a ref on base inode when adding a shard to lru listKrutika Dhananjay2018-10-251-14/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of: > Change-Id: Ic15ca41444dd04684a9458bd4a526b1d3e160499 > Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> > (cherry picked from commit e627977) > BUG: 1605056 In __shard_update_shards_inode_list(), previously shard translator was not holding a ref on the base inode whenever a shard was added to the lru list. But if the base shard is forgotten and destroyed either by fuse due to memory pressure or due to the file being deleted at some point by a different client with this client still containing stale shards in its lru list, the client would crash at the time of locking lru_base_inode->lock owing to illegal memory access. So now the base shard is ref'd into the inode ctx of every shard that is added to lru list until it gets lru'd out. The patch also handles the case where none of the shards associated with a file that is about to be deleted are part of the LRU list and where an unlink at the beginning of the operation destroys the base inode (because there are no refkeepers) and hence all of the shards that are about to be deleted will be resolved without the existence of a base shard in-memory. This, if not handled properly, could lead to a crash. Change-Id: Ic15ca41444dd04684a9458bd4a526b1d3e160499 updates: bz#1641440 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* mgmt/glusterd: use proper path to the volfileRaghavendra Bhat2018-10-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Till now, glusterd was generating the volfile path for the snapshot volume's bricks like this. /snaps/<snap name>/<brick volfile> But in reality, the path to the brick volfile for a snapshot volume is /snaps/<snap name>/<snap volume name>/<brick volfile> The above workaround was used to distinguish between a mount command used to mount the snapshot volume, and a brick of the snapshot volume, so that based on what is actually happening, glusterd can return the proper volfile (client volfile for the former and the brick volfile for the latter). But, this was causing problems for snapshot restore when brick multiplexing is enabled. Because, with brick multiplexing, it tries to find the volfile and sends GETSPEC rpc call to glusterd using the 2nd style of path i.e. /snaps/<snap name>/<snap volume name>/<brick volfile> So, when the snapshot brick (which is multiplexed) sends a GETSPEC rpc request to glusterd for obtaining the brick volume file, glusterd was returning the client volume file of the snapshot volume instead of the brick volume file. Change-Id: I28b2dfa5d9b379fe943db92c2fdfea879a6a594e fixes: bz#1636162 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> (cherry picked from commit 83a89296a3d12a3fc2a643c0630be5ce659204ea)
* ctime: Provide noatime optionKotresh HR2018-10-028-9/+123
| | | | | | | | | | | | | | | | | | | | | | | | Most of the applications are {c|m}time dependant and very few are atime dependant. So provide noatime option to not update atime when ctime feature is enabled. Also this option has to be enabled with ctime feature to avoid unnecessary self heal. Since AFR/EC reads data from single subvolume, atime is only updated in one subvolume triggering self heal. Backport of: > Patch: https://review.gluster.org/21073 > BUG: 1593538 > Change-Id: I085fb33c882296545345f5df194cde7b6cbc337e > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 89636be4c73b12de2e11c75d8e59527bb243f147) updates: bz#1633015 Change-Id: I085fb33c882296545345f5df194cde7b6cbc337e Signed-off-by: Kotresh HR <khiremat@redhat.com>
* glusterd: Update op-version from 4.2 to 5.0ShyamsundarR2018-09-171-2/+2
| | | | | | | | | | | | Post changing the max op-version to 4.2, after release 4.1 branching, the decision was to go with increasing release numbers. Thus this needs to change to 5.0. This commit addresses the above change. Fixes: bz#1628668 Change-Id: Ifcc0c6da90fdd51e4eceea40749511110a432cce Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: revert several patchs that introduced pre/post attrsShyamsundarR2018-09-173-4/+15
| | | | | | | | | | | | | | Reverted the following: - 248152767b0599986bbb6bb35fc27197f6be6964 - 09943beb499617212f2985ca8ea9ecd1ed1b470e - d01f7244e9d9f7e3ef84e0ba7b48ef1b1b09d809 The reverts are redone by hand, due to clang format changes that made using git to revert the changes more tedious. Change-Id: I96489638a2b641fb2206a110298543225783f7be Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* build: cleanup xlator link, --no-undefined, libuuidv6devKaleb S. KEITHLEY2018-09-123-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While attempting to build a (pre-)5.0 of glusterfs on Ubuntu bionic and cosmic, it became apparent that there are some gremlins hiding in the combination of the xlator export-symbols, the newish addition of -Wl,--no-undefined, and the new switch to libuuid from the old contrib/uuid. Note: even though Fedora 28 (and later) and Ubuntu bionic (and later) have the same nominal version of libtool, the Fedora version appears to do a better job of recursing through dependencies to determine the libraries to link with. Examination of the build logs showed that despite appearing to work on Fedora, not all xlators and shared libs were linked with -Wl, --no-undefined, and -luuid. And in the case of the gnfs xlator, it was not only not linked with -Wl,--no-undefined but alsos not linked with -lgfxdr and -lgfrpc. Added GF_XLATOR_LDFLAGS, similar to GF_XLATOR_DEFAULT_LDFLAGS. GF_XLATOR_DEFAULT_LDFLAGS is for xlators that export/expose the default or common set of symbols. GF_XLATOR_LDFLAGS is for those remaining xlators that export/expose non-default symbols, e.g. dht and glupy. This removes the need in the future to add things like $(UUID_LIBS) to every xlator's Makefile.am. Just add it to GF_XLATOR_LDFLAGS and GF_XLATOR_DEFAULT_LDFLAGS in configure.ac and you're done. This patch was tested on Fedora 28 (build, rpmbuild), Fedora Rawhide/30 (rpmbuild), RHEL8 (rpmbuild), CentOS7 (rpmbuild), Fedora koji --scratch build for f30/rawhide, and a Launchpad build for Ubuntu cosmic/18.10. Change-Id: Ieca104fa5c5d3c094e701c8ca4a73754dd0292b0 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-1274-62912/+61142
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Land clang-format changesGluster Ant2018-09-1293-4564/+4296
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* misc: fix misc. shebangsKaleb S. KEITHLEY2018-09-114-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * One #!/usr/bin/env python and three #!/usr/bin/python were overlooked in all the other python fixups. Ugh. * Two new python files missed the memo about #!/usr/bin/python3. * One #!/usr/bin/env bash. Various distribution packaging policies have strong wording about the use of #!/usr/bin/env ... Note: this patch does not change the use of #!/usr/bin/env bash in the two files extras/{clang-checker.sh,check_goto.pl} as these are not included in any packages. (Although I'm not actually sure why anyone would ever use '/usr/bin/env {sh,bash}' as I'm not aware of any version-specific differences like there are with, e.g., python.) * One #!/usr/bin/bash. On Fedora and CentOS > 6, /bin is a symlink to /usr/bin, so it makes little difference. But Debian & Ubuntu still have separate /bin and /usr/bin; and sh and bash are in /bin, not /usr/bin. (Historically, in BSD and SYSV Unix it was /bin/sh.) Note: Fedora and CentOS package build runs a script that converts all /bin/sh and /bin/bash to /usr/bin/sh and /usr/bin/bash. Change-Id: I9171265829af78dd0cd7622c22b56d22179ff8a3 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* features/snapview-server: remove the ret variableRaghavendra Bhat2018-09-111-4/+2
| | | | | | | | Used only once to store the return value of a function. Change-Id: Ib2b9db847b5f652ce46f220297d9c7c3503eaea2 fixes: #230 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* Quota: Fix for clang issuessanoj-unnikrishnan2018-09-072-4/+5
| | | | | | Updates: bz#1193929 Change-Id: I549f741fbf97ee3fa2d63edacee5b705ef37c49a Signed-off-by: sanoj-unnikrishnan <sunnikri@redhat.com>
* features/uss: Use xxh64 to generate gfid instead of md5sumRaghavendra Manjunath2018-09-053-13/+44
| | | | | | | | | | | | | | | | * This is to ensure FIPS support * Also changed the signature of svs_uuid_generate to get xlator argument * Added xxh64 wrapper functions in common-utils to generate gfid using xxh64 - Those wrapper functions can be used by other xlators as well to generate gfids using xxh64. But as of now snapview-server is going to be the only consumer. Change-Id: Ide66573125dd74122430cccc4c4dc2a376d642a2 Updates: #230 Signed-off-by: Raghavendra Manjunath <raghavendra@redhat.com> Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* xlators: add classification flag to someAmar Tumballi2018-09-041-0/+1
| | | | | | | | | Add classification to those translators which has `xlator_api_t` already defined and used. Updates: #430 Change-Id: I9d2772cb2c4ed4ab06aaa546500cf3b7d00bddac Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Multiple files: calloc -> mallocYaniv Kaul2018-09-0410-22/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xlators/storage/posix/src/posix-inode-fd-ops.c: xlators/storage/posix/src/posix-helpers.c: xlators/storage/bd/src/bd.c: xlators/protocol/client/src/client-lk.c: xlators/performance/quick-read/src/quick-read.c: xlators/performance/io-cache/src/page.c xlators/nfs/server/src/nfs3-helpers.c xlators/nfs/server/src/nfs-fops.c xlators/nfs/server/src/mount3udp_svc.c xlators/nfs/server/src/mount3.c xlators/mount/fuse/src/fuse-helpers.c xlators/mount/fuse/src/fuse-bridge.c xlators/mgmt/glusterd/src/glusterd-utils.c xlators/mgmt/glusterd/src/glusterd-syncop.h xlators/mgmt/glusterd/src/glusterd-snapshot.c xlators/mgmt/glusterd/src/glusterd-rpc-ops.c xlators/mgmt/glusterd/src/glusterd-replace-brick.c xlators/mgmt/glusterd/src/glusterd-op-sm.c xlators/mgmt/glusterd/src/glusterd-mgmt.c xlators/meta/src/subvolumes-dir.c xlators/meta/src/graph-dir.c xlators/features/trash/src/trash.c xlators/features/shard/src/shard.h xlators/features/shard/src/shard.c xlators/features/marker/src/marker-quota.c xlators/features/locks/src/common.c xlators/features/leases/src/leases-internal.c xlators/features/gfid-access/src/gfid-access.c xlators/features/cloudsync/src/cloudsync-plugins/src/cloudsyncs3/src/libcloudsyncs3.c xlators/features/bit-rot/src/bitd/bit-rot.c xlators/features/bit-rot/src/bitd/bit-rot-scrub.c bxlators/encryption/crypt/src/metadata.c xlators/encryption/crypt/src/crypt.c xlators/performance/md-cache/src/md-cache.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible It doesn't make sense to calloc (allocate and clear) memory when the code right away fills that memory with data. It may be optimized by the compiler, or have a microscopic performance improvement. In some cases, also changed allocation size to be sizeof some struct or type instead of a pointer - easier to read. In some cases, removed redundant strlen() calls by saving the result into a variable. 1. Only done for the straightforward cases. There's room for improvement. 2. Please review carefully, especially for string allocation, with the terminating NULL string. Only compile-tested! .. and allocate memory as much as needed. xlators/nfs/server/src/mount3.c : Don't blindly allocate PATH_MAX, but strlen() the string and allocate appropriately. Also, align error messges. updates: bz#1193929 Original-Author: Yaniv Kaul <ykaul@redhat.com> Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ibda6f33dd180b7f7694f20a12af1e9576fe197f5
* core: python3Kaleb S. KEITHLEY2018-09-033-10/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* libgfchangelog: Fix changelog history APIKotresh HR2018-08-311-5/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: If requested start time and end time doesn't fall into first HTIME file, then history API fails even though continuous changelogs are avaiable for the requested range in other HTIME files. This is induced by changelog disable and enable which creates fresh HTIME index file. Cause and Analysis: Each HTIME index file represents the availability of continuous changelogs. If changelog is disabled and enabled, a new HTIME index file is created represents non availability of continuous changelogs. So as long as the requested start and end falls into single HTIME index file and not across, history API should succeed. But History API checks for the changelogs only in first HTIME index file and errors out if not available. Fix: Check in all HTIME index files for availability of continuous changelogs for requested change. fixes: bz#1622549 Change-Id: I80eeceb5afbd1b89f86a9dc4c320e161907d3559 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* Various files: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-313-4/+15
| | | | | | | | | | | | | | | | | | | | strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Check for truncated output where applicable. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN (sizeof() ) for const strings. Compile-tested only! Change-Id: I54e80d4f4a80e98d3775e376efe05c51af0b29eb updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* changetimerecoder xlator: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-312-12/+19
| | | | | | | | | | | | | | | | | | | | | | | xlators/features/changetimerecorder/src/changetimerecorder.c xlators/features/changetimerecorder/src/ctr-helper.h strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Check for output truncation where applicable. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN (sizeof() ) for const strings. Compile-tested only! Change-Id: I2dd5629183222da8c9251af43b8b29aacf12a20a updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* changelog xlator: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-314-37/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | xlators/features/changelog/lib/src/gf-changelog-journal-handler.c xlators/features/changelog/lib/src/gf-changelog.c xlators/features/changelog/src/changelog-helpers.c xlators/features/changelog/src/changelog-misc.h strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Where possible, ensure there's no truncation of the output. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN (sizeof() ) for const strings. - switch a strncpy to a memcpy. Compile-tested only! Change-Id: Ia7a52bce0b243613ad910192ec163c93d944e077 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* xlators: move from strlen() to sizeof()Yaniv Kaul2018-08-313-5/+5
| | | | | | | | | | | | | | | | | | xlators/features/index/src/index.c xlators/features/shard/src/shard.c xlators/features/upcall/src/upcall-internal.c xlators/mgmt/glusterd/src/glusterd-bitrot.c xlators/mgmt/glusterd/src/glusterd-locks.c xlators/mgmt/glusterd/src/glusterd-mountbroker.c xlators/mgmt/glusterd/src/glusterd-op-sm.c For const strings, just do compile time size calc instead of runtime. Compile-tested only! Change-Id: I995b2b89f14454b3855a4cd0ca90b3f01d5e080f updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* bit-rot xlator: strncpy()->sprintf(), reduce strlen()'sYaniv Kaul2018-08-314-19/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | xlators/features/bit-rot/src/bitd/bit-rot-scrub-status.c xlators/features/bit-rot/src/stub/bit-rot-stub-helpers.c xlators/features/bit-rot/src/stub/bit-rot-stub.c xlators/features/bit-rot/src/stub/bit-rot-stub.h strncpy may not be very efficient for short strings copied into a large buffer: If the length of src is less than n, strncpy() writes additional null bytes to dest to ensure that a total of n bytes are written. Instead, use snprintf(). Ensure sprintf() results do not truncate. Also: - save the result of strlen() and re-use it when possible. - move from strlen to SLEN or sizeof() for const strings. - move ret from int32 to int. Compile-tested only! Change-Id: Ib9b923c45d2d59ac15a105410e8160b252061018 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* build: add --enable-asan configure optionsNiels de Vos2018-08-302-2/+2
| | | | | | | | | | | | | | Introduce a `./configure --enable-asan` to build with `-fsanitize=address -fno-omit-frame-pointer` options. This uses the libasan.so shared library, so that needs to be available. While running builds with the ASAN options, several linker issues surfaced and these have been addressed with this change as well. Building with --enable-asan has been tested on Fedora 28. Change-Id: I428a9da70dd8f7d0056cfbe5c398619a571469b2 Updates: #492 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* features/namespace: Fix a probable null dereferenceVijay Bellur2018-08-271-1/+1
| | | | | | | | Addresses CID: 1388656 Change-Id: I614c29e993dd112ee02c5bffe8c737deb225c1f9 Updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* feature/shard: Fix Coverity issueAshish Pandey2018-08-271-7/+5
| | | | | | | | | | | | | | | | | | | | | Fix following coverity issues- CID: 1394660 1394668 1394667 1389008 1389434 https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=84880983&defectInstanceId=25821108&mergedDefectId=1389008 https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=84880983&defectInstanceId=25821101&mergedDefectId=1389434 https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=84880983&defectInstanceId=25821001&mergedDefectId=1394660 https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=84880983&defectInstanceId=25821010&mergedDefectId=1394667 https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=84880983&defectInstanceId=25821017&mergedDefectId=1394668 Change-Id: I08f09649dbe758ba0d367ae5330b48b18784dec3 updates: bz#789278 Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* ctr: skip ctr xlator init if ctr is not enabledMohit Agrawal2018-08-272-74/+120
| | | | | | | | | | | | Problem: If ctr xlator is not required it consumes resources unnecessarily Solution: Call ctr xlator init only while feature is enabled Fixes: bz#1524323 Change-Id: I378113a390a286be20c4ade1b1bac170a8ef1b14 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* features/cloudsync: Fix various resource leaksVijay Bellur2018-08-241-0/+6
| | | | | | | | Addresses CID: 1394648, 1394653 Change-Id: Ie75d4a268bba090faa5c3fe0e87f0e5cef3ff773 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* locks: coverity fixesBhumika Goyal2018-08-241-4/+21
| | | | | | | | | | Fixes CID: 1356583 1356582 1356581 1356580 Add logs and also replace gf_log with gf_msg_debug. Change-Id: I348dc94f7a52034542d29096c12a2cf60aabe97f updates: bz#789278 Signed-off-by: Bhumika Goyal <bgoyal@redhat.com>
* marker: fix for coverity issuesHari Gowtham2018-08-242-12/+15
| | | | | | | | Fixes CID: 1325591 1356503 1325540 1351704 Change-Id: I325b1184baa81402eaa009cb36ab95e5de812f6c updates: bz#789278 Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
* features/snapview-server: validate the fs instance before doing fop thereRaghavendra Bhat2018-08-243-30/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROBLEM: ======== USS design depends on snapview-server translator communicating with each individual snapshot via gfapi. So, the snapview-server xlator maintains the glfs instance (thus the snapshot) to which a inode belongs to by storing it inside the inode context. Suppose, a file from a snapshot is opened by a application, and the fd is still valid from application's point of view (i.e. application has not yet closed fd). Now, if the snapshot to which the opened file belongs to is deleted, then the glfs_t instance corresponding to the snapshot is destroyed by snapview-server as part of snap deletion. But now, if the application does IO on the fd it has kept open, then snapview server tries to send that request to the corresponding snap via glfs instance for that snapshot stored in the inode context for the file on which the application is sending the fop. And this results in freed up glfs_t pointer being accessed and causes a segfault. FIX: === For fd based operations, check whether the glfs instance that the inode contains in its context, is still valid or not. For non fd based operations, usually lookup should guarantee that. But if the file was already looked up, and the client accessing the snap data (either NFS, or native glusterfs fuse) does not bother to send a lookup and directly sends a path based fop, then that path based fop should ensure that the fs instance is valid. Change-Id: I881be15ec46ecb51aa844d7fd41d5630f0d644fb updates: bz#1602070 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* features/changelog: Fix a file handle leak in htime_create()Vijay Bellur2018-08-231-2/+7
| | | | | | | | | In a few error scenarios, ht_file_hd was not being cleaned up. Addresses CID: 1325549 Change-Id: If9b4388aa700303c1eebbf1410dc35d18c4637df updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* snapview/server: Set uid,gid,and groups for gfapi callMohammed Rafi KC2018-08-231-1/+131
| | | | | | | | | | Before calling gfapi from snapd, we need to set uid, gid and groups in the context. This is required to do the validation from posix acl xlator. Change-Id: I181bea2570a69554ff363bf5a52478ff0363ea47 fixes: bz#1614168 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
* features/leases: Initialize op_errno in leases_open()Vijay Bellur2018-08-231-2/+1
| | | | | | | | | Addresses CID: 1356483 by getting rid of an unessential assignment for op_errno Change-Id: I2327ac17b2875b4dd5f79b76be59e2c8e570b031 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* features/sdfs: Minor improvementsVijay Bellur2018-08-211-3/+3
| | | | | | | | | | | | | | | | Addresses: CID 1389688: Prevent op_errno from being -ve in sdfs_rename() CID 1389286: Check for retval from loc_copy() to avoid logically dead code and Initializes op_errno to ENOMEM in sdfs_link() Change-Id: I12e17a98faa5887da94a33ba9ca775e8e0fef359 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* marker: FORWARD_NULL coverity fixShwetha Acharya2018-08-201-1/+1
| | | | | | | | | | | | | | Problem: Coverity false positive. Solution: Added a comment to ignore the false positive. CID: 1325591 BUG: 789278 Change-Id: I95037efd12c059efcfc04f3c4c13f60c530150b4 Signed-off-by: Shwetha Acharya <shwetha174@gmail.com>
* quota: coverity fixesBhumika Goyal2018-08-201-6/+3
| | | | | | | | Fixes CID: 1325638 1288766 1124357 Change-Id: I425f857d0d549da2dfaa6a6e6b8440cb14c21121 updates: bz#789278 Signed-off-by: Bhumika Goyal <bgoyal@redhat.com>
* features/changelog: close htime_fd in fini()Vijay Bellur2018-08-171-0/+5
| | | | | | | | Addresses CID: 1325549 Change-Id: Ib041c7c288db6810b2e13a05a19ee894a47c9b05 updates: bz#789278 Signed-off-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-29 09:47:00 +0000