summaryrefslogtreecommitdiffstats
path: root/xlators
Commit message (Collapse)AuthorAgeFilesLines
* glusterd: fix tier-enabled flag op-version checkAtin Mukherjee2018-02-131-2/+2
| | | | | | | | | | | | | | tier-enabled flag in volinfo structure was introduced in 3.10, however while writing this value to the glusterd store was done with a wrong op-version check which results into volume checksum failure during upgrades. >Change-Id: I4330d0c4594eee19cba42e2cdf49a63f106627d4 >BUG: 1544600 >Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Change-Id: I4330d0c4594eee19cba42e2cdf49a63f106627d4 BUG: 1544637 Signed-off-by: hari gowtham <hgowtham@redhat.com>
* cluster/dht: Skip '..' for the volume root dirN Balachandran2018-02-121-0/+5
| | | | | | | | | | | | | | | | dht_populate_inode_for_dentry tries to update the layout for the '..' entry when listing the root of the volume. This entry does not correspond to an entry in the volume and therefore does not have a gfid or a layout on disk, causing layout processing to fail. > Change-Id: I2b7470e1c5e20d87b5545160697f24d041045140 > BUG: 1537457 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I2b7470e1c5e20d87b5545160697f24d041045140 BUG: 1539516 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Cleanup on fallocate failureN Balachandran2018-02-091-1/+17
| | | | | | | | | | | | | | | It looks like fallocate leaves a non-empty file behind in case of some failures. We now truncate the file to 0 bytes on failure in __dht_rebalance_create_dst_file. > Change-Id: Ia4ad7b94bb3624a301fcc87d9e36c4dc751edb59 > BUG: 1541916 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: Ia4ad7b94bb3624a301fcc87d9e36c4dc751edb59 BUG: 1542601 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Unlink linkto files as rootN Balachandran2018-02-081-3/+7
| | | | | | | | | | | | | | | Non-privileged users cannot delete linkto files. However the failure to unlink a stale linkto causes DHT to fail the lookup with EIO and hence prevent access to the file. > Change-Id: Id295362d41e52263790694602f36f1219f0646a2 > BUG: 1542318 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: Id295362d41e52263790694602f36f1219f0646a2 BUG: 1543016 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Fixed leak in dht_populate_inode_for_dentryN Balachandran2018-02-072-6/+10
| | | | | | | | | | | | | | | | Fixed an issue in dht_populate_inode_for_dentry where a layout is set in the inode without checking if it is already set. This overwrites the value each time without freeing the already existing layout. Also includes the changes in https://review.gluster.org/19471 > Change-Id: I651bf539a0b82b4ddc4c355890c16a8e91f5f1fd > BUG: 1541264 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I651bf539a0b82b4ddc4c355890c16a8e91f5f1fd BUG: 1541267 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/afr: remove unnecessary child_up initializationXavier Hernandez2018-02-061-7/+0
| | | | | | | | | | | | | | | The child_up array was initialized with all elements being -1 to allow afr_notify() to differentiate down bricks from bricks that haven't reported yet. With current implementation this is not needed anymore and it was causing unexpected results when other parts of the code considered that if child_up[i] != 0, it meant that it was up. Backport of: > BUG: 1541038 Change-Id: I2a9d712ee64c512f24bd5cd3a48dcb37e3139472 BUG: 1541930 Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* cluster/dht: Add migration checks to dht_(f)xattropN Balachandran2018-02-068-45/+362
| | | | | | | | | | | | | | | | The dht_(f)xattrop implementation did not implement migration phase1/phase2 checks which could cause issues with rebalance on sharded volumes. This does not solve the issue where fops may reach the target out of order. > Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c > BUG: 1471031 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c BUG: 1540224 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/ec: OpenFD heal implementation for ECSunil Kumar Acharya2018-02-027-32/+184
| | | | | | | | | | | | | | | | Existing EC code doesn't try to heal the OpenFD to avoid unnecessary healing of the data later. Fix implements the healing of open FDs before carrying out file operations on them by making an attempt to open the FDs on required up nodes. Backport of: >BUG: 1431955 BUG: 1536334 Change-Id: Ib696f59c41ffd8d5678a484b23a00bb02764ed15 Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
* glusterd: process pmap sign in only when port is marked as freeAtin Mukherjee2018-02-021-0/+15
| | | | | | | | | | | | | | | | | | | Because of some crazy race in volume start code path because of friend handshaking with volumes with quorum enabled we might end up into a situation where glusterd would start a brick and get a disconnect and then immediately try to start the same brick instance based on another friend update request. And then if for the very first brick even if the process doesn't come up at the end sign in event gets sent and we end up having two duplicate portmap entries for the same brick. Since in brick start we mark the previous port as free, its better to consider a sign in request as no op if the corresponding port type is marked as free. >mainline patch : https://review.gluster.org/#/c/19263/ Change-Id: I995c348c7b6988956d24b06bf3f09ab64280fc32 BUG: 1537346 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 9d708a3739c8201d23f996c413d6b08f8b13dd90)
* selinux-xlator : validate dict before calling dict_rename_key()Jiffin Tony Thottan2018-02-021-4/+4
| | | | | | | | | | | Upstream reference >Change-Id: I71da3b64e5e8c82e8842e119b2b05da3e2ace550 >BUG: 1535772 >Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> >(cherry picked from commit bee06ccd7b80e3f5804f0c7c7c56936fed6d2b4e) Change-Id: I71da3b64e5e8c82e8842e119b2b05da3e2ace550 BUG: 1533269
* posix: delete stale gfid handles in nameless lookupRavishankar N2018-01-161-1/+16
| | | | | | | | | | | ..in order for self-heal of symlinks to work properly (see BZ for details). Backport of https://review.gluster.org/#/c/19070/ Signed-off-by: Ravishankar N <ravishankar@redhat.com> Change-Id: I9a011d00b07a690446f7fd3589e96f840e8b7501 BUG: 1534847
* glusterd: connect to an existing brick process when qourum status is ↵Atin Mukherjee2018-01-129-15/+41
| | | | | | | | | | | | | | | | NOT_APPLICABLE_QUORUM First of all, this patch reverts commit 635c1c3 as the same is causing a regression with bricks not coming up on time when a node is rebooted. This patch tries to fix the problem in a different way by just trying to connect to an existing running brick when quorum status is not applicable. >mainline patch : https://review.gluster.org/#/c/19134/ Change-Id: I0efb5901832824b1c15dcac529bffac85173e097 BUG: 1511301 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* cluster/dht: Use percentages for space checkN Balachandran2018-01-121-5/+20
| | | | | | | | | | | | | | | | | | | | | With heterogenous bricks now being supported in DHT we could run into issues where files are not migrated even though there is sufficient space in newly added bricks which just happen to be considerably smaller than older bricks. Using percentages instead of absolute available space for space checks can mitigate that to some extent. Marking bug-1247563.t as that used to depend on the easier code to prevent a file from migrating. This will be removed once we find a way to force a file migration failure. > Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647 > BUG: 1529440 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647 BUG: 1530455 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* quota: fixes issue in quota.conf when setting large number of limitsSanoj Unnikrishnan2018-01-101-12/+33
| | | | | | | | | | | | | | | | Problem: It was not possible to configure more than 7712 quota limits. This was because a stack buffer of size 131072 was used to read from quota.conf file. In the new format of quota.conf file each gfid entry takes 17bytes (16byte gfid + 1 byte type). So, the buf_size was not a multiple of gfid entry size and as per code this was considered as corruption. Solution: make buf size multiple of gfid entry size Change-Id: Id036225505a47a4f6fa515a572ee7b0c958f30ed BUG: 1489043 Signed-off-by: Sanoj Unnikrishnan <sunnikri@redhat.com> (cherry picked from commit 2899a4f125735636fe7cd8db73c0b8a13289df9b)
* Revert "mount/fuse: report ESTALE as ENOENT"Raghavendra G2018-01-101-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 26d16b90ec7f8acbe07e56e8fe1baf9c9fa1519e. Consider rename (index.new, store.idx) and open (store.idx) being executed in parallel. When we break down operations following sequence is possible. * lookup (store.idx) - as part of open(store.idx) returns gfid1 as the result. * rename (index.new, store.idx) changes gfid of store.idx to gfid2. Note that gfid2 was the nodeid of index.new. Since rename is successful, gfid2 is associated with store.idx. * open (store.idx) resumes and issues open fop to glusterfs with gfid1. open in glusterfs fails as gfid1 doesn't exist and the error returned by glusterfs to kernel-fuse is ENOENT. * kernel passes back the same error to application as a result to open. This error could've been prevented if kernel retries open with gfid2. Interestingly kernel do retry open when it receives ESTALE error. Even though failure to find gfid resulted in ESTALE error, commit 26d16b90ec7f8acb converted that error to ENOENT while sending an error reply to kernel. This prevented kernel from retrying open resulting in error. >Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4 >BUG: 1500269 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 019a55e708375d2b1e576fcc948a691bcdc5c749) Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4 BUG: 1529088 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: Nullify pmap entry for bricks belonging to same portAtin Mukherjee2018-01-101-1/+1
| | | | | | | | | | | | | Commit 30e0b86 tried to address all the stale port issues glusterd had in case of a brick is abruptly killed. For brick multiplexing case because of a bug the portmap entry was not getting removed. This patch addresses the same. >mainline patch : https://review.gluster.org/#/c/19119/ Change-Id: Ib020b967a9b92f1abae9cab9492f0cacec59aaa1 BUG: 1530448 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* mount/fuse: never fail open(dir) with ENOENTRaghavendra G2018-01-031-0/+7
| | | | | | | | | | | | | | | open(dir) being an operation on inode should never fail with ENOENT. If gfid is not present, the appropriate error is ESTALE. This will enable kernel to retry open after a revalidate lookup. >Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b >BUG: 1500269 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit fb4b914ce84bc83a5f418719c5ba7c25689a9251) Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b BUG: 1529088 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* performance/write-behind: fix bug while handling short writesRaghavendra G2018-01-021-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The variabled "fulfilled" in wb_fulfill_short_write is not reset to 0 while handling every member of the list. This has some interesting consequences: * If we break from the loop while processing last member of the list head->winds, req is reset to head as the list is a circular one. However, head is already fulfilled and can potentially be freed. So, we end up adding a freed request to wb_inode->todo list. This is the RCA for the crash tracked by the bug associated with this patch (Note that we saw "holder" which is freed in todo list). * If we break from the loop while processing any of the last but one member of the list head->winds, req is set to next member in the list, skipping the current request, even though it is not entirely synced. This can lead to data corruption. The fix is very simple and we've to change the code to make sure "fulfilled" reflects whether the current request is fulfilled or not and it doesn't carry history of previous requests in the list. >Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8 >BUG: 1528558 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 0bc22bef7f3c24663aadfb3548b348aa121e3047) Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8 BUG: 1529095 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* mount/fuse: use fstat in getattr implementation if any opened fd is availableRaghavendra G2018-01-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The restriction of using fds opened by the same Pid means fds cannot be shared across threads of multithreaded application. Note that fops from kernel have different Pid for different threads. Imagine following sequence of operations: * Turn off performance.open-behind * Thread t1 opens an fd - fd1 - on file "file". Let's assume nodeid of "file" is "nodeid-file". * Thread t2 does RENAME ("newfile", "file"). Let's assume nodeid of "newfile" as "nodeid-newfile". * t2 proceeds to do fstat (fd1) The above set of operations can sometimes result in ESTALE/ENOENT errors. RENAME overwrites "file" with "newfile" changing its nodeid from "nodeid-file" to "nodeid-newfile" and post RENAME, "nodeid-file" is removed from the backend. If fstat carries nodeid-file as argument, which can happen if lookup has not refreshed the nodeid of "file" and since t2 doesn't have an fd opened, fuse_getattr_resume uses STAT which will fail as "nodeid-file" no longer exists. Since the above set of operations and sharing of fds across multiple threads are valid, this is a bug. The fix is to use any fd opened on the inode. In this specific example fuse_getattr_resume will find fd1 and winds down the call as fstat (fd1) which won't fail. Cross-checked with "Miklos Szeredi" <mszeredi.at.redhat.dot.com> for any security issues with this solution and he approves the solution. Thanks to "Miklos Szeredi" <mszeredi.at.redhat.dot.com> for all the pointers and discussions. >Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c >BUG: 1510401 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 8b57378e5596f287a7b9d106dd6fb56a624b42ee) Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c BUG: 1529085 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* feature/bitrot: remove internal xattrs from lookup cbkRavishankar N2017-12-192-7/+21
| | | | | | | | | | | | | | | | | | | | | | Problem: afr requests all xattrs in lookup via the list-xattr key. If bitrot is enabled and later disabled, or if the bitrot xattrs were present due to an older version of bitrot which used to create the xattrs without enabling the feature, the xattrs (trusted.bit-rot.version in particular) was not getting filtered and ended up reaching the client stack. AFR, on noticing different values of the xattr across bricks of the replica, started triggering spurious metadata heals. Fix: Filter all internal xattrs in bitrot xlator before unwinding lookup, (f)getxattr. Thanks to Kotresh for the help in RCA'ing. Change-Id: I5bc70e4b901359c3daefc67b8e4fa6ddb47f046c BUG: 1527276 Signed-off-by: Ravishankar N <ravishankar@redhat.com> (cherry picked from commit d341f20230b9921391aff22337eaf9be82f44d88)
* glusterd: Free up svc->conn on volume deleteAtin Mukherjee2017-12-121-0/+4
| | | | | | | | | | | | | Daemons like snapd, tierd and gfproxyd are maintained on per volume basis and on a volume delete we should destroy the rpc connection established for them. >mainline patch : https://review.gluster.org/#/c/18957/ Change-Id: Id1440e39da07b990fdb9b207df18da04b1ca8014 BUG: 1523048 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 36ce4c614a3391043a3417aa061d0aa16e60b2d3)
* Disable gfid2path by default on NetBSDEmmanuel Dreyfus2017-12-081-0/+11
| | | | | | | | | | | | | | | NetBSD storage of extended attributes for UFS1 badly scales when the list of extended attributes names rises. gfid2path can add as many extended attributes names as we have files, hence we keep it disabled for performance sake. > Change-Id: Id77b5f5ceb4d5eba1b3362b4b9fc693450ffbc2b > Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> > BUG: 1129939 Change-Id: Id77b5f5ceb4d5eba1b3362b4b9fc693450ffbc2b Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> BUG: 1513258
* cluster/dht: don't overfill the buffer in readdir(p)Raghavendra G2017-12-081-3/+18
| | | | | | | | | | | | | | | | | | | | | | | | Superflous dentries that cannot be fit in the buffer size provided by kernel are thrown away by fuse-bridge. This means, * the next readdir(p) seen by readdir-ahead would have an offset of a dentry returned in a previous readdir(p) response. When readdir-ahead detects non-monotonic offset it turns itself off which can result in poor readdir performance. * readdirp can be cpu-intensive on brick and there is no point to read all those dentries just to be thrown away by fuse-bridge. So, the best strategy would be to fill the buffer optimally - neither overfill nor underfill. > Change-Id: Idb3d85dd4c08fdc4526b2df801d49e69e439ba84 > BUG: 1492625 > Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit e785faead91f74dce7c832848f2e8f3f43bd0be5) Change-Id: Idb3d85dd4c08fdc4526b2df801d49e69e439ba84 BUG: 1478411 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: populate inode in dentry for single subvolume dhtRaghavendra G2017-12-062-1/+69
| | | | | | | | | | | | | | | ... in readdirp response if dentry points to a directory inode. This is a special case where the entire layout is stored in one single subvolume and hence no need for lookup to construct the layout >Change-Id: I44fd951e2393ec9dac2af120469be47081a32185 >BUG: 1492625 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 59d1cc720f52357f7a6f20bb630febc6a622c99c) Change-Id: I44fd951e2393ec9dac2af120469be47081a32185 BUG: 1478411 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: display gluster volume status, when quorum type is serverSanju Rakonde2017-11-301-0/+6
| | | | | | | | | | | | | | Problem: when server-quorum-type is server, after restarting glusterd in the node which is up, gluster volume status is giving incorrect information. Fix: check whether server is blank, before adding other keys into the dictionary. Change-Id: I926ebdffab330ccef844f23f6d6556e137914047 BUG: 1511782 Signed-off-by: Sanju Rakonde <srakonde@redhat.com> (cherry picked from commit 046c7e3199fca715592762e271e6061ac99b0c4b)
* cluster/afr: Honor default timeout of 5min for analyzing split-brain fileskarthik-us2017-11-301-1/+5
| | | | | | | | | | | | | | | | | | | | Problem: After setting split-brain-choice option to analyze the file to resolve the split brain using the command "setfattr -n replica.split-brain-choice -v "choiceX" <path-to-file>" should allow to access the file from mount for default timeout of 5mins. But the timeout was not honored and was able to access the file even after the timeout. Fix: Call the inode_invalidate() in afr_set_split_brain_choice_cbk() so that it will triger the cache invalidate after resetting the timer and the split brain choice. So the next calls to access the file will fail with EIO. Change-Id: I698cb833676b22ff3e4c6daf8b883a0958f51a64 BUG: 1514380 Signed-off-by: karthik-us <ksubrahm@redhat.com> (cherry picked from commit 933ec57ccda2c1ba5ce6f207313c3b6802e67ca3)
* features/locks: Fix memory leaksXavier Hernandez2017-11-305-5/+11
| | | | | | | | | Backport of: > BUG: 1515161 Change-Id: Ic1d2e17a7d14389b6734d1b88bd28c0a2907bbd6 BUG: 1517689 Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* cluster/dht: make rebalance use truncate incaseSusant Palai2017-11-233-71/+99
| | | | | | | | | | | | | .. the brick file system does not support fallocate. > Change-Id: Id76cda2d8bb3b223b779e5e7a34f17c8bfa6283c > BUG: 1488103 > Signed-off-by: Susant Palai <spalai@redhat.com> Change-Id: Id76cda2d8bb3b223b779e5e7a34f17c8bfa6283c BUG: 1516691 Signed-off-by: Susant Palai <spalai@redhat.com>
* cluster/dht: Don't set ACLs on linkto fileN Balachandran2017-11-201-0/+11
| | | | | | | | | | | | | | | | | The trusted.SGI_ACL_FILE appears to set posix ACLs on the linkto file that is a target of file migration. This can mess up file permissions and cause linkto identification to fail. Now we remove all ACL xattrs from the results of the listxattr call on the source before setting them on the target. > BUG: 1514329 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I56802dbaed783a16e3fb90f59f4ce849f8a4a9b4 BUG: 1515042 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* glusterd: restart the brick if qorum status is NOT_APPLICABLE_QUORUMAtin Mukherjee2017-11-101-1/+2
| | | | | | | | | | | | | | If a volume is not having server quorum enabled and in a trusted storage pool all the glusterd instances from other peers are down, on restarting glusterd the brick start trigger doesn't happen resulting into the brick not coming up. > mainline patch : https://review.gluster.org/#/c/18669/ Change-Id: If1458e03b50a113f1653db553bb2350d11577539 BUG: 1511301 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 635c1c3691a102aa658cf1219fa41ca30dd134ba)
* md-cache: avoid checking the xattr value buffer with string functions.Günther Deschner2017-11-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | xattrs may very well contain binary, non-text data with leading 0 values. Using strcmp for checking empty values is not the appropriate thing to do: In the best case, it might treat a binary xattr value starting with 0 from being cached (and hence also from being reported back with xattr). In the worst case, we might read beyond the end of a data blob that does contain any zero byte. We fix this by checking the length of the data blob and checking the first byte against 0 if the length is one. > Signed-off-by: Guenther Deschner <gd@samba.org> > Pair-Programmed-With: Michael Adam <obnox@samba.org> > Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11 > BUG: 1476324 > Reviewed-on: https://review.gluster.org/17910 > Reviewed-by: Michael Adam <obnox@samba.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Poornima G <pgurusid@redhat.com> > Tested-by: Poornima G <pgurusid@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > (cherry picked from commit ab4ffdac9dec1867f2d9b33242179cf2b347319d) Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11 BUG: 1499892 Signed-off-by: Günther Deschner <gd@samba.org>
* glusterd : introduce timer in mgmt_v3_lockGaurav Yadav2017-11-064-17/+241
| | | | | | | | | | | | | | | | | | Problem: In a multinode environment, if two of the op-sm transactions are initiated on one of the receiver nodes at the same time, there might be a possibility that glusterd may end up in stale lock. Solution: During mgmt_v3_lock a registration is made to gf_timer_call_after which release the lock after certain period of time >mainline patch : https://review.gluster.org/#/c/18437/ Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843 BUG: 1503239 Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
* protocol/server: fix the comparision logic in case of subdir mountAmar Tumballi2017-11-061-30/+30
| | | | | | | | | | | | | | without the fix, the stat entry on a file would return inode==1 for many files, in case of subdir mount This happened with the confusion of return value of 'gf_uuid_compare()', it is more like strcmp, instead of a gf_boolean return value, and hence resulted in the bug. Change-Id: I31b8cbd95eaa3af5ff916a969458e8e4020c86bb BUG: 1505527 Signed-off-by: Amar Tumballi <amarts@redhat.com> (cherry picked from commit 2ade36cd98ea0f5bd2a8f619a19c20438318afaf)
* protocol/client: handle the subdir handshake properly for add-brickAmar Tumballi2017-11-061-1/+9
| | | | | | | | | | There should be different way we handle handshake in case of subdir mount for the first time, and in case of subsequent graph changes. Change-Id: I2a7ba836433bb0a0f4a861809e2bb0d7fbc4da54 BUG: 1505323 Signed-off-by: Amar Tumballi <amarts@redhat.com> (cherry picked from commit 9aa574a51b84717c1f3949ed2e28a49e49840a93)
* md-cache: Use correct xattr keynames for virtual glusterfs ACLs.Günther Deschner2017-11-061-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The "glusterfs.posix_acl." prefix does not catch the glusterfs posix acl xattr keynames which are * "glusterfs.posix.acl" and * "glusterfs.posix.default_acl" Using the GF_POSIX_ACL_ACCESS and GF_POSIX_ACL_DEFAULT defines directly is the savest option. Guenther > BUG: 1476295 > Signed-off-by: Guenther Deschner <gd@samba.org> > Reviewed-on: https://review.gluster.org/17909 > Reviewed-by: Michael Adam <obnox@samba.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Tested-by: Niels de Vos <ndevos@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > (cherry picked from commit 5fe8555800cbc9818e7c976f63499795a378cd8d) > Signed-off-by: Günther Deschner <gd@samba.org> Change-Id: I5aba64b26b6cbec850ea02316dd9f069400e857f BUG: 1499889 Signed-off-by: Günther Deschner <gd@samba.org>
* glusterd: clean up portmap on brick disconnectAtin Mukherjee2017-11-064-11/+46
| | | | | | | | | | | | | | | | | | | | | | | GlusterD's portmap entry for a brick is cleaned up when a PMAP_SIGNOUT event is initiated by the brick process at the shutdown. But if the brick process crashes or gets killed through SIGKILL then this event is not initiated and glusterd ends up with a stale port. Since GlusterD's portmap traversal happens both ways, forward for allocation and backward for registry search, there is a possibility that glusterd might end up running with a stale port for a brick which eventually will end up with clients to fail to connect to the bricks. Solution is to clean up the port entry in case the process is down as part of the brick disconnect event. Although with this the handling PMAP_SIGNOUT event becomes redundant in most of the cases, but this is the safeguard method to avoid glusterd getting into the stale port issues. > mainline patch : https://review.gluster.org/#/c/18541/ Change-Id: I04c5be6d11e772ee4de16caf56dbb37d5c944303 BUG: 1507747 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 30e0b86aae00430823f2523c6efa3c4ebbf0a478)
* glusterd: fix brick restart parallelismAtin Mukherjee2017-11-066-32/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | glusterd's brick restart logic is not always sequential as there is atleast three different ways how the bricks are restarted. 1. through friend-sm and glusterd_spawn_daemons () 2. through friend-sm and handling volume quorum action 3. through friend handshaking when there is a mimatch on quorum on friend import. In a brick multiplexing setup, glusterd ended up trying to spawn the same brick process couple of times as almost in fraction of milliseconds two threads hit glusterd_brick_start () because of which glusterd didn't have any choice of rejecting any one of them as for both the case brick start criteria met. As a solution, it'd be better to control this madness by two different flags, one is a boolean called start_triggered which indicates a brick start has been triggered and it continues to be true till a brick dies or killed, the second is a mutex lock to ensure for a particular brick we don't end up getting into glusterd_brick_start () more than once at same point of time. Change-Id: I292f1e58d6971e111725e1baea1fe98b890b43e2 BUG: 1508283 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 82be66ef8e9e3127d41a4c843daf74c1d8aec4aa)
* glusterd: delete source brick only once in reset-brick commit forceAtin Mukherjee2017-11-021-1/+1
| | | | | | | | | | | | | | While stopping the brick which is to be reset and replaced delete_brick flag was passed as true which resulted glusterd to free up to source brick before the actual operation. This results commit force to fail failing to find the source brickinfo. > mainline patch : https://review.gluster.org/#/c/18581/ Change-Id: I1aa7508eff7cc9c9b5d6f5163f3bb92736d6df44 BUG: 1507877 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 0fb8acaa6ff80c43e46deac0ce66b29ae0df0ca4)
* glusterd: persist brickinfo's port change into glusterd's storeGaurav Yadav2017-11-025-10/+61
| | | | | | | | | | | | | | | | | | Problem: Consider a case where node reboot is performed and prior to reboot brick was listening to 49153. Post reboot glusterd assigned 49152 to brick and started the brick process but the new port was never persisted. Now when glusterd restarts glusterd always read the port from its persisted store i.e 49153 however pmap signin happens with the correct port i.e 49152. Fix: Make sure when glusterd_brick_start is called, glusterd_store_volinfo is eventually invoked. Change-Id: Ic0efbd48c51d39729ed951a42922d0e59f7115a1 BUG: 1507748 Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
* features/worm: Adding check to newloc when doing renameluneo72017-10-251-2/+16
| | | | | | | | | | | | | | | | | | | | | | | Problem: Since rename didn't check if newloc exists and it's retention state it was possible to rename a new file that wasn't in retention over a existing file that was in read-only state. Cherry picked from commit 00a4dc0: > Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229 > BUG: 1484490 > Signed-off-by: luneo7 <luneo7@gmail.com> > Reviewed-on: https://review.gluster.org/18104 > Tested-by: jiffin tony Thottan <jthottan@redhat.com> > Tested-by: Prashanth Pai <ppai@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Prashanth Pai <ppai@redhat.com> > Reviewed-by: Karthik U S <ksubrahm@redhat.com> > Reviewed-by: Amar Tumballi <amarts@redhat.com> Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229 BUG: 1484489 Signed-off-by: luneo7 <luneo7@gmail.com>
* glusterd: documenting server.allow-insecureSanju Rakonde2017-10-251-1/+1
| | | | | | | | | | | | | problem: "server.allow-insecure" is invisible in gluster volume set help. Fix: "server.allow-insecure" is defined as NO_DOC type, chainging it to DOC type solve the problem. Change-Id: I327f1e4c1684ff846deb8b7df07d4d8a09073274 BUG: 1505373 Signed-off-by: Sanju Rakonde <srakonde@redhat.com> (cherry picked from commit c0b08f10ed07bfe06309e31a7fff85cadb733ce2)
* libgfchangelog: Fix possible null pointer dereferenceKotresh HR2017-10-251-6/+6
| | | | | | | | | | | | | | | | If pthread_attr_init fails, gf_msg uses this->name where 'this' is not initialized yet. This patch fixes the same. > Change-Id: Ie004cbe1015a0d62fc3b5512e8954c5606eeeb5f > Signed-off-by: Kotresh HR <khiremat@redhat.com> > BUG: 1505325 (cherry picked from commit 738c38f0efa7b4d4dab0cf23d00589d68e4eb88d) Change-Id: Ie004cbe1015a0d62fc3b5512e8954c5606eeeb5f Signed-off-by: Kotresh HR <khiremat@redhat.com> BUG: 1505856
* cluster/dht: fix crash when deleting directoriesZhang Huan2017-10-251-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | In DHT, after locks on all subvolumes are acquired, it would perform the following steps sequentially, 1. send remove dir on all other subvolumes except the hashed one in a loop; 2. wait for all pending rmdir to be done 3. remove dir on the hashed subvolume The problem is that in step 1 there is a check to skip hashed subvolume in the loop. If the last subvolume to check is actually the hashed one, and step 3 is quickly done before the last and hashed subvolume is checked, by accessing shared context data be destroyed in step 3, would cause a crash. Fix by saving shared data in a local variable to access later in the loop. > BUG: 1490642 > Signed-off-by: Zhang Huan <zhanghuan@open-fs.com> (cherry picked from commit 206120126d455417a81a48ae473d49be337e9463) Change-Id: I8db7cf7cb262d74efcb58eb00f02ea37df4be4e2 BUG: 1505221 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* protocol-auth: use the proper validation methodAmar Tumballi2017-10-251-33/+7
| | | | | | | | | | Currently, server protocol's init and glusterd's option validation methods are different, causing an issue. They should be same for having consistent behavior Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4 BUG: 1501315 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* geo-rep: Filter out volume-mark xattrKotresh HR2017-10-231-0/+1
| | | | | | | | | | | | | | | | | | | The volume-mark xattr, maintained at brick root of slave volume is specific to geo-replication and should be filtered out for all other clients. It should also be filtered out from list getxattr from all mounts including geo-rep mount as it might cause rsync to read and set. > Change-Id: If9eb5a3af18051083c853e70d93b2819e8eea222 > BUG: 1500433 > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit c64fd0d4b0ef313bb44aae68a376ec0c9ee8657a) Change-Id: If9eb5a3af18051083c853e70d93b2819e8eea222 BUG: 1502104 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* mount/fuse : Fix parsing of vol_id for snapshot volumeMohammed Rafi KC2017-10-151-2/+4
| | | | | | | | | | | | | | | | | | | | | For supporting sub-dir mount, we changed the volid. Which means anything after a '/' in volume_id will be considered as sub-dir path. But snapshot volume has vol_id stracture of /snaps/<volname>/<snapname> which has to be considered as during the parsing. Note 1: sub-dir mount is not supported on snapshot volume Note 2: With sub-dir mount changes brick based mount for quota cannot be executed via mount command. It has to be a direct call via glusterfs Backport of> >Change-Id: I0d824de0236b803db8a918f683dabb0cb523cb04 >BUG: 1501235 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Change-Id: I0d824de0236b803db8a918f683dabb0cb523cb04 BUG: 1501238 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> (cherry picked from commit 067f38063e13fc75d4e3f7adf93441d15099c557)
* cluster/ec: Improve performance with xattrop updateSunil Kumar Acharya2017-10-121-24/+102
| | | | | | | | | | | | | | | | Existing EC code updates the xattr on the subvolume in a sequential pattern resulting in very poor performance. With this fix EC now updates the xattr on the subvolume in parallel which improves the xattr update performance. >BUG: 1445663 >Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486 >Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> BUG: 1499150 Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486 Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
* glusterd:Marking all the brick status as stopped when a process goes down in ↵Sanju Rakonde2017-10-121-1/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | brick multiplexing In brick multiplexing environment, if a brick process goes down i.e., if we kill it with SIGKILL, the status of the brick for which the process came up for the first time is only changing to stopped. all other brick statuses are remain started. This is happening because the process was killed abruptly using SIGKILL signal and signal handler wasn't invoked and further cleanup wasn't triggered. When we try to start a volume using force, it shows error saying "Request timed out", since all the brickinfo->status are still in started state, we're waiting for one of the brick process to come up which never going to happen since the brick process was killed. To resolve this, In the disconnect event, We are checking all the processes that whether the brick which got disconnected belongs the process. Once we get the process we are calling a function named glusterd_mark_bricks_stopped_by_proc() and sending brick_proc_t object as an argument. From the glusterd_brick_proc_t we can get all the bricks attached to that process. but these are duplicated ones. To get the original brickinfo we are reading volinfo from brick. In volinfo we will have original brickinfo copies. We are changing brickinfo->status to stopped for all the bricks. >Change-Id: Ifb9054b3ee081ef56b39b2903ae686984fe827e7 >BUG: 1499509 >Signed-off-by: Sanju Rakonde <srakonde@redhat.com> >Reviewed-on: https://review.gluster.org/#/c/18444/ >Smoke: Gluster Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> >cherry picked from commit 9422446d72bc054962d72ace9912ecb885946d49) Change-Id: Ifb9054b3ee081ef56b39b2903ae686984fe827e7 BUG: 1501154 Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* cluster/afr: Make choose-local "reconfigurable"Krutika Dhananjay2017-10-121-0/+11
| | | | | | | | | | | | | | | | Backport of: > Change-Id: Ibab292ba705d993b475cd0303fb3318211fb2500 > Reviewed-on: https://review.gluster.org/18026 > BUG: 1480525 > cherry-picked from commit 1e2d6537875d16b783e3c50ada7ee61487c6d796 With this change, enabling choose-local (which means its state makes transition from "off" to "on") will be effective after the first gfid-lookup on "/" since volume-set was executed. Change-Id: Ibab292ba705d993b475cd0303fb3318211fb2500 BUG: 1501022 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* fuse/readdirp: Remove need_lookup from fuse_readdirp_cbkSusant Palai2017-10-121-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | background: Various xlators used to populate their ctx, on an explicit lookup. That means without a lookup, the translator will have either null or stale data to function. E.g. dht would depend on lookup to create linkto files on the correct node/hashed subvol, afr would rely on this lookup to heal pending data/metadata etc. So to complete above actions a lookup used to be issued on files, even their inode was populated on a readdirp_cbk. This was done by setting the need_lookup flag on all the files those were read on readdirp fop. We tried a small test on "ACL client". For listing 50k files on root itself, it took around 50seconds with readdirp enabled while the same operation took 5-6 seconds with readdirp disabled. Both the times md-cache was enabled. We observed that on the 1st test case (readdirp enabled), post readdirp a getxattr is done. The number of getxattr depends on the number of acl xattrs (I saw requests on these two: system.posix_acl_default, system.posix_acl_access). Since need_lookup flag is set, during fuse_resolve a nameless lookup is executed on the inode(getxattr being inode operation, hence the nameless lookup). Since md-cache does not serve nameless lookup, a network hop is needed for each file, costing the time. With readdirp disabled, the getxattrs are served from md-cache itself(note: we are discussing the 2nd attempt of ls -l use case). _Current affairs around need of lookup for a file to populate it's ctx_: For the xlators on client stack we discussed quite extensively about the need for a lookup fop post readdirp in all three cluster translators - afr, EC and dht. EC and dht don't really need a nameless lookup post readdirp. For afr too, the need for lookup was negated with patch (http://review.gluster.org/6010 - AFRV2), where afr added a function called afr_inode_refresh() which does a lookup and populates its inode context in case a FOP came to AFR without a lookup being issued prior to it. We ran a thread on gluster-devel asking for feedback on the need of explicit lookup post readdirp. For responses refer [1]. Refer [2] for discussions happened on gerrit. After gathering inputs from [1] and [2], it looks like there is no xlator in current state that requires an explicit lookup post readdirp to function properly. * A separate similar patch will be sent for gfapi/nfs/nfs-ganesha. Note: Only file's inode is built with readdirp. [1] http://lists.gluster.org/pipermail/gluster-devel/2017-August/053505.html [2] https://review.gluster.org/#/c/17985/ > Change-Id: Ie1d68ce7bea5e1f8a1fab9a62217f478322554f5 > BUG: 1492996 > Signed-off-by: Susant Palai <spalai@redhat.com> Change-Id: Ie1d68ce7bea5e1f8a1fab9a62217f478322554f5 BUG: 1499123 Signed-off-by: Susant Palai <spalai@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-19 15:23:36 +0000