summaryrefslogtreecommitdiffstats
path: root/xlators/storage/posix/src
Commit message (Collapse)AuthorAgeFilesLines
* posix: Reorganize posix xlator to prepare for reuse with rioShyamsundarR2017-12-0210-8338/+8830
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Split out entry and inode/fd based FOPs into separate files from posix.c 2. Split out common routines (init, fini, reconf, and such) into its own file, from posix.c 3. Retain just the method assignments in posix.c (such that posix2 for RIO can assign its own methods in the future for entry operations and such) 4. Based on the split in (1) and (2) split out posix-handle.h into 2 files, such that macros that are needed for inode ops are in one and rest are in the other If the split is done as above, posix2 can compile with its own entry ops, and hence not compile, the entry ops as split in (1) above. The split described in (4) can again help posix2 to define its own macros to make entry and inode handles, thus not impact existing POSIX xlator code. Noted problems - There are path references in certain cases where quota is used (in the xattr FOPs), and thus will fail on reuse in posix2, this needs to be handled when we get there. - posix_init does set root GFID on the brick root, and this is incorrect for posix2, again will need handling later when posix2 evolves based on this code (other init checks seem fine on current inspection) Merge of experimental branch patches with the following gerrit change-IDs > Change-Id: I965ce6dffe70a62c697f790f3438559520e0af20 > Change-Id: I089a4d9cf470c2f9c121611e8ef18dea92b2be70 > Change-Id: I2cec103f6ba8f3084443f3066bcc70b2f5ecb49a Fixes gluster/glusterfs#327 Change-Id: I0ccfa78559a7c5a68f5e861e144cf856f5c9e19c Signed-off-by: ShyamsundarR <srangana@redhat.com>
* storage/posix : options to override umaskSubha sree Mohankumar2017-12-022-12/+137
| | | | | | | | | | | | | | | | | | | | | Options "create-mask" and "create-directory-mask" are added to remove the mode bits set on a file or directory when its created. Default value of these options is 0777. Options "force-create-mode" and "force-create-directory" sets the default permission for a file or directory irrespective of the clients umask. Default value of these options is 0000. Command to set option: volume set <volume name> storage.<option-name> <value> The valid value range from 0000 to 0777. Updates #301 Change-Id: Ia33d13f2117202ca55a056c747ccc3674eb8bae1 Signed-off-by: Subha sree Mohankumar <smohanku@redhat.com>
* posix: Convert posix_fs_health_check asynchrnously to save timestampMohit Agrawal2017-12-013-16/+99
| | | | | | | | | | | Problem: Sometime posix_fs_health_check thread is blocked on write/read call while backend device deleted abruptly. Solution: To resolve it convert code to update timestamp asynchrnously. BUG: 1501132 Change-Id: Id68ea6a572bf68fbf437e1d9be5221b63d47ff9c Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* posix: Change GD_OP_VERSION to 3_13_0 from 3_12_0 for storage.reserveMohit Agrawal2017-11-301-1/+1
| | | | | | | | | | | Problem: Change GD_OP_VERSION to 3_13_0 from 3_12_0 for option storage.reserve Solution: Actually feature was merged in 3.13.0 branch so GD_OP_VERSION needs to change from 3_12_0 to 3_13_0 BUG: 1518508 Change-Id: I5856ab3447b465879ec068cecb0933b91dd697f9 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* posix: Fix coverity issues in several posix functionsMohit Agrawal2017-11-292-63/+110
| | | | | | | | | | | | | | Fixes issues 528, 763, 778, 792, 793, 86, 28, 29, 30, 39, 42, 769, 783, 794, 795 from the report at [1]. [1]: https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-10-30-9aa574a5/html/ Note: Apart from coverity resolve other issues in posix_get(f)xattr more cleaner way. BUG: 789278 Change-Id: If0737492198481ad7a8d75a3801c862fd61b8c6e Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* storage/posix: Update xlator options tableKaushal M2017-11-241-13/+41
| | | | | | | Updates #302 Change-Id: Ib8100c69267202266a7f03d0d632d45afb61d946 Signed-off-by: Kaushal M <kaushal@redhat.com>
* libglusterfs: Handle FS errors gracefullyPranith Kumar K2017-11-222-2/+15
| | | | | | | | | | | | | | | Problem: FS sometimes doesn't give the expected return values. We need our common functions to guard against this. Example BUG: https://bugzilla.redhat.com/show_bug.cgi?id=864401 Fix: When the return value is not as per specification, change the return value to -1 and errno to EIO BUG: 1469487 Change-Id: I14739ab2e5ae225b1a91438b87f8928af56f2934 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* Disable gfid2path by default on NetBSDEmmanuel Dreyfus2017-11-211-0/+11
| | | | | | | | | | | NetBSD storage of extended attributes for UFS1 badly scales when the list of extended attributes names rises. gfid2path can add as many extended attributes names as we have files, hence we keep it disabled for performance sake. Change-Id: Id77b5f5ceb4d5eba1b3362b4b9fc693450ffbc2b Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> BUG: 1129939
* Coverity Issue: PW.INCLUDE_RECURSION in several filesGirjesh Rajoria2017-11-092-2/+0
| | | | | | | | | | | | | | Coverity ID: 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 423, 424, 425, 426, 427, 428, 429, 436, 437, 438, 439, 440, 441, 442, 443 Issue: Event include_recursion Removed redundant, recursive includes from the files. Change-Id: I920776b1fa089a2d4917ca722d0075a9239911a7 BUG: 789278 Signed-off-by: Girjesh Rajoria <grajoria@redhat.com>
* posix: Ignore disk space reserve check for internal FOPSMohit Agrawal2017-10-305-36/+36
| | | | | | | | | | | | | Problem: Currently disk space reserve check is applicable for internal FOP also it needs to be ignore for internal FOP. Solution: Update the DISK_SPACE_CHECK_AND_GOTO macro at posix component. Macro will call only while key "GLUSTERFS_INTERNAL_FOP_KEY" exists in xdata. BUG: 1506083 Change-Id: I2b0840bbf4fa14bc247855b024ca136773d68d16 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* geo-rep: Filter out volume-mark xattrKotresh HR2017-10-131-0/+1
| | | | | | | | | | | | | The volume-mark xattr, maintained at brick root of slave volume is specific to geo-replication and should be filtered out for all other clients. It should also be filtered out from list getxattr from all mounts including geo-rep mount as it might cause rsync to read and set. Change-Id: If9eb5a3af18051083c853e70d93b2819e8eea222 BUG: 1500433 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* storage/posix: Adding implementation for posix_do_futimeskarthik-us2017-10-051-8/+37
| | | | | | | | | Adding the implementation for the posix_do_futimes function which is not complete in the current implementation and giving the ENOSYS error. Change-Id: I9cfc95a7ea293b0a2df8efd4ac80d0120b3120e4 BUG: 1350406 Signed-off-by: karthik-us <ksubrahm@redhat.com>
* Posix: fix additional inode refhari gowtham2017-09-291-4/+1
| | | | | | | | | | | Problem: In the iteration, the inode is being ref-ed twice and unref-ed once. this leads to ref leak. Fix: assign the parent to the inode instead of referencing it. Change-Id: Ib154b12d38ad68220f8f5288bbc50081beccc2b9 BUG: 1496379 Signed-off-by: hari gowtham <hgowtham@redhat.com>
* Posix: Unused valueSanju Rakonde2017-09-261-1/+0
| | | | | | | | | | In line number 5179 we are storing -1 into op_ret variable. Before we use the value we are overwriting the same variable in line number 5339. So we are removing the value assigning statement. Change-Id: I8c6dae9f6b0f9f1e3c09d0744d451b9296d12db8 BUG: 789278 Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* Fix the type of len, a warning is triggeredMichael Scherer2017-09-081-1/+1
| | | | | | | | | | | | | | | | | | | clang show this warning on FreeBSD: posix.c:133:17: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] A closer look on the code show that sys_readlink is returning a ssize_t, not a size_t, who is unsigned. Change-Id: Idb6a440ea44cc2168b0dd85f791c7955caa67c8c BUG: 1488909 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/18220 Tested-by: Michael Scherer <misc@fedoraproject.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* feature/posix: Enabled gfid2path by defaultKotresh HR2017-08-291-13/+3
| | | | | | | | | | | | | | | | Enable gfid2path feature by default. The basic performance tests are carried out and it doesn't show significant depreciation. The results are updated in issue. Updates: #139 Change-Id: I5f1949a608d0827018ef9d548d5d69f3bb7744fd Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17950 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* posix: add null gfid checksRavishankar N2017-08-083-0/+43
| | | | | | | | | | | | | | | | | | ...in file/dir creation and lookup codepaths. The check is relaxed for fops coming from trash xlator at the moment until trash has client side logic to send the create fops with gfid-req. Also fixed the missing trash pid assignment in creates sent by trash xlator. Without this, truncated files won't be moved to .trashcan. Change-Id: Ieddd7f0634850e7c7010e4fbb4ad1eead35888c8 BUG: 1478297 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/17975 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* cluster/dht: Check for open fd only on EBADFN Balachandran2017-08-081-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | DHT fd based fops used to check if the fd was open on the cached subvol before winding the call. However, this introduced a performance regression of about 30% for reads. This check was introduced to handle cases where files were migrated while IOs were happening. As this is not the common case, dht will now check if the fd is open on the cached subvol only if the call fails with EBADF. This will prevent a performance hit where a rebalance is not running. Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808 BUG: 1476665 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/17976 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Susant Palai <spalai@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* posix: add sanity checks for removing the gfid symlink for directoriesRavishankar N2017-08-041-14/+56
| | | | | | | | | | | | | | | ...during mkdir and rmdir. Otherwise, during entry self-heal, the directory could be left out without a .glusterfs symlink causing fops like opendir, readdir to fail. The only chance the missing symlink will be created is when a fresh lookup comes on it. Change-Id: I2e1cf1bce8962ea80187edd8f6d73e0a09cf9f8e BUG: 1477169 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/17945 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
* storage/posix: Add virtual xattr to fetch path from gfidKotresh HR2017-07-285-0/+338
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The gfid2path infra stores the "pargfid/bname" as on xattr value for each non directory entry. Hardlinks would have a separate xattr. This xattr key is internal and is not exposed to applications. A virtual xattr is exposed for the applications to fetch the path from gfid. Internal xattr: trusted.gfid2path.<xxhash> Virtual xattr: glusterfs.gfidtopath getfattr -h -n glusterfs.gfidtopath /<aux-mnt>/.gfid/<gfid> If there are hardlinks, it returns all the paths separated by ':'. A volume set option is introduced to change the delimiter to required string of max length 7. gluster vol set gfid2path-separator ":::" Updates: #139 Change-Id: Ie3b0c3fd8bd5333c4a27410011e608333918c02a Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17785 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
* posix: Needs to reserve disk space to prevent the brick from getting fullMohit Agrawal2017-07-255-6/+217
| | | | | | | | | | | | | | | | | | | | Problem: Currently there is no option available at posix xlator to save the disk from getting full Solution: Introduce a new option storage.reserve at posix xlator to configure disk threshold.posix xlator spawn a thread to update the disk space status in posix private structure and same flag is checked by every posix fop before start operation.If flag value is 1 then it sets op_errno to ENOSPC and goto out from the fop. BUG: 1471366 Change-Id: I98287cd409860f4c754fc69a332e0521bfb1b67e Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17780 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* storage/posix: Use the ret value of posix_gfid_heal()Krutika Dhananjay2017-07-252-10/+13
| | | | | | | | | | | | | | | | ... to make the change in commit acf8cfdf truly useful. Without this, a race between entry creation fops and lookup at posix layer can cause lookups to fail with ENODATA, as opposed to ENOENT. Change-Id: I44a226872283a25f1f4812f03f68921c5eb335bb BUG: 1472758 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/17821 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* posix: option to handle the shared bricks for statvfs()Amar Tumballi2017-07-242-1/+36
| | | | | | | | | | | | | | | | | | | | Currently 'storage/posix' xlator has an option called option `export-statfs-size no`, which exports zero as values for few fields in `struct statvfs`. In a case of backend brick shared between multiple brick processes, the values of these variables should be `field_value / number-of-bricks-at-node`. This way, even the issue of 'min-free-disk' etc at different layers would also be handled properly when the statfs() sys call is made. Fixes #241 Change-Id: I2e320e1fdcc819ab9173277ef3498201432c275f Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/17618 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* posix/gfid2path: Block access to gfid2path xattr via mountKotresh HR2017-07-244-2/+44
| | | | | | | | | | | | | | | | gfid2path xattr is an internal xattr and should not be allowed to modify by other applications via gluster mount. This patch blocks the same. Updates: #139 Change-Id: Id2cb29797ee1bd77e0e0d2203a47469fd7203355 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17744 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* libglusterfs: Name threads on creationRaghavendra Talur2017-07-193-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set names to threads on creation for easier debugging. Output of top -H -p <PID-OF-GLUSTERFSD> Before: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd After: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustertimer 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustermemsweep 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc0 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc1 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll0 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteridxwrker 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteriotwr0 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrssign 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrswrker 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterclogecon 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd0 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd1 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd2 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixjan 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixfsy 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll1 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll2 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixhc Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703 BUG: 1254002 Updates: #271 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: https://review.gluster.org/11926 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* storage/posix: Don't allow gfid/volume-id xattr to be removedPranith Kumar K2017-07-183-140/+134
| | | | | | | | | | | | | | | | | | | | | | | Problem: Bulk xattr removal doesn't check if the xattrs that are coming in xdata have gfid/volume-id xattrs, so there is potential for bulkremovexattr removing gfid/volume-id. I also observed that bulkremovexattr is not available for fremovexattr. Fix: Do proper checks in bulk removexattr to remove gfid/volume-id. Refactor [f]removexattr to reduce the differences. BUG: 1470489 Change-Id: Ia845b31846a149500111c0996646e648f72cdce6 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17765 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Anuradha Talur <atalur@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
* posix: brick process crash after stop the volume while brick mux is onMohit Agrawal2017-07-131-1/+3
| | | | | | | | | | | | | | | | | Problem: sometime brick process is getting crash after stop the volume while brick mux is enabled and no. of volumes are high Solution: In posix notify at the time close mount_lock dir , dir handle needs to set NULL to avoid the reuse of same dir handle. BUG: 1470533 Change-Id: Ifd41c20b3c597317851f91049a7c801949840b16 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17767 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* cluster/ec: Get size of file in EC [f]xattropPranith Kumar K2017-07-131-0/+5
| | | | | | | | | | | | | | | | | | | | Problem: For allowing parallel writes we shouldn't depend on ia_size to be same for all the bricks in each write_cbk(). But we need to make sure backend size is correct on all the bricks and no crashes/manual modifications happened. Fix: At the time of get_size_version() we do 1 check to make sure size of the file is same across the bricks. From then on the FOPs will give the status of the fop, so we rely on this information to keep which bricks are good/bad. Updates #251 Change-Id: I1df645347e2e9f2e09cfa4411b6cc305d7f4e4e5 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17741 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
* storage/posix: Handle [f]xattrop xdataPranith Kumar K2017-07-132-26/+43
| | | | | | | | | | | | Updates #251 Change-Id: I13d89c3b5dc39aa0a232a70be8ec6b64394cfa6e Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17740 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
* storage/posix: New gfid2path infraKotresh HR2017-07-105-2/+193
| | | | | | | | | | | | | | | | | | | | | | | | | With this infra, a new xattr is stored on each entry creation as below. trusted.gfid2path.<xxhash> = <pargfid>/<basename> If there are hardlinks, multiple xattrs would be present. Fops which are impacted: create, mknod, link, symlink, rename, unlink Option to enable: gluster vol set <VOLNAME> storage.gfid2path on Updates: #139 Change-Id: I369974cd16703c45ee87f82e6c2ff5a987a6cc6a Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17488 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* posix: Avoid one extra call of l(get|list)xattr call after use buffer in ↵Mohit Agrawal2017-07-032-177/+300
| | | | | | | | | | | | | | | | | | | | | posix_getxattr Problem: In posix xlator posix_(f)getxattr is calling system call(sys_lgetxattr) two times to fetch the xattr value. Solution: After use the extra buffer for first time calling we can avoid second attempt of system call(sys_lgetxattr) calling in posix_getxattr for most of the xattrs. BUG: 1460659 Change-Id: I0d8da776c5bc86653d874a4629a73bbf65c621b8 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17520 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kinglong Mee
* dht/hardlink : Remove stale linkto file incase of failureJiffin Tony Thottan2017-06-221-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a similar issue fixed for rename in https://review.gluster.org/#/c/16016/ For hardlinks, if cached and hashed subvolumes are different, then it will first create linkto file in hashed using root permission, but actually hardlink creation fails with EACESS and stale linkto file is never removed.All the followup hardlink calls with file name will result ESTALE because linktofile creation fails with EEXIST and follow up lookup on linkto file returns gfid-mismatching(old linkto file) and finally fails with ESTALE Steps to produce : (From link/00.t test from posix-testsuite) Steps executed in script * create a file "abc" using root * change the ownership of file to a non root user * create hardlink "link" for "abc" using a non root user, it fails with EACESS * delete "abc" * create directory "abc" using root * again try to create hadrlink "link" for "abc" using non root user, fails with ESTALE Also tried to fix other bugs in dht_linkfile_create_cbk() and posix_lookup. Thanks Susant for the help in debugging the issue and suggestion for this patch. Change-Id: I7a5a1899d3fd1fdb13578b37f9d52a084492e35d BUG: 1452084 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: https://review.gluster.org/17331 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* posix: Revert modifying op_errno in __posix_fd_ctx_getRavishankar N2017-06-191-10/+6
| | | | | | | | | | | | | | | | | https://review.gluster.org/#/c/17414/ converted ENOENT to EBADFD because ENOENT is not a valid error for fd based operations, but this apparently breaks dht rebalance behaviour (see comments in the backport 17517. So reverting that part of the change. Change-Id: Idcf5c65a47b096a3766cf7f20ca938d988572052 BUG: 1456582 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/17565 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* glusterfsd: Deletion of brick dir throw emerg msgs after stop volumeMohit Agrawal2017-06-092-3/+4
| | | | | | | | | | | | | | | | | | | | Problem: Deletion of brick directories throw emerg messages after stop volume while brick mux is enabled. Solution: Modify the posix health check monitor thread code to handled correctly. BUG: 1459781 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Change-Id: I2d22a84f9a98b0da261e5fb7850ba1368f3601d7 Reviewed-on: https://review.gluster.org/17492 Tested-by: MOHIT AGRAWAL <moagrawa@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* glusterfs: Not able to mount running volume after enable brick mux and ↵Mohit Agrawal2017-05-312-15/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | stopped any volume Problem: After enabled brick mux if any volume has down and then try ot run mount with running volume , mount command is hung. Solution: After enable brick mux server has shared one data structure server_conf for all associated subvolumes.After down any subvolume in some ungraceful manner (remove brick directory) posix xlator sends GF_EVENT_CHILD_DOWN event to parent xlatros and server notify updates the child_up to false in server_conf.When client is trying to communicate with server through mount it checks conf->child_up and it is FALSE so it throws message "translator are not yet ready". From this patch updated structure server_conf to save child_up status for xlator wise. Another improtant correction from this patch is cleanup threads from server side xlators after stop the volume. BUG: 1453977 Change-Id: Ic54da3f01881b7c9429ce92cc569236eb1d43e0d Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17356 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* posix: use the correct op_errnoRavishankar N2017-05-312-8/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | Problem: If readdir/fstat was performed on a directory that was removed, posix_fd_ctx_get() fails with ENOENT but we incorrectly use the ret value (-1 in this case) as op_errno, logging "Operation not permitted" messages in the brick logs. Also in case of fstat, the -1 op_errno was also propagated to the client via stack unwind, causing the message to appear in protocol/client logs as well. Fix: Use the right op_errno in readdir, fstat and writev. Also, if posix_fd_ctx_get() failed with ENOENT, convert it into EBADF because ENOENT is not a valid error for an fd operation. Change-Id: Ie43c0789d5040ec73b7cf885d015a183b8c64d70 BUG: 1456582 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/17414 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* cluster/ec: Implement FALLOCATE FOP for ECSunil Kumar Acharya2017-05-231-5/+5
| | | | | | | | | | | | | | | FALLOCATE file operations is not implemented in the existing EC code. This change set implements it for EC. BUG: 1448293 Change-Id: Id9ed914db984c327c16878a5b2304a0ea461b623 Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> Reviewed-on: https://review.gluster.org/15200 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* brick mux: Detach brick on posix health check failureAtin Mukherjee2017-05-152-4/+23
| | | | | | | | | | | | | | With brick mux enabled, we'd need to detach a particular brick if the underlying backend has gone bad. This patch addresses the same. Change-Id: Icfd469c7407cd2d21d02e4906375ec770afeacc3 BUG: 1450630 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/17287 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* posix: Send SIGKILL in 2nd attemptAtin Mukherjee2017-05-091-2/+2
| | | | | | | | | | | | | | | Commit 21c7f7ba changed the signal from SIGKILL to SIGTERM for the 2nd attempt to terminate the brick process if SIGTERM fails. This patch fixes this problem. Change-Id: I856df607b7109a215f2a2a4827ba3ea42d8a9729 BUG: 1444596 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: https://review.gluster.org/17208 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* glusterd: socketfile & pidfile related fixes for brick multiplexing featureMohit Agrawal2017-05-091-9/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: While brick-muliplexing is on after restarting glusterd, CLI is not showing pid of all brick processes in all volumes. Solution: While brick-mux is on all local brick process communicated through one UNIX socket but as per current code (glusterd_brick_start) it is trying to communicate with separate UNIX socket for each volume which is populated based on brick-name and vol-name.Because of multiplexing design only one UNIX socket is opened so it is throwing poller error and not able to fetch correct status of brick process through cli process. To resolve the problem write a new function glusterd_set_socket_filepath_for_mux that will call by glusterd_brick_start to validate about the existence of socketpath. To avoid the continuous EPOLLERR erros in logs update socket_connect code. Test: To reproduce the issue followed below steps 1) Create two distributed volumes(dist1 and dist2) 2) Set cluster.brick-multiplex is on 3) kill glusterd 4) run command gluster v status After apply the patch it shows correct pid for all volumes BUG: 1444596 Change-Id: I5d10af69dea0d0ca19511f43870f34295a54a4d2 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/17101 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* Refactor the code for posix_handle_pairMichael Scherer2017-05-041-20/+8
| | | | | | | | | | | | | | | | | | It was no longer compiling on Darwin, and triggered a cppcheck error: [xlators/storage/posix/src/posix-helpers.c:1097]: (error) Invalid number of character '{' when these macros are defined: 'GF_DARWIN_HOST_OS'. Change-Id: I986609cc6cfc13ddea0ed35d7776736171f40a41 BUG: 789278 Signed-off-by: Michael Scherer <misc@redhat.com> Reviewed-on: https://review.gluster.org/16729 Tested-by: Michael Scherer <misc@fedoraproject.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* cluster/dht: Make rebalance honor min-free-diskSusant Palai2017-04-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test: Manual created files of size 1K on 2 brick(of size 1GB) setup . added a brick of size 16GB. set min-free-disk to 12GB(so that first two bricks won't receive any files). removed one of the 1st brick of size 1GB. Logs from test: [2017-04-12 08:52:08.196484] W [MSGID: 0] [dht-rebalance.c:895:__dht_check_free_space] 0-test1-dht: Write will cross min-free-disk for file - /tile32 on subvol - test1-client-1. Looking for new subvol. [2017-04-12 08:52:08.196904] I [MSGID: 0] [dht-rebalance.c:925:__dht_check_free_space] 0-test1-dht: new target found - test1-client-2 for file - /tile32 - Post migration we have two files. The new destination (/brick/1) has the data file [root@vm1 ~]# ll /brick/1/tile32 -rw-r--r--. 2 root root 0 Apr 12 14:22 /brick/1/tile32 - On the old target the linkto file is there with linkto xattr pointing to /brick/1 [root@vm1 ~]# ll /tmp/2/tile32 ---------T. 2 root root 1000 Apr 12 14:22 /tmp/2/tile32 [root@vm1 ~]# getfattr -m . -de text /tmp/2/tile32 getfattr: Removing leading '/' from absolute path names security.selinux="unconfined_u:object_r:user_tmp_t:s0" trusted.gfid="����:Aс�#�/'b2" trusted.glusterfs.dht.linkto="test1-client-2" Marking ./tests/features/worm_sh.t as bad test. Reason being, this patch failed on master branch as well and it has nothing to do with rebalance/remove-brick. BUG: 1441508 Change-Id: I90bae251cda3d957a49cdceda90cd08311a392fb Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17034 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* geo-rep: filter out xtime attribute during getxattrSaravanakumar Arumugam2017-04-113-8/+61
| | | | | | | | | | | | | | | | | | | | | | georep gsyncd's xtime needs to filtered irrespective of any process access. This way, we can avoid (unnecessarily)syncing xtime attribute to slave, which may raise permission denied errors. test case modified to check for xtime xattr only in backend. Change-Id: I2390b703048d5cc747d91fa2ae884dc55de58669 BUG: 1353952 Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/14880 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Kotresh HR <khiremat@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* storage/posix: Use granular mutex locks for pgfid update syscallsKrutika Dhananjay2017-03-123-11/+54
| | | | | | | | | | | Change-Id: Ie5d635951c483d858dc4be2a90fb24b8b5f4f02d BUG: 1421938 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16869 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* posix: use nanosecond accuracy when availableNiklas Hambüchen2017-03-071-7/+30
| | | | | | | | | | | | | | | | | | | | | | | | Programs that set mtime, such as `rsync -a`, don't work correctly on GlusterFS, because it sets the nanoseconds to 000. This creates problems for incremental backups, where files get accidentally copied again and again. For example, consider `myfile` on an ext4 system, being copied to a GlusterFS volume, with `rsync -a` and then `cp -u` in turn. You'd expect that after the first `rsync -a`, `cp -u` agrees that the file need not be copied. BUG: 1422074 Change-Id: I89c7b6a73e2e06c02851ff76b7e5cdfaa271e985 Signed-off-by: Niklas Hambüchen <mail@nh2.me> Reviewed-on: https://review.gluster.org/16667 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Jeff Darcy <jdarcy@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* storage/posix: Use more granular mutex locks for atomic writesKrutika Dhananjay2017-03-033-8/+33
| | | | | | | | | | | | | Change-Id: I7a5167de77fabf19c5151775b553913a1af5a765 BUG: 1421938 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16785 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* storage/posix: Set ret value correctly before exitingKrutika Dhananjay2017-03-011-1/+3
| | | | | | | | | | | Change-Id: I07c3a21c1c0625a517964693351356eead962571 BUG: 1427404 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16792 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* storage/posix: Execute syscalls in xattrop under different locksKrutika Dhananjay2017-02-274-36/+111
| | | | | | | | | | | | | | | | | | | | | ... and not inode->lock. This is to prevent the epoll thread from *potentially* being blocked on this lock in the worst case for extended period elsewhere in the brick stack, while the syscalls in xattrop are being performed under the same lock by a different thread. This could potentially lead to ping-timeout, if the only available epoll thread is busy waiting on the inode->lock, thereby preventing it from picking up the ping request from the client(s). Also removed some unused functions. Change-Id: I2054a06701ecab11aed1c04e80ee57bbe2e52564 BUG: 1421938 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16462 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* posix: Fix creation of files with S_ISVTX on FreeBSDXavier Hernandez2017-02-181-3/+3
| | | | | | | | | | | | | | | | | | | On FreeBSD the S_ISVTX flag is completely ignored when creating a regular file. Since gluster needs to create files with this flag set, specialy for DHT link files, it's necessary to force the flag. This fix does this by calling fchmod() after creating a file that must have this flag set. Change-Id: I51eecfe4642974df6106b9084a0b144835a4997a BUG: 1411228 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: https://review.gluster.org/16417 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* dht : file rename operation is successful but log has error File existsMohit Agrawal2017-01-092-7/+20
| | | | | | | | | | | | | | | | | | | | Problem: file rename operation is successful but log has error 'key:trusted.glusterfs.dht.linkto error:File exists' Solution: The error comes in log only when file already has set this xattr (trusted.glusterfs.dht.linkto) before run rename operation and in that case sys_lsetxattr throws this kind of message. To avoid the error message in logs update the condition in posix_handle_pair. BUG: 1404905 Change-Id: Iafd8cb45f9d7f4fe247e297a6ef0af978a8d0b30 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: http://review.gluster.org/16185 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-26 02:54:28 +0000