summaryrefslogtreecommitdiffstats
path: root/xlators/cluster/dht/src
Commit message (Collapse)AuthorAgeFilesLines
* dht/rename : Incase of failure remove linkto file properlyJiffin Tony Thottan2017-01-092-2/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Generally linkto file is created using root user. Consider following case, a user is trying to rename a file which he is not permitted. So the rename fails with EACESS and when rename tries to cleanup the linkto file, it fails. The above issue happens when rename/00.t test executed on nfs-ganesha clients : Steps executed in script * create a file "abc" using root * rename the file "abc" to "xyz" using a non root user, it fails with EACESS * delete "abc" * create directory "abc" using root * again try ot rename "abc" to "xyz" using non root user, test hungs here which slowly leds to OOM kill of ganesha process RCA put forwarded by Du for OOM kill of ganesha Note that when we hit this bug, we've a scenario of a dentry being present as: * a linkto file on one subvol * a directory on rest of subvols When a lookup happens on the dentry in such a scenario, the control flow goes into an infinite loop of: dht_lookup_everywhere dht_lookup_everywhere_cbk dht_lookup_unlink_cbk dht_lookup_everywhere_done dht_lookup_directory (as local->dir_count > 0) dht_lookup_dir_cbk (sets to local->need_selfheal = 1 as the entry is a linkto file on one of the subvol) dht_lookup_everywhere (as need_selfheal = 1). This infinite loop can cause increased consumption of memory due to: 1) dht_lookup_directory assigns a new layout to local->layout unconditionally 2) Most of the functions in this loop do a stack_wind of various fops. This results in growing of call stack (note that call-stack is destroyed only after lookup response is received by fuse - which never happens in this case) Thanks Du for root causing the oom kill and Sushant for suggesting the fix Upstream reference : >Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d >BUG: 1397052 >Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> >Reviewed-on: http://review.gluster.org/15894 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> >(cherry picked from commit 57d59f4be205ae0c7888758366dc0049bdcfe449) Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d BUG: 1401032 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: http://review.gluster.org/16016 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: "replica.split-brain-status" attribute value is not correctMohit Agrawal2016-09-122-12/+210
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: In a distributed-replicate volume attribute "replica.split-brain-status" value does not display split-brain condition though directory is in split-brain. If directory is in split brain on mutiple replica-pairs it does not show full list of replica pairs. Solution: Update the dht_aggregate code to aggregate the xattr value in this specific condition. Fix: 1) function getChoices returns the choices from split-brain status string. 2) function add_opt adding the choices to local buffer to store in dictionary 3) For the key "replica.split-brain-status" function dht_aggregate call dht_aggregate_split_brain_xattr to prepare the list. Test: To verify the patch followed below steps 1) Create a distributed replica volume and create mount point 2) Stop heal daemon 3) Touch file and directories on mount point mkdir test{1..5};touch tmp{1..5} 4) Down brick process on one of the replica set pkill -9 glusterfsd 5) Change permission of dir on mount point chmod 755 test{1..5} 6) Restart brick process on node with force option 7) kill brick process on other node in same replica set 8) Change permission of dir again on mount point chmod 766 test{1..5} 9) Reexecute same step from 4-9 on other replica set also 10) After check heal status on server it will show dir's are in split brain on all replica sets 11) After check the replica.split-brain-status attr on mount point it will show wrong status of split brain. 12) After apply the patch the attribute shows correct value. > Change-Id: Icdfd72005a4aa82337c342762775a3d1761bbe4a > Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> > Reviewed-on: http://review.gluster.org/15201 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > (cherry picked from commit c4e9ec653c946002ab6d4c71ee8e6df056438a04) Change-Id: Ia5234e8a2291a7e8a7211c82368f4df1c99fa099 Backport of commit c4e9ec653c946002ab6d4c71ee8e6df056438a04 BUG: 1375099 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: http://review.gluster.org/15466 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: initialize cbk before attempting inode-linkRaghavendra G2016-08-161-9/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Otherwise inode-link failures in selfheal codepath will result in a crash. This regression was introduced in master as fix to 1334164. But, that patch never made into 3.7. Hence, in essence this patch is 3.7 version of fix to 1334164, minus the regression. > Change-Id: I9061629ae9d1eb1ac945af5f448d0d8b397a5022 > BUG: 1345748 > Signed-off-by: Raghavendra G <rgowdapp@redhat.com> > Reviewed-on: http://review.gluster.org/14707 > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Poornima G <pgurusid@redhat.com> > Reviewed-by: Susant Palai <spalai@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I9061629ae9d1eb1ac945af5f448d0d8b397a6022 BUG: 1366483 Reviewed-on: http://review.gluster.org/15163 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* dht/rebalance: allocate migrator thread pool dynamicallySusant Palai2016-08-051-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | Problems: The maximum number of migratior threads created was static set to "40". And the number of these threads get created in rebalance depends on the number of cores user has. If the number of cores exceeds 40, a crash or memory corruption can be seen. Fix: Make the migratior thread pool dynamic. > Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839 > BUG: 1362070 > Reviewed-on: http://review.gluster.org/15000 > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b) Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839 BUG: 1362070 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/15062 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Fix unsafe iteration on inode->fd_listXavier Hernandez2016-06-241-16/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When DHT traverses the inode->fd_list, it does that in an unsafe way that can generate races with fd_unref() called from other threads. This patch fixes this problem taking the inode->lock and adding a reference to the fd while it's being used outside of the mutex protected region. A minor change in storage/posix has been done to also access the inode->fd_list in a safe way. Backport of: > Change-Id: I10d469ca6a8f76e950a8c9779ae9c8b70f88ef93 > BUG: 1344340 > Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> > Reviewed-on: http://review.gluster.org/14682 > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I10d469ca6a8f76e950a8c9779ae9c8b70f88ef93 BUG: 1346751 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/14734 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/distribute: heal layout in discover codepath tooRaghavendra G2016-06-221-33/+7
| | | | | | | | | | Change-Id: Ic559c220a1f0051e531314d13940604e2dead08c BUG: 1336284 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/14349 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* dht/afr/client/posix: Fail mkdir without gfid-reqPranith Kumar K2016-06-161-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | Do not allow directory creations without gfids as after the directories are created, operations on them fail anyway. So it is better to fail mkdir. >BUG: 1317361 >Change-Id: I8f8e3b38bbded1960b7215bac0432500f7e78038 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/13690 >Smoke: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >(cherry picked from commit b246b07896fefb261c9fb07f3f29f0d03b81b88d) Change-Id: Ibf9c84add7265e3e1755a37958e1de38307624b2 BUG: 1332372 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/14188 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* dht: selfheal should wind mkdir call to subvols with ESTALE errorSakshi Bansal2016-05-261-1/+2
| | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/14496/ > Change-Id: I7140e50263b5f28b900829592c664fa1d79f3f99 > BUG: 1338634 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Change-Id: I7140e50263b5f28b900829592c664fa1d79f3f99 BUG: 1338668 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/14497 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* features/shard: Get hard-link-count in {unlink,rename}_cbk before deleting ↵Krutika Dhananjay2016-05-231-8/+13
| | | | | | | | | | | | | | | shards Backport of http://review.gluster.org/#/c/14334/ Change-Id: I41321d8b00a10f1bd5b0a7b008f673b1aa240d0c BUG: 1337837 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/14450 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* cluster/tier: downgrade max-cycle-time log message to INFODan Lambright2016-05-211-1/+1
| | | | | | | | | | | | | | | | | | | | The "max cycle time" log message was incorrectly logged as an error. Downgrade it to INFO. This is a backport of 14336 > Change-Id: Ia7d074423019fa79443bc6ea694148b7b8da455d > BUG: 1335973 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I29514c66781f49d5c36a0d3ad5dee6ab0c0368cd BUG: 1336470 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/14361 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com>
* dht: rename takes lock on parent directory if destination existsSakshi Bansal2016-05-181-7/+32
| | | | | | | | | | | | | | | | | | | | | | For directory rename if destination exists the source directory is created as a child of the given destination directory. Since the new child directory does not exist take lock on parent of the child directory. Backport of http://review.gluster.org/#/c/14371/ > Change-Id: I24a34605a2cd65984910643ff5462f35e8fc7e71 > BUG: 1336698 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Change-Id: I24a34605a2cd65984910643ff5462f35e8fc7e71 BUG: 1337022 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/14407 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/detach: Clear tier-fix-layout-complete xattr after migration threads joinDan Lambright2016-05-161-33/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously we had wrongly placed the clearing tier-fix-layout-complete xattr before the joining of migration threads. This would lead to situations where failure of clearing the xattr would cause the premature death of migration threads. Now we clear the xattr only after the data movement threads join, ensuring that all migration is done. This is a backport of 14285 > Change-Id: I829b671efa165ae13dbff7b00707434970b37a09 > BUG: 1334839 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I475242e6a05cacd2252dc5c29b160e7abc5d1791 BUG: 1336148 Reviewed-on: http://review.gluster.org/14341 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: N Balachandran <nbalacha@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/detach : During detach check if background fixlayout is doneJoseph Fernandes2016-05-141-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | During detach check if background fixlayout is done, if not done ignore the case and continue detach. Backport of http://review.gluster.org/14147 > Change-Id: I5d5cfc0e73d0eb217fdeab54c432dc4af8bc598d > BUG: 1332136 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/14147 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I2161673cf6861b02a8e323366a13a13587258bef BUG: 1333934 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/14246 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: Joseph Fernandes CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht:remember locked subvol and send unlock to the sameMohammed Rafi KC2016-05-065-21/+221
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During locking we send lock request to cached subvol, and normally we unlock to the cached subvol But with parallel fresh lookup on a directory, there is a race window where the cached subvol can change and the unlock can go into a different subvol from which we took lock. This will result in a stale lock held on one of the subvol. So we will store the details of subvol which we took the lock and will unlock from the same subvol back port of> >Change-Id: I47df99491671b10624eb37d1d17e40bacf0b15eb >BUG: 1311002 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13492 >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Smoke: Gluster Build System <jenkins@build.gluster.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Change-Id: Ia847e7115d2296ae9811b14a956f3b6bf39bd86d BUG: 1333645 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/14236 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Perform NULL check on xdata before dict_get()Krutika Dhananjay2016-05-051-1/+1
| | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/14212 .. to prevent unnecessary logs from gf_msg_callingfn() Change-Id: I443322d26f2b5238320bc14f1ddc94affe030943 BUG: 1333241 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/14216 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* cluster/dht: Handle rmdir failure correctlyN Balachandran2016-05-022-13/+108
| | | | | | | | | | | | | | | | | | DHT did not handle rmdir failures on non-hashed subvols correctly in a 2x2 dist-rep volume, causing the directory do be deleted from the hashed subvol. Also fixed an issue where the dht_selfheal_restore errcodes were overwriting the rmdir error codes. Change-Id: If2c6f8dc8ee72e3e6a7e04a04c2108243faca468 BUG: 1331933 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/14123 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* libglusterfs: Add debug and trace logs for stack traceRaghavendra Talur2016-05-021-1/+2
| | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/13448 commit c4d67a8338b42d6485f49999f310cbb9ed5359c5 It has become very difficult to identify the xlator which returned negative op_ret. Being able to just change the log level and visualize the stack is helpful in such cases. Change-Id: I6545b4802c1ab4d0d230d5e9e036afb2384882e1 BUG: 1330739 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/14099 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* tier/dht: check for rebalance completion for EIO errorMohammed Rafi KC2016-04-281-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When an ongoing rebalance completion check task been triggered by dht, there is a possibility of a race between afr setting subvol as non-readable and dht updates the cached subvol. In this window a write can fail with EIO. Back port of> >Change-Id: I42638e6d4104c0dbe893d1bc73e1366188458c5d >BUG: 1329503 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/14049 >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >Smoke: Gluster Build System <jenkins@build.gluster.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Reviewed-by: Jeff Darcy <jdarcy@redhat.com> (cherry picked from commit a9ccd0c8ea6989c72073028b296f73a6fcb6b896) Change-Id: Ifc741f05a9cfe5c1d9c03085312ebbf21b8b798b BUG: 1330428 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/14070 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/tier: fixes for migration over ec as cold tierDan Lambright2016-04-271-1/+1
| | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11433 The above mentioned patch was backported but it was not complete. This patch backports whatever was left out. For more info, refer to previous backport at http://review.gluster.org/#/c/11652. Change-Id: I6ccda8ca8e43485b9b354341bbfcb302496f632c BUG: 1330765 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/14084 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* afr: replica pair going offline does not require CHILD_MODIFIED eventSakshi Bansal2016-04-271-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As a part of CHILD_MODIFIED event DHT forgets the current layout and performs fresh lookup. However this is not required when a replica pair goes offline as the xattrs can be read from other replica pairs. Hence setting different event to handle replica pair going down. > Backport of http://review.gluster.org/#/c/12573/ > Change-Id: I5ede2a6398e63f34f89f9d3c9bc30598974402e3 > BUG: 1281230 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/12573 > Reviewed-by: Ravishankar N <ravishankar@redhat.com> > Reviewed-by: Susant Palai <spalai@redhat.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Change-Id: Ida30240d1ad8b8730af7ab50b129dfb05264fdf9 BUG: 1283972 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/12767 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* quota: setting 'read-only' option in xdata to instruct DHT to not healSakshi Bansal2016-04-261-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When quota is enabled the quota enforcer tries to get the size of the source directory by sending nameless lookup to quotad. But if the rename is successful even on one subvol or the source layout has anomalies then this nameless lookup in quotad tries to heal the directory which requires a lock on as many subvols as it can. But src is already locked as part of rename. For rename to proceed in brick it needs to complete a cluster-wide lookup. But cluster-wide lookup in quotad is blocked on locks held by rename, hence a deadlock. To avoid this quota sends an option in xdata which instructs DHT not to heal. Backport of http://review.gluster.org/#/c/13988/ > Change-Id: I792f9322331def0b1f4e16e88deef55d0c9f17f0 > BUG: 1252244 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13988 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I792f9322331def0b1f4e16e88deef55d0c9f17f0 BUG: 1328473 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/14031 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* dht/rebalance: Handle GF_DEFRAG_STOPSusant Palai2016-04-261-0/+17
| | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/14004 Problem: On a rebal stop, the migrator threads don't intimate the crawler thread to wake up in case it is waiting on signal from migrator thread. BUG: 1330529 Change-Id: I9019a715c7b4673b8bb5a75d7d33a18add85ce33 Reviewed-by: N Balachandran <nbalacha@redhat.com> Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/14076 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* dht: add "nuke" functionality for efficient server-side deletionJeff Darcy2016-04-261-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of the following two patches (of which the second is a trivial adjustment to a timeout for a test added by the first). http://review.gluster.org/13878 http://review.gluster.org/13935 This turns a special xattr into an rmdir with flags set. When that hits the posix translator on the server side, that causes the file/directory to be moved into the special "landfill" directory. From there, the posix janitor thread will take care of deleting it entirely on the server side - traversing it recursively if necessary. A couple of secondary issues were fixed to make this effective. * FUSE now ensures that setxattr values are NUL terminated. * The janitor thread now gets woken up immediately when something is placed in 'landfill' instead of only when file descriptors need to be closed. * The default landfill-emptying interval was reduced to 10s. To use the feature, issue a setxattr something like this: setfattr -n glusterfs.dht.nuke -v "" /mnt/glusterfs/vol/some_dir The value doesn't actually matter; the mere receipt of a request with this key is sufficient. Some day it might be useful to allow setting a required value as a sort of password, so that only those who know it can access the underlying special functionality. Change-Id: I4132a30d1faa53a6682399ad1d9041e2c4519951 BUG: 1330241 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/14065 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/distribute: detect stale layouts in entry fopsRaghavendra G2016-04-255-27/+645
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dht_mkdir () { first-hashed-subvol = hashed-subvol for "bname" in in-memory layout of "parent"; inodelk (SETLKW, parent, "LAYOUT_HEAL_DOMAIN", "can be any subvol, but we choose first-hashed-subvol randomly"); { begin: hashed-subvol = hashed-subvol for "bname" in in-memory layout of "parent"; hash-range = extract hashe-range from layout of "parent"; ret = mkdir (parent/bname, hashed-subvol, hash-range); if (ret == "hash-value doesn't fall into layout stored on the brick (this error is returned by posix-mkdir)") { refresh_parent_layout (); goto begin; } } inodelk (UNLCK, parent, "LAYOUT_HEAL_DOMAIN", "first-hashed-subvol"); proceed with other parts of dht_mkdir; } posix_mkdir (parent/bname, client-hash-range) { disk-hash-range = getxattr (parent, "dht-layout-key"); if (disk-hash-range != client-hash-range) { fail-with-error ("hash-value doesn't fall into layout stored on the brick"); return 0; } continue-with-posix-mkdir; } Similar changes need to be done for dentry operations like create, symlink, link, unlink, rmdir, rename. These will be addressed in subsequent patches. This patch addresses only mkdir codepath. This change breaks stripe tests, as on some striped subvols dht layout xattrs are not set for some reason. This results in failure of mkdir. Since striped volumes are always created with dht, some tests associated with stripe also fail. So, I am making following tests changes (since stripe is out of maintainance): * modify ./tests/basic/rpc-coverage.t to not to use striped volumes * mark all (2) tests in tests/bugs/stripe/ as bad tests Change-Id: Idd1ae879f24a48303dc743c1bb4d91f89a629e25 BUG: 1329062 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/14040 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* dht: lock on subvols to prevent rename and lookup selfheal raceSakshi2016-04-101-43/+158
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch addresses two races while renaming directories: 1) While renaming src to dst, if a lookup selfheal is triggered it can recreate src on those subvols where rename was successful. This leads to multiple directories (src and dst) having same gfid. To avoid this we must take locks on all subvols with src. 2) While renaming if the dst exists and a lookup selfheal is triggered it will find anomalies in the dst layout and try to heal the stale layout. To avoid this we must take lock on any one subvol with dst. Backport of http://review.gluster.org/#/c/11880/ > Change-Id: I637f637d3241d9065cd5be59a671c7e7ca3eed53 > BUG: 1252244 > Signed-off-by: Sakshi <sabansal@redhat.com> Change-Id: I637f637d3241d9065cd5be59a671c7e7ca3eed53 BUG: 1324381 Signed-off-by: Sakshi <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13917 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: lock on subvols to prevent lookup vs rmdir raceSakshi2016-04-065-88/+435
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a possibility that while an rmdir is completed on some non-hashed subvol and proceeding to others, a lookup selfheal can recreate the same directory on those subvols for which the rmdir had succeeded. Now the deletion of the parent directory will fail with an ENOTEMPTY. To fix this take blocking inodelk on the subvols before starting rmdir. Selfheal must also take blocking inodelk before creating the entry. Backport of http://review.gluster.org/13528 > Change-Id: I168a195c35ac1230ba7124d3b0ca157755b3df96 > BUG: 1245065 > Signed-off-by: Sakshi <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13528 > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Tested-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I168a195c35ac1230ba7124d3b0ca157755b3df96 BUG: 1257894 Signed-off-by: Sakshi <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13915 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* cluster/dht/rebalance: rebalance failure handlingSusant Palai2016-04-051-21/+141
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At current state rebalance aborts basically on any failure like fix-layout of a directory, readdirp, opendir etc. Unless it is not a remove-brick process we can ignore these failures. Major impact: Any failure in the gf_defrag_process_dir means there are files left unmigrated in the directory. Fix-layout(setxattr) failure will impact it's child subtree i.e. the child subtree will not be rebalanced. Settle-hash (commit-hash)failure will trigger lookup_everywhere for immediate children until the next commit-hash. Note: Remove-brick opertaion is still sensitive to any kind of failure. Change-Id: I2f67a490e4e7d06423bb5bc010a1373a74a6af1d BUG: 1318196 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/12013 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/13749 Tested-by: N Balachandran <nbalacha@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com>
* tier/dht : Attach tier fix layout to run in backgroundJoseph Fernandes2016-04-012-24/+247
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Spawn a thread for background fix-layout for tier process. 2. Once the fix-layout is completed a marker xttr is set on the root of volume to mark the completion of the background fixlayout, so that even if the tier process is spawned again, fixlayout will not be issued, if it was completed last time. 3. Please note that promotion of legacy files will happen eventually as the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy files OR the ctr lookup heal happend due to a name lookup. 4. When a detach tier is successful in evacuation data from hot tier, we remove the marker xattr is removed. So that next attach tier runs the background tier fixlayout. what is remaining ? 1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start clear it during detach commit. But the issue is detach commit is a glusterd operation and the volume is not mounted in glusterd. The reason we want to do it in detach commit is that if the admin wants to attach the same tier again, then a background fixlayout will be triggered, which would not be needed. 2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having entries which will be stale when the volume is used, with ctr off (ctr is switched off only when we have detach commit.) Backport of http://review.gluster.org/13491 > Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5 > BUG: 1313228 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/13491 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Tested-by: Joseph Fernandes > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ic28affdf78d2ac0f394f3dd59f0126df7915d609 BUG: 1323016 Reviewed-on: http://review.gluster.org/13879 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht: report constant directory sizeJeff Darcy2016-03-284-1/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Directory size is meaningless. Every filesystem has its own unpredictable way of increasing or decreasing it, based on internal data structures and even transient conditions. Some filesystems (e.g. ext4) never decrease it at all. Others (e.g. btrfs) don't even report it. Very few programs look at it, and those that do are broken. Unfortunately, one such program is GNU tar, which will complain when it sees different values because at different times we got the value from different DHT subvolumes. To avoid such problems, just report a constant value. Backport of http://review.gluster.org/#/c/13770/ > Change-Id: Id64ce917c75b5f7ff50cb55b6e997f3b3556e7e3 > BUG: 1302948 > Original-author: Shyam <srangana@redhat.com> > Signed-off-by: Jeff Darcy <jdarcy@redhat.com> > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13770 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: Id64ce917c75b5f7ff50cb55b6e997f3b3556e7e3 BUG: 1312721 Original-author: Shyam <srangana@redhat.com> Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13831 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: update attr information in refresh layout to avoidSakshi Bansal2016-03-281-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | stale timestamp Consider the scenario where an mkdir has just created the directory but has not healed it yet. A parallel lookup on this entry will find anomalies and trigger a selfheal which will sample the ctime of the directory after the mkdir phase. Meanwhile the mkdir has completed setting the layout and updated the ctime. The selfheal then sees the layout to be healed and returns with the ctime it got after the mkdir phase which has now become stale. However if the lookup happens to unwind before the mkdir then the inode associated with lookup will get linked in the inode table which has the stale ctime. To avoid this selfheal must do an iatt_merge in refresh layout to get the latest timestamp irrespective of whether it needs to heal the layout or not. Backport of http://review.gluster.org/#/c/13781/ > Change-Id: I3634c3978bcc1710705f44b48f3876601682d33e > BUG: 1302948 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13781 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Change-Id: I3634c3978bcc1710705f44b48f3876601682d33e BUG: 1312721 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13830 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* Tier: Avoiding stale entries from causing demotion to stophari gowtham2016-03-222-3/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | back-port of : http://review.gluster.org/#/c/13501/ When the parent GFID is a stale entry, the lookup on this parent fails and this in turn fails the demotion process. This patch will make the stale entry error to be skipped. Situation for pargfid to be stale: Consider a folder from a tar file. Once the tar file is untared the files in the tar-file will start to demote. when the demotion is under progress, if we tend to delete the actual folder, then the files under it which are undergoing demotion will do a lookup on the parent which was deleted and become stale entry. This stale entry fails the Lookup and this will fail the demotion of the other files(not from tar) that are supposed to be demoted. >Change-Id: I3d47c32c4077526d477a25912b0135bab98b23fc >BUG: 1311178 >Signed-off-by: hari gowtham <hgowtham@redhat.com> >Reviewed-on: http://review.gluster.org/13501 >Tested-by: hari gowtham <hari.gowtham005@gmail.com> >Smoke: Gluster Build System <jenkins@build.gluster.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Change-Id: I80a77d149180e2a05b3d7943f370b3ef162e545b BUG: 1317366 Signed-off-by: Hari Gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/13692 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: hari gowtham <hari.gowtham005@gmail.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* TIER: stopping the tierd when the volume goes downhari gowtham2016-03-221-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | back-port of : http://review.gluster.org/#/c/13646/ If there are large number of files to be migrated and by this time if the volume goes down, then the tierd has to be stopped. But on a huge query file list it keeps checking for each file before stopping. If the volume comes up before the old tierd dies then due to the presence of old tierd new one won't be created. After the old one completes the task, it dies and the status ends up as failed. This patch will check if the status is still running and then let it continue its work. Else it will stop running the tierd. >Change-Id: I6522a4e2919e84bf502b99b13873795b9274f3cd >BUG: 1315659 >Signed-off-by: hari gowtham <hgowtham@redhat.com> >Reviewed-on: http://review.gluster.org/13646 >Tested-by: Dan Lambright <dlambrig@redhat.com> >Smoke: Gluster Build System <jenkins@build.gluster.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I8326dbe5edaaea921e5401f39d148aac322c78d0 BUG: 1318498 Signed-off-by: hari <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/13756 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: hari gowtham <hari.gowtham005@gmail.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: break out of iterating query file once cycle time endsDan Lambright2016-03-161-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When iterating the query file during migration, tiering should break out of the loop once cycle time completes. Otherwise it may be possible to stay in the loop for a long time. If that happens updates to files will become stale and have not impact migration. This is a backport of 13284 > Change-Id: Ib60cf74bc84e8646e6a0da21ff04954b1b83c414 > BUG: 1301227 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/13284 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia305784b0aa570b70e54f532bf5d68a91b0cf305 BUG: 1318203 Reviewed-on: http://review.gluster.org/13750 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: hari gowtham <hari.gowtham005@gmail.com> Tested-by: Dan Lambright <dlambrig@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: Additional details in error messagesN Balachandran2016-03-071-37/+41
| | | | | | | | | | | | | | | | | | | | | | | | | Added file path/gfid when available to the tier log messages to make debugging easier. > Change-Id: I22dda329367df2b846dcf254594312c997b66083 > BUG: 1273043 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13114 > Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I5cbf8407fb7c493c4be1dda65c0f722ede33487f BUG: 1311865 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13518 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier: Fix unused-but-set-variable warningRaghavendra Talur2016-03-031-2/+0
| | | | | | | | | | | | | Backport of http://review.gluster.org/13591 Change-Id: I20c205849f511acf4797a8a53ea59bfaad60bd35 BUG: 1314548 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/13598 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* dht: mkdir must unwind with latest ctimeSakshi Bansal2016-02-292-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently fops like mkdir used the the ctime it gets after creating the directory entry. But setting layout also updates the ctime of a directory. Hence DHT must get the ctime after the setxattr call and unwind with the latest ctime to avoid mismatch in time seen by applications like tar. Backport of http://review.gluster.org/#/c/13352/ > Change-Id: Iecbbe3aac5244af5da9788b48ccf299ca56b4bae > BUG: 1302948 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13352 > Smoke: Gluster Build System <jenkins@build.gluster.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1312721 Change-Id: Ie09342d3705b40ce98b2935f05ad4402f74ba069 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13536 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/tier: Create linkfiles to hardlinks correctlyN Balachandran2016-02-273-1/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a bug in the way hardlinks are handled in tiered volumes. Ideally, the tier linkto files on the cold tier to files that are hardlinks to each other on the hot tier, should themselves be hardlinks of each other. As they are not, they end up being files with the same gfid but different names for the cold tier dht, and end up overwriting the cached-subvol information stored in the dht inode-ctx. > Change-Id: Ic658a316836e6a1729cfea848b7d212674b0edd2 > BUG: 1305277 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13391 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit eb362c74db84d95aac07febf0d888bd98b3fb2b9) Change-Id: I5dd98e6d248619147974a630d7d72e1942a1cc83 BUG: 1311836 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13517 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier : Fixed wrong variable comparisonN Balachandran2016-02-271-2/+15
| | | | | | | | | | | | | | | | | | | | | | | | | The wrong variable was being checked to determine the watermark value. > Change-Id: If4c97fa70b772187f1fcbdf5193e077cb356a8b1 > BUG: 1303895 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13357 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I0a98e0efbc093a727912107038477239e6d85765 BUG: 1306129 Reviewed-on: http://review.gluster.org/13516 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Tested-by: mohammed rafi kc <rkavunga@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht: cleanup dict and free memory in rename code pathSakshi Bansal2016-02-251-0/+4
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/13392/ > Change-Id: I2458e18197bdf7565563a85e9021b5b2850c1825 > BUG: 1303945 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13392 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1304889 Change-Id: Ibda96769be2bd5b2708066d83363633fd7286929 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13513 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: loc should store proper gfidSakshi Bansal2016-02-231-0/+1
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/12574 > Change-Id: Ic1393d44a9ed4aaba23d7c9ddea45977b9dae5e4 > BUG: 1281265 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/12574 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1308410 Change-Id: I6577971f3db139eccf3b8ca3b25e66d292a6a150 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13443 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: file rename must take blocking inode locksSakshi Bansal2016-02-221-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/13366 Currently DHT takes non-blocking locks for file rename. Due to this during parallel renames some clients fail with EBUSY or ESTALE errors. Hence to avoid application discontinuity file rename must take blocking inode locks. > Change-Id: I986e9d08b3be359f20b1a3e1564e049b0f3dffd3 > BUG: 1304966 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13366 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I986e9d08b3be359f20b1a3e1564e049b0f3dffd3 BUG: 1310544 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13481 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/create: store TIER_LINKFILE_GFID in xattr dictionaryMohammed Rafi KC2016-02-221-17/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In tier_create, a new key TIER_LINKFILE_GFID was introduced to avoid a race in stale linkfile deletion. Storing this key in xattr dictionary instead of using local->params dictionary. Because local->params dictionary was also used to create the file before stale linkfile deletion, that leads posix_create to fail, trying to set the added key as extended attributes Back port of> >Change-Id: I24fecb62b47bee65a1e86103925a67d13304c5df >BUG: 1290677 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13130 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I301c159ee51c77d3876e30f216d0375d761eb91e BUG: 1295359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13410 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/create: Dynamically allocate gfid memoryMohammed Rafi KC2016-02-221-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently we are storing the memory as a static pointer. There is a chance to go that variable in out of scope. So we should allocate in Dynamic way. Backport of> >Change-Id: I096876deb8055ac3a44681599591a0a032bc0c24 >BUG: 1290677 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13102 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ibc5cde24db4846ef3152366cdaca81dd1d0e7b21 BUG: 1295359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13162 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier:delete the linkfile if data file creation failsMohammed Rafi KC2016-02-224-82/+288
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we are creating data file in a hot subvolume then we will create a linkfile in cold subvolume. Linkfile creation happens first. If linkfile creation was successful and data file creation failed, then linkfile in cold subvolume will become stale This patch will delete the linkfile as well, if data file creation fails Also this code duplicates dht_create to make tier_create backport of> >Change-Id: I377a90dad47f288e9576c7323b23cf694a91a7a3 >BUG: 1290677 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/12948 >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I3689e8ee387fb6731b4b3a7fe8f8190143482eaa BUG: 1295359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13161 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/unlink: symlink failed to unlinkMohammed Rafi KC2016-02-221-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | during unlink of a file, we will get stat just after deleting the file, to see if the file is under migration or not but this stat call will fail for symlink if the actual file was deleted. So it is better not to send stat request from client if it is a symlink as we are not migrating symlink back port of> >Change-Id: Idc033b24fa3522b5261e579889d2195b43419682 >BUG: 1293963 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13074 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ic53a9ad29722927f44004d2e7289c807c7485df0 BUG: 1295347 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13159 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier:unlink during migrationMohammed Rafi KC2016-02-225-75/+374
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | files deleted during promotion were not deleting as the files are moving from hashed to non-hashed On deleting a file that is undergoing promotion, the unlink call is not sent to the dst file as the hashed subvol == cached subvol. This causes the file to reappear once the migration is complete. This patch also fixes a problem with stale linkfile deleting. Backport of> >Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5 >BUG: 1282390 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/12829 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1) Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d BUG: 1282388 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12991 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht: set proper errno when hashed subvol is not foundSakshi Bansal2016-02-211-5/+5
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/12559/ > Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479 > BUG: 1279705 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/12559 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Susant Palai <spalai@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479 BUG: 1285829 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13351 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Skip subvols if no layout presentN Balachandran2016-02-211-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Running "rm -rf" on a tiered volume sometimes caused the client to crash because dht_readdirp_cbk referenced a NULL layout for the hot tier subvol. Now, entries are skipped if the layout is NULL. This can cause "rm -rf" to fail with ENOTEMPTY rmdir failures. > Change-Id: Idd71a9d0f7ee712899cc7113bbf2cd3dcb25808b > BUG: 1307208 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13440 > Smoke: Gluster Build System <jenkins@build.gluster.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit bfd8f92490454f261330da34565334ed5decd0e1) Change-Id: I34e413d669fdfba343446e84fef5d8e9dd7bd1da BUG: 1308400 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13441 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/tier : Reset watermarks in tierN Balachandran2016-02-191-9/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | A node which contains only cold bricks and has detected that the high watermark value has been breached on the hot tier will never reset the watermark to the correct value. The promotion check will thus always fail and no promotions will occur from that node. > Change-Id: I0f0804744cd184c263acbea1ee50cd6010a49ec5 > BUG: 1303895 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13341 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit 545f4ed2c7195a21210e6a055c27c1b7a115e18c) Change-Id: Iba3aa9c57cf5828ab87140c2c8257146a8772836 BUG: 1306129 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13411 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: allow db queries to be interruptableDan Lambright2016-02-183-76/+167
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | A query to the database may take a long time if the database has many entries. The tier daemon also sends IPC calls to the bricks which can run slowly, espcially in RHEL6. While it is possible to track down each such instance, the snapshot feature should not be affected by database operations. It requires no migration be underway. Therefore it is okay to pause tiering at any time except when DHT is moving a file. This fix implements this strategy by monitoring when control passes to DHT to migrate a file using the GF_XATTR_FILE_MIGRATE_KEY trigger. If it is not, the pause operation is successful. > Change-Id: I21f168b1bd424077ad5f38cf82f794060a1fabf6 > BUG: 1287842 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/13104 > Reviewed-by: Joseph Fernandes > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I667e0af24eaa66afefa860c4d73b324e4f39b997 BUG: 1288352 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/13199 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
generated by cgit (git 2.34.1) at 2024-04-24 09:19:34 +0000