summaryrefslogtreecommitdiffstats
path: root/xlators/cluster/dht
Commit message (Collapse)AuthorAgeFilesLines
...
* gfapi: return pre/post attributes from glfs_pread/pwriteKinglong Mee2018-02-121-2/+3
| | | | | | | | | | | | | | | As nfs-ganesha, a wcc data contains pre/post attributes is return in read/write rpc reply. nfs-ganesha get those attributes by two getattr between the real read/write right now. But, gluster has return pre/post attributes from glusterfsd, those attributes are skipped in syncop/gfapi, if gfapi return them, the upper user (nfs-ganesha) can use them directly without any duplicate getattr. Updates: #389 Change-Id: I7b643ae4241cfe2aeb17063de00192d81674024a Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
* performance/io-threads: expose io-thread queue depthsVarsha Rao2018-02-081-1/+4
| | | | | | | | | | | | | | | | | | | | The following release-3.8-fb branch patch is upstreamed: > io-stats: Expose io-thread queue depths > Commit ID: 69509ee7d2 > https://review.gluster.org/#/c/18143/ > By Shreyas Siravara <sshreyas@fb.com> Changes in this patch: - Replace iot_pri_t with gf_fop_pri_t - Replace IOT_PRI_{HI, LO, NORMAL, MAX, LEAST} with GF_FOP_PRI_{HI, LO, NORMAL, MAX, LEAST} - Use dict_unref() instead of dict_destroy() This patch is required to forward port io-threads namespace patch. Updates: #401 Change-Id: I1b47a63185a441a30fbc423ca1015df7b36c2518 Signed-off-by: Varsha Rao <varao@redhat.com>
* cluster/dht: Unlink linkto files as rootN Balachandran2018-02-061-3/+7
| | | | | | | | | | | Non-privileged users cannot delete linkto files. However the failure to unlink a stale linkto causes DHT to fail the lookup with EIO and hence prevent access to the file. Change-Id: Id295362d41e52263790694602f36f1219f0646a2 BUG: 1542318 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Cleanup on fallocate failureN Balachandran2018-02-051-1/+17
| | | | | | | | | | | It looks like fallocate leaves a non-empty file behind in case of some failures. We now truncate the file to 0 bytes on failure in __dht_rebalance_create_dst_file. Change-Id: Ia4ad7b94bb3624a301fcc87d9e36c4dc751edb59 BUG: 1541916 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Fixed a leak in inode_refN Balachandran2018-02-021-3/+2
| | | | | | | | Introduced by commit d9f773ba719397c128 Change-Id: I3f3103a5a80daed7562ace72e5aa53b77e74fb94 BUG: 1541264 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: avoid overwriting client writes during migrationSusant Palai2018-02-024-12/+138
| | | | | | | | | | | | | | | | | | | | | | | | For more details on this issue see https://github.com/gluster/glusterfs/issues/308 Solution: This is a restrictive solution where a file will not be migrated if a client writes to it during the migration. This does not check if the writes from the rebalance and the client actually do overlap. If dht_writev_cbk finds that the file is being migrated (PHASE1) it will set an xattr on the destination file indicating the file was updated by a non-rebalance client. Rebalance checks if any other client has written to the dst file and aborts the file migration if it finds the xattr. updates gluster/glusterfs#308 Change-Id: I73aec28bc9dbb8da57c7425ec88c6b6af0fbc9dd Signed-off-by: Susant Palai <spalai@redhat.com> Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Fixed leak in dht_populate_inode_for_dentryN Balachandran2018-02-022-4/+10
| | | | | | | | | | Fixed an issue in dht_populate_inode_for_dentry where a layout is set in the inode without checking if it is already set. This overwrites the value each time without freeing the already existing layout. Change-Id: I651bf539a0b82b4ddc4c355890c16a8e91f5f1fd BUG: 1541264 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Skip '..' for the volume root dirN Balachandran2018-01-241-0/+5
| | | | | | | | | | | | dht_populate_inode_for_dentry tries to update the layout for the '..' entry when listing the root of the volume. This entry does not correspond to an entry in the volume and therefore does not have a gfid or a layout on disk, causing layout processing to fail. Change-Id: I2b7470e1c5e20d87b5545160697f24d041045140 BUG: 1537457 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* core: fix some of the dict_{get,set} with proper APIsAmar Tumballi2018-01-174-6/+6
| | | | | | | updates #220 Change-Id: I6e25dbb69b2c7021e00073e8f025d212db7de0be Signed-off-by: Amar Tumballi <amarts@redhat.com>
* cluster/dht: Update options for gd2N Balachandran2018-01-121-15/+40
| | | | | | | | | Update DHT options for GD2 Updates gluster/glusterfs#302 Change-Id: Ia597fe364e97edd7bcf72d89f4ccdd50713a8837 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Change datatype of search_unhashed variableVarsha Rao2018-01-101-1/+1
| | | | | | | | | | Variable search_unhashed is of type boolean, change it to integer type. This fixes the warning increment of a boolean expression. BUG: 1531987 Change-Id: Ibf153f6a9ad704da38bff346b6a21a71323ed9bb Signed-off-by: Varsha Rao <varao@redhat.com>
* cluster/dht: Use percentages for space checkN Balachandran2018-01-021-5/+20
| | | | | | | | | | | | | | | | | With heterogenous bricks now being supported in DHT we could run into issues where files are not migrated even though there is sufficient space in newly added bricks which just happen to be considerably smaller than older bricks. Using percentages instead of absolute available space for space checks can mitigate that to some extent. Marking bug-1247563.t as that used to depend on the easier code to prevent a file from migrating. This will be removed once we find a way to force a file migration failure. Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647 BUG: 1529440 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Add migration checks to dht_(f)xattropN Balachandran2017-12-265-47/+326
| | | | | | | | | | | | The dht_(f)xattrop implementation did not implement migration phase1/phase2 checks which could cause issues with rebalance on sharded volumes. This does not solve the issue where fops may reach the target out of order. Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c BUG: 1471031 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* dht: Fill first_up_subvol before use in dht_opendirPoornima G2017-12-151-0/+5
| | | | | | | | Reported by: Sam McLeod Change-Id: Ic8f9b46b173796afd70aff1042834b03ac3e80b2 BUG: 1512437 Signed-off-by: Poornima G <pgurusid@redhat.com>
* all: Simplify component message id's definitionXavier Hernandez2017-12-141-1153/+141
| | | | | | | | | This patch creates a new way of defining message id's that is easier and less error prone because it doesn't require so many manual changes each time a new component is defined or a new message created. Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* glusterfs: Use gcc builtin ATOMIC operator to increase/decreate refcount.Mohit Agrawal2017-12-123-37/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: In glusterfs code base we call mutex_lock/unlock to take reference/dereference for a object.Sometime it could be reason for lock contention also. Solution: There is no need to use mutex to increase/decrease ref counter, instead of using mutex use gcc builtin ATOMIC operation. Test: I have not observed yet how much performance gain after apply this patch specific to glusterfs but i have tested same with below small program(mutex and atomic both) and get good difference. static int numOuterLoops; static void * threadFunc(void *arg) { int j; for (j = 0; j < numOuterLoops; j++) { __atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL); } return NULL; } int main(int argc, char *argv[]) { int opt, s, j; int numThreads; pthread_t *thread; int verbose; int64_t n = 0; if (argc < 2 ) { printf(" Please provide 2 args Num of threads && Outer Loop\n"); exit (-1); } numThreads = atoi(argv[1]); numOuterLoops = atoi (argv[2]); if (1) { printf("\tthreads: %d; outer loops: %d;\n", numThreads, numOuterLoops); } thread = calloc(numThreads, sizeof(pthread_t)); if (thread == NULL) { printf ("calloc error so exit\n"); exit (-1); } __atomic_store (&glob, &n, __ATOMIC_RELEASE); for (j = 0; j < numThreads; j++) { s = pthread_create(&thread[j], NULL, threadFunc, NULL); if (s != 0) { printf ("pthread_create failed so exit\n"); exit (-1); } } for (j = 0; j < numThreads; j++) { s = pthread_join(thread[j], NULL); if (s != 0) { printf ("pthread_join failed so exit\n"); exit (-1); } } printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED)); exit(0); } time ./thr_count 800 800000 threads: 800; outer loops: 800000; glob value is 640000000 real 1m10.288s user 0m57.269s sys 3m31.565s time ./thr_count_atomic 800 800000 threads: 800; outer loops: 800000; glob value is 640000000 real 0m20.313s user 1m20.558s sys 0m0.028 Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* cluster/dht: fix memory leaks in rebalanceSusant Palai2017-12-111-12/+19
| | | | | | | | | | | | From code reading it was found that in gf_defrag_process_dir, GF_FREE was called directly on dir_dfmeta->equeue leading to leaks of memory for list of entries read from all the local subvols in case of a failure. This patch frees the entries read from all the local subvols. Change-Id: If5e8f557372a8fc2af86628b401e8de1b54986a1 BUG: 1430305 Signed-off-by: Susant Palai <spalai@redhat.com>
* dht: Send an event when disks get fullAnkit raj2017-12-091-4/+22
| | | | | | | | | | Send an event if DHT determines that a subvol is getting full or running out of inodes. Change-Id: Ie026f4ee1832b5df1e80b16cb949b2cc31a25d6f Bug: 1440659 Signed-off-by: Ankit raj <anraj@redhat.com> Signed-off-by: N Balachandran <nbalacha@redhat.com>
* dht/crypt/tier: Fix use of booleans as integersShyamsundarR2017-12-062-16/+6
| | | | | | BUG: 1520974 Change-Id: I19ea40c888e88a7a4ac271168ed1820c2075be93 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* cluster/dht: Fix build error due to switch statement on a booleanShreyas Siravara2017-12-051-16/+5
| | | | | Change-Id: Idf672b435e389baada732f609398404479306909 BUG: 1520974
* cluster/dht: populate inode in dentry for single subvolume dhtRaghavendra G2017-12-022-1/+69
| | | | | | | | | | ... in readdirp response if dentry points to a directory inode. This is a special case where the entire layout is stored in one single subvolume and hence no need for lookup to construct the layout Change-Id: I44fd951e2393ec9dac2af120469be47081a32185 BUG: 1492625 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: don't overfill the buffer in readdir(p)Raghavendra G2017-12-021-3/+18
| | | | | | | | | | | | | | | | | | | | Superflous dentries that cannot be fit in the buffer size provided by kernel are thrown away by fuse-bridge. This means, * the next readdir(p) seen by readdir-ahead would have an offset of a dentry returned in a previous readdir(p) response. When readdir-ahead detects non-monotonic offset it turns itself off which can result in poor readdir performance. * readdirp can be cpu-intensive on brick and there is no point to read all those dentries just to be thrown away by fuse-bridge. So, the best strategy would be to fill the buffer optimally - neither overfill nor underfill. Change-Id: Idb3d85dd4c08fdc4526b2df801d49e69e439ba84 BUG: 1492625 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* Tier: Stop tierd for detach starthari gowtham2017-12-011-3/+10
| | | | | | | | | | | | | | | | | | | Problem: tierd was stopped only after detach commit This makes the detach take a longer time. The detach demotes the files to the cold brick and if the promotion frequency is hit, then the tierd starts to promote files to hot tier again. Fix: stop tierd after detach start so the files get demoted faster. Note: the is_tier_enabled was not maintained properly. That has been fixed too. some code clean up has been done. Signed-off-by: hari gowtham <hgowtham@redhat.com> Change-Id: I532f7410cea04fbb960105483810ea3560ca149b BUG: 1446381
* dht: coverity fix in dht-rebalance.ckarthik-us2017-11-301-1/+0
| | | | | | | | | | | Fixed UNUSED_VALUE warning in dht_migrate_file. Issue ID: 526 From: http://download.gluster.org/pub/gluster/glusterfs/ static-analysis/master/glusterfs-coverity/2017-11-30-eb013e4c Change-Id: I37395e8ce7088742501424fcce918f0ee8ab4f3d BUG: 789278 Signed-off-by: karthik-us <ksubrahm@redhat.com>
* tier: coverity fixes in tier.chari gowtham2017-11-231-25/+16
| | | | | | | | | | | | fixes coverity issues: 127, 83, 312, 314, 48, and 506 from https://download.gluster.org/pub/gluster/glusterfs/static-analysis /master/glusterfs-coverity/2017-10-30-9aa574a5/html/ Change-Id: Ifb206a8758790faf96619bcc9961dcf169aaad25 BUG: 789278 Signed-off-by: hari gowtham <hgowtham@redhat.com>
* cluster/dht: Serialize mds update code path with lookup unwind in selfhealMohit Agrawal2017-11-223-306/+216
| | | | | | | | | | | | | | | | Problem: Sometime test case ./tests/bugs/bug-1371806_1.t is failing on centos due to race condition between fresh lookup and setxattr fop. Solution: In selfheal code path we do save mds on inode_ctx, it was not serialize with lookup unwind. Due to this behavior after lookup unwind if mds is not saved on inode_ctx and if any subsequent setxattr fop call it has failed with ENOENT because no mds has found on inode ctx.To resolve it save mds on inode ctx has been serialize with lookup unwind. BUG: 1498966 Change-Id: I8d4bb40a6cbf0cec35d181ec0095cc7142b02e29 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* cluster/dht: make rebalance use truncate incaseSusant Palai2017-11-223-71/+99
| | | | | | | | | .. the brick file system does not support fallocate. Change-Id: Id76cda2d8bb3b223b779e5e7a34f17c8bfa6283c BUG: 1488103 Signed-off-by: Susant Palai <spalai@redhat.com>
* cluster/dht: Coverity fixes for dht-rebalance.ckarthik-us2017-11-211-80/+54
| | | | | | | | | | | | | | | | | | Warning Functions DEADCODE gf_defrag_handle_migrate_error gf_defrag_get_entry gf_defrag_process_dir gf_defrag_start_crawl dht_migrate_file UNUSED_VALUE migrate_special_files dht_migrate_file FORWARD_NULL gf_tier_do_fix_layout Change-Id: I6f408585b83a267581a4273dae7c22b8993163d5 BUG: 789278 Signed-off-by: karthik-us <ksubrahm@redhat.com>
* cluster/dht: dead code coverity fixKartik_Burmee2017-11-211-3/+0
| | | | | | | | | | | | issue: Execution cannot reach this statement: "call_stub_destroy(stub);" function: dht_mkdir_hashed_cbk fix: removed the statement and the corresponding 'if' condition block. Change-Id: I3e31056ee489ede6864e51a8e666edc7da3c175f BUG: 789278 Signed-off-by: Kartik_Burmee <kburmee@redhat.com>
* cluster/dht: Don't set ACLs on linkto fileN Balachandran2017-11-191-0/+11
| | | | | | | | | | | | | | The trusted.SGI_ACL_FILE appears to set posix ACLs on the linkto file that is a target of file migration. This can mess up file permissions and cause linkto identification to fail. Now we remove all ACL xattrs from the results of the listxattr call on the source before setting them on the target. Change-Id: I56802dbaed783a16e3fb90f59f4ce849f8a4a9b4 BUG: 1514329 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* tier: coverity fix for tier-common.chari gowtham2017-11-161-28/+5
| | | | | | | | | fix for the coverity id: 258, 162 Change-Id: I35ba21e37e186b7c1ce54faf5b24f48858e6fc70 BUG: 789278 Signed-off-by: hari gowtham <hgowtham@redhat.com>
* cluster/dht: Check for NULL local before using itAshish Pandey2017-11-081-2/+6
| | | | | | | | | | | Coverity ID: 261 Coverity ID: 257 Check if local is NULL or not Change-Id: I570f87f32e1347f891c6a02ee5641fcc5a262a96 BUG: 789278 Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* cluster/dht: Check for NULL before using variableAshish Pandey2017-11-062-4/+6
| | | | | | | | | | | | Coverity ID: 245 Check statvfs received as cbk before using it Coverity ID: 228 Check NULL loc before freeing it. Change-Id: I1b153ed5e7b81bcf7033bf710808e95908dcfef4 BUG: 789278 Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* core: make gf_boolean_t a C99 bool instead of an enumJeff Darcy2017-11-032-4/+10
| | | | | | | | | | | | This reduces the space used from four bytes to one, and allows new code to use familiar C99 types/values interoperably with our old cruft. It does *not* change current declarations or code; that will be left for a separate - much larger - patch. Updates: #80 Change-Id: I5baedd17d3fb05b38f0d8b8bb9dd62824475842e Signed-off-by: Jeff Darcy <jdarcy@fb.com>
* xlator/tier: flood of -Wformat-truncation warnings with gcc-7.1v4.0dev1Kaleb S. KEITHLEY2017-11-011-8/+9
| | | | | | | | | | | | | | | | Starting in Fedora 26 which has gcc-7.1.x, -Wformat-trunction is enabled with -Wformat, resulting in a flood of new warnings. This many warnings is a concern because it makes it hard(er) to see other warnings that should be addressed. An example is at https://kojipkgs.fedoraproject.org//packages/glusterfs/3.12.0/1.fc28/data/logs/x86_64/build.log For more info see https://review.gluster.org/#/c/18267/ Change-Id: Id7ef8e0dedd28ada55f72c03d91facbe1c9888bd BUG: 1492849 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* cluster/dht: fix crash when deleting directoriesZhang Huan2017-10-161-2/+4
| | | | | | | | | | | | | | | | | | | | | In DHT, after locks on all subvolumes are acquired, it would perform the following steps sequentially, 1. send remove dir on all other subvolumes except the hashed one in a loop; 2. wait for all pending rmdir to be done 3. remove dir on the hashed subvolume The problem is that in step 1 there is a check to skip hashed subvolume in the loop. If the last subvolume to check is actually the hashed one, and step 3 is quickly done before the last and hashed subvolume is checked, by accessing shared context data be destroyed in step 3, would cause a crash. Fix by saving shared data in a local variable to access later in the loop. Change-Id: I8db7cf7cb262d74efcb58eb00f02ea37df4be4e2 BUG: 1490642 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* dht: free memory allocated in dht_init() and dht_init_subvolumes()Niels de Vos2017-10-111-1/+17
| | | | | | | | | When glfs_fini() is called, DHT fails to free all memory allocations which result in a considerable leak. Change-Id: I37c6de5c93ca4516266dbe8288b4a416f5589901 BUG: 1443145 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* cluster/dht: Don't store the entire uuid for subvolsN Balachandran2017-10-104-19/+40
| | | | | | | | | | | | Comparing the uuid string of the local node against that stored in the local_subvol information is inefficient, especially as it is done for every file to be migrated. The code has now been changed to set the value of info to 1 if the nodeuuid is that of the node making the comparison so this becomes an integer comparison. Change-Id: I7491d59caad3b71dbf5facc94dcde0cd53962775 BUG: 1451434 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht : User xattrs are not healed after brick stop/startMohit Agrawal2017-10-047-111/+2077
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: In a distributed volume custom extended attribute value for a directory does not display correct value after stop/start or added newly brick. If any extended(acl) attribute value is set for a directory after stop/added the brick the attribute(user|acl|quota) value is not updated on brick after start the brick. Solution: First store hashed subvol or subvol(has internal xattr) on inode ctx and consider it as a MDS subvol.At the time of update custom xattr (user,quota,acl, selinux) on directory first check the mds from inode ctx, if mds is not present on inode ctx then throw EINVAL error to application otherwise set xattr on MDS subvol with internal xattr value of -1 and then try to update the attribute on other non MDS volumes also.If mds subvol is down in that case throw an error "Transport endpoint is not connected". In dht_dir_lookup_cbk| dht_revalidate_cbk|dht_discover_complete call dht_call_dir_xattr_heal to heal custom extended attribute. In case of gnfs server if hashed subvol has not found based on loc then wind a call on all subvol to update xattr. Fix: 1) Save MDS subvol on inode ctx 2) Check if mds subvol is present on inode ctx 3) If mds subvol is down then call unwind with error ENOTCONN and if it is up then set new xattr "GF_DHT_XATTR_MDS" to -1 and wind a call on other subvol. 4) If setxattr fop is successful on non-mds subvol then increment the value of internal xattr to +1 5) At the time of directory_lookup check the value of new xattr GF_DHT_XATTR_MDS 6) If value is not 0 in dht_lookup_dir_cbk(other cbk) functions then call heal function to heal user xattr 7) syncop_setxattr on hashed_subvol to reset the value of xattr to 0 if heal is successful on all subvol. Test : To reproduce the issue followed below steps 1) Create a distributed volume and create mount point 2) Create some directory from mount point mkdir tmp{1..5} 3) Kill any one brick from the volume 4) Set extended attribute from mount point on directory setfattr -n user.foo -v "abc" ./tmp{1..5} It will throw error " Transport End point is not connected " for those hashed subvol is down 5) Start volume with force option to start brick process 6) Execute getfattr command on mount point for directory 7) Check extended attribute on brick getfattr -n user.foo <volume-location>/tmp{1..5} It shows correct value for directories for those xattr fop were executed successfully. Note: The patch will resolve xattr healing problem only for fuse mount not for nfs mount. BUG: 1371806 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Change-Id: I4eb137eace24a8cb796712b742f1d177a65343d5
* dht: fix a coverity error of type - UNREACHABLEKamal Mohanan2017-09-281-3/+0
| | | | | | | | | | | | | | Problem: Unreachable assignment statement at dht-rebalance.c:1040 Fix: Delete line dht-rebalance.c:1040. The goto statements at lines 1037 and 1031 are also deleted since both branches of the if statement finally go to the same immediately-following label anyway. Change-Id: I5f47ea99244cae2a0a9f2aec7284faadf2ea286a BUG: 789278 Signed-off-by: Kamal Mohanan <kmohanan@redhat.com>
* Fix a coverity error of checker type: CHECKED_RETURNKamal Mohanan2017-09-261-1/+1
| | | | | | | | | | | Problem: dht_frame_return was being called without checking the return value. Solution: Typecast the value returned by the function to void. Change-Id: Idfc6a7ed467d1c8f5f8d09ec26d9059f3d23b760 BUG: 789278 Signed-off-by: Kamal Mohanan <kmohanan@redhat.com>
* dht: add FOP check to dht_file_setattr_cbkRavishankar N2017-09-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | Problem: bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a non fd-based fop, namely setattr. This caused dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL. Fix: Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk only for dht_fsetattr and not dht_setattr or dht_setattr2 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99 BUG: 1488399 Reviewed-on: https://review.gluster.org/18208 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Susant Palai <spalai@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* cluster/dht: Log files skipped by rebalanceN Balachandran2017-08-302-1/+19
| | | | | | | | | | | | | | | | | There was no easy way to find out which files were skipped during a rebalance. Rebalance now logs a message for every skipped file using msgid 109126, making it easier to find all files that were skipped. Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e BUG: 1480445 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18021 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: hari gowtham <hari.gowtham005@gmail.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Aggregate xattrs only for dirs in dht_discover_cbkN Balachandran2017-08-301-2/+11
| | | | | | | | | | | | | | | | | If dht_discover finds data files on more than one subvol, racing calls to dht_discover_cbk could end up calling dht_aggregate_xattr which could delete dictionary data that is being accessed by higher layer translators. Fixed to call dht_aggregate_xattr only for directories and consider only the first file to be found. Change-Id: I4f3d2a405ec735d4f1bb33a04b7255eb2d179f8a BUG: 1484709 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18137 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Reorder dir operations in gf_defrag_fix_layoutN Balachandran2017-08-201-76/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, rebalance performed a fix-layout on a directory before healing its subdirectories. If there were a lot of subdirs, it could take a while before all subdirs were created on the newly added bricks. As dht_readdirp only lists dirs from their hashed subvol, those dirs which hashed to the newly added bricks but were not yet created on them were not listed. Now, the child dirs are listed and processed before the layout of the parent is fixed. This introduces a change in behaviour where files in subdirs are migrated before those in parent directories. Credit: Shyam <srangana@redhat.com> Github issue: #239 Change-Id: I8ae7f24a510754cd8d1b31e5d608bcf1928599e2 BUG: 1248393 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18045 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: EBADF handling for fremovexattr and fsetxattrN Balachandran2017-08-093-3/+44
| | | | | | | | | | | | | Add EBADF handling for dht_fremovexattr and dht_fsetxattr. Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309 BUG: 1476665 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/17999 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Check for open fd only on EBADFN Balachandran2017-08-084-160/+160
| | | | | | | | | | | | | | | | | | | | | | | | | | DHT fd based fops used to check if the fd was open on the cached subvol before winding the call. However, this introduced a performance regression of about 30% for reads. This check was introduced to handle cases where files were migrated while IOs were happening. As this is not the common case, dht will now check if the fd is open on the cached subvol only if the call fails with EBADF. This will prevent a performance hit where a rebalance is not running. Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808 BUG: 1476665 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/17976 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Susant Palai <spalai@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* Revert "cluster/dht: Check for open fd only on EBADF"N Balachandran2017-08-042-116/+160
| | | | | | | | | | | | This reverts commit 91c9f4a19fde4894576b398252c77f730832a26a. This patch needs to be reworked. Change-Id: I4c24f647c2b1abc68fc4e9fe6eb810418e2033aa BUG: 1476665 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/17970 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* cluster/dht: Handle wrong rebalance status reportingSusant Palai2017-08-011-29/+32
| | | | | | | | | | | Change-Id: Id91ef35f890055cd42b9a94462f92297c77f1fff Bug: 1475282 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17868 Tested-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* cluster/dht: rebalance min-free-disk fixSusant Palai2017-08-011-6/+12
| | | | | | | | | | | | | | | | | | | | | | | | To calculate available space on a subvolume we used to do the following in __dht_check_free_space. post_availspace = (dst_statfs.f_bavail * dst_statfs.f_frsize) - stbuf->ia_size Now to subtracting the file size from available space is tricky here. Sometime available space will be lesser than the file size and since all the participating members in calculation are unsigned int, the result is a large number (integer overflow). Solution: We do not need to subtract the file size from the space available, since fallocate would have reserved file size space already. Change-Id: I4f724358c44b9911933742ff3ff8d55b3dfda1cb BUG: 1475282 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17876 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
generated by cgit (git 2.34.1) at 2024-06-17 12:31:25 +0000