summaryrefslogtreecommitdiffstats
path: root/xlators
Commit message (Collapse)AuthorAgeFilesLines
* md-cache: Do not invalidate cache post set/remove xattrPoornima G2018-07-118-68/+206
| | | | | | | | | | | | | | | Since setxattr and removexattr fops cbk do not carry poststat, the stat cache was being invalidated in setxatr/remoxattr cbk. Hence the further lookup wouldn't be served from cache. To prevent this invalidation, md-cache is modified to get the poststat in set/removexattr_cbk in dict. Co-authored with Xavi Hernandez. Change-Id: I6b946be2d20b807e2578825743c25ba5927a60b4 fixes: bz#1586018 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> Signed-off-by: Poornima G <pgurusid@redhat.com>
* server: Set ssl-allow option in options table and rename IDPrashanth Pai2018-07-111-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change explicitly adds 'ssl-allow' options to the server xlator's options table so that glusterd2 can see it as a settable option. This change also marks 'auth.allow' and 'auth.reject' options as a settable. Glusterd2 doesn't maintain a separate volume options table. Glusterd2 dynamically loads shared objects of xlators to read their option table and other information. Glusterd2 reads 'xlator_api_t' if available. If that's not available, it falls back to reading just the options table directly. In glusterd2, volume set operations are performed by users on keys of the format <xlator>.<option-name>. Glusterd2 uses xlator name set in 'xlator_api_t.identifier'. If that's not present it will use the shared object's file name as xlator name. Hence, it is important for 'xlator_api_t.identifier' to be set properly, and in this case, the proper value is "server". This name shall be used by users as prefix while setting volume options implemented in server xlator. The name will also be used in volfile. A user in glusterd2 can authorize a client over TLS as follows: $ glustercli volume set <volname> server.ssl-allow <client1-CN>[,<clientN-CN>] gd2 References: https://github.com/gluster/glusterd2/issues/971 https://github.com/gluster/glusterd2/issues/214 https://github.com/gluster/glusterd2/pull/967 Updates: bz#1193929 Change-Id: I59ef58acb8d51917e6365a83be03e79ae7c5ad17 Signed-off-by: Prashanth Pai <ppai@redhat.com>
* dht: Inconsistent permission for directories after brick stop/startMohit Agrawal2018-07-113-9/+77
| | | | | | | | | | | | | | | | Problem: Inconsistent access permissions on directories after bringing back the down sub-volumes, in case of directories dht_setattr first wind a call on MDS once call is finished on MDS then wind a call on NON-MDS.At the time of revalidating dht just compare the uid/gid with stbuf uid/gid and if anyone differs set a flag to heal the same. Solution: Add a condition to compare permission also in dht_revalidate_cbk to set a flag to call dht_dir_attr_heal. BUG: 1584517 Change-Id: I3e039607148005015b5d93364536158380d4c5aa fixes: bz#1584517 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* Fix compile warningsXavi Hernandez2018-07-1032-600/+1345
| | | | | | | | | | | This patch fixes compile warnings that appear with newer compilers. The solution applied is only to remove the warnings, but it doesn't always solve the problem in the best way. It assumes that the problem will never happen, as the previous code assumed. Change-Id: I6e8470d6c2e2dbd3bd7d324b5fd2f92ffdc3d6ec updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* cluster/afr: Prevent execution of code after call_count decrementingPranith Kumar K2018-07-101-7/+8
| | | | | | | | | | | | | | Problem: When call_count is decremented by one thread, another thread can go ahead with the operation leading to undefined behavior for the thread executing statements after decrementing call count. Fix: Do the operations necessary before decrementing call count. fixes bz#1598663 Change-Id: Icc90cd92ac16e5fbdfe534d9f0a61312943393fe Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* posix: Do not log ENXIO errors for seek fopXavi Hernandez2018-07-101-2/+3
| | | | | | | | | | | | | When lseek is used with SEEK_DATA and SEEK_HOLE, it's expected that the last operation fails with ENXIO when offset is beyond the end of file. In this case it doesn't make sense to report this as an error log message. This patch reports ENXIO failure messages for seek fops in debug level instead of error level. Change-Id: I62a4f61f99b0e4d7ea6a2cdcd40afe15072794ac fixes: bz#1598926 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* io-stats: Terminate dump thread when dump interval is set to 0ShyamsundarR2018-07-091-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | _ios_dump_thread is not terminated by the function _ios_destroy_dump_thread when the diagnostic interval is set to 0 (which means disable auto dumping). During reconfigure, if the value changes from 0 to another then the thread is started, but on reconfiguring this to 0 the thread is not being terminated. Further, if the value is changed from 0 to X to 0 to Y, where X and Y are 2 arbitrary duration numbers, the reconfigure code ends up starting one more thread (for each change from 0 to a valid interval). This patch fixes the same by terminating the thread when the value changes from non-zero to 0. NOTE: It would seem nicer to use conf->dump_thread and check its value for thread presence etc. but there is no documented invalid value for the same, and hence an invalid check is not feasible, thus introducing a new running bool to determine the same. Fixes: bz#1598548 Change-Id: I3e7d2ce8f033879542932ac730d57bfcaf14af73 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-07-098-31/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, and https://review.gluster.org/#/c/20364/ Fixes glupy.py python2isms, iteritems -> items, and some overlooked print() in georep/peer_mountbroker.in Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: Idda031c1ec975417c79323aea33e7b694e752b2a updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* glusterfs: Brick process is crash at the time of call server_first_lookupMohit Agrawal2018-07-091-3/+11
| | | | | | | | | | | | | | | | | | | | Problem: Brick process is getting crash while executing test case tests/bugs/core/bug-1432542-mpx-restart-crash.t Solution: At the time of initiating connection with brick process by client brick process call server_setvolume.If cleanup thread has set cleanup_starting flag after check flag by server_setvolume then a brick process can crash at the time of calling lookup on brick root.To avoid crash check cleanup_starting flag before just call server_first_lookup BUG: 1597627 Change-Id: I12542c124c76429184df34a04c1eae1a30052ca7 fixes: bz#1597627 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Note: To test the patch executing test case tests/bugs/core/bug-1432542-mpx-restart-crash.t in a loop around 100 times
* glusterd: Fix glusterd crashSanju Rakonde2018-07-061-9/+0
| | | | | | | | | | | | | | | | | | | | Problem: gluster get-state command is crashing glusterd process, when geo-replication session is configured. Cause: Crash is happening due to the double free of memory. In glusterd_print_gsync_status_by_vol we are calling dict_unref(), which will free all the keys and values in the dictionary. Before calling dict_unref(), glusterd_print_gsync_status_by_vol is calling glusterd_print_gsync_status(). glusterd_print_gsync_status is freeing up values in the dictionary and again when dict_unref() is called, it tries to free up the values which are already freed. Solution: Remove the code which will free the memory in glusterd_print_gsync_status function. Fixes: bz#1598345 Change-Id: Id3d8aae109f377b462bbbdb96a8e3c5f6b0be752 Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* protocol/client: handle the fdctx_destroy properly with different versionsAmar Tumballi2018-07-052-74/+133
| | | | | | | | while adding the new version of RPC, this part was not handled properly Updates: bz#1193929 Change-Id: If4cc4c2db075221b9ed731bacb7cc035f7007c5b Signed-off-by: Amar Tumballi <amarts@redhat.com>
* glusterd: show brick online after port registration even in brick-muxPranith Kumar K2018-07-053-10/+31
| | | | | | | | | | | | | | | | | Problem: With brick-mux even before brick attach is complete on the bricks glusterd marks them as online. This can lead to a race where scripts that check if the bricks are online to assume that the brick is online before it is completely online. Fix: Wait for the callback from the brick before marking the port as registered so that volume status will show the correct status of the brick. fixes bz#1597568 Change-Id: Icd3dc62506af0cf75195e96746695db823312051 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* cluster/afr: Make sure lk-owner is assigned at the time of lockPranith Kumar K2018-07-041-2/+1
| | | | | | | | | | | | | | Problem: In the new eager-lock implementation lk-owner is assigned after the 'local' is added to the eager-lock list, so there exists a possibility of lock being sent even before lk-owner is assigned. Fix: Make sure to assign lk-owner before adding local to eager-lock list fixes bz#1597805 Change-Id: I26d1b7bcf3e8b22531f1dc0b952cae2d92889ef2 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* glusterd: Introduce daemon-log-level cluster wide optionAtin Mukherjee2018-07-036-1/+69
| | | | | | | | | | | | This option, applicable to the node level daemons can be very helpful in controlling the log level of these services. Please note any daemon which is started prior to setting the specific value of this option (if not INFO) will need to go through a restart to have this change into effect. Change-Id: I7f6d2620bab2b094c737f5cc816bc093e9c9c4c9 fixes: bz#1597473 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* glusterd: start the services after all the bricks are upAtin Mukherjee2018-07-031-9/+5
| | | | | | | | | glusterd_svcs_manager () should be called post starting all the volumes at one go. Change-Id: I838cc50c29f3930a483aa9671958cdc186904030 Fixes: bz#1597247 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* afr,ec: Print if the subvolume is up in statedumpPranith Kumar K2018-07-032-0/+2
| | | | | | fixes bz#1597156 Change-Id: I323eb9190e40b12df216698dcdba74a6d336beeb Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* storage/posix: Add warning logs on failureN Balachandran2018-07-021-2/+12
| | | | | | | | | | | posix_readdirp_fill will fail to update the iatt information if posix_handle_path fails. There is currently no log message to indicate this making debugging difficult. Change-Id: I6bce360ea7d1696501637433f80e02794fe1368f updates: bz#1564071 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* cluster/dht: Do not try to use up the readdirp bufferN Balachandran2018-06-293-53/+66
| | | | | | | | | | | | | | | | | | | | | | | | | DHT attempts to use up the entire buffer in readdirp before unwinding in an attempt to reduce the number of calls. However, this has 2 disadvantages: 1. This can cause a stack overflow when parallel readdir is enabled. If the buffer only has a little space,rda can send back only one or two entries. If those entries are stripped out by dht_readdirp_cbk (linkto files for example) it will once again wind down to rda in an attempt to fill the buffer before unwinding to FUSE. This process can continue for several iterations, causing the stack to grow and eventually overflow, causing the process to crash. 2. If parallel readdir is disabled, dht could send readdirp calls with small buffers to the bricks, thus increasing the number of network calls. We are therefore reverting to the existing behaviour. Please note, this only mitigates the stack overflow, it does not prevent it from happening. This is still possible if a subvol has thousands of linkto files for instance. Change-Id: I291bc181c5249762d0c4fe27fa4fc2631166adf5 fixes: bz#1593548 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* xlators/meta: Fix resource_leakVarsha Rao2018-06-291-7/+12
| | | | | | | | Free priv to fix the resource_leak coverity issue. Change-Id: Id116bd2f72332d2f665c25c9ae27241286ac570f updates: bz#789278 Signed-off-by: Varsha Rao <varao@redhat.com>
* xlators/features/barrier: Fix RESOURCE_LEAKVarsha Rao2018-06-281-0/+3
| | | | | | | | Free priv to fix the resource_leak coverity issue. Change-Id: I1f93f54d7d59eddb0b9523ac2b67ef0b279122bd updates: bz#789278 Signed-off-by: Varsha Rao <varao@redhat.com>
* performance/md-cache: Fix issue on lock being used before init.Zhang Huan2018-06-271-1/+2
| | | | | | | | | lock is used in mdc_xattr_list_populate(), but got init after call. Fix this issue by moving initing lock ahead. Change-Id: I94b08303a8ba74b1e9388f700587a00b7ae3fd78 fixes: bz#1595174 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* storage/posix: Fix posix_symlinks_match()Pranith Kumar K2018-06-261-3/+13
| | | | | | | | | | 1) snprintf into linkname_expected should happen with PATH_MAX 2) comparison should happen with linkname_actual with complete string linkname_expected fixes bz#1595190 Change-Id: Ic3b3c362dc6c69c046b9a13e031989be47ecff14 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* cloudsync: avoid null pointer deref in error pathsThomas Hindoe Paaboel Andersen2018-06-251-5/+2
| | | | | | | | | | In the case of OOM we would end up dereferencing priv in the out label of aws_init(). A similar deref would happen if a null priv is passed to aws_reconfigure(). Change-Id: I57c80aad2b52ecb1eda86a0077ed6b46816e5dea fixes: bz#1592141 Signed-off-by: Thomas Hindoe Paaboel Andersen <phomes@gmail.com>
* nfs: add missing rpcsvc_program_unregister_rpcbind6Xie Changlong2018-06-251-0/+3
| | | | | | | | | Otherwise, event if nfs.register-with-portmap is disabled, we could also use ipv6 to mount nfs through portmap. Change-Id: I7546e03d942dfaf2252d57e17be9c1cca986a715 fixes: bz#1593651 Signed-off-by: Xie Changlong <xiechanglong@cmss.chinamobile.com>
* cluster/dht: Fix rename journal in changelogKotresh HR2018-06-241-0/+11
| | | | | | | | | | | | | | | With patch [1], renames are journalled only on cached subvolume. The dht sends the special key on the cached subvolume so that the changelog journals the rename. With single distribute sub-volume, the key is not being set. This patch fixes the same. [1] https://review.gluster.org/10410 fixes: bz#1583018 Change-Id: Ic2e35b40535916fa506a714f257ba325e22d0961 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-211-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, and https://review.gluster.org/#/c/20320/ This patch fixes more selected comma white space (ws_comma) as suggested by the 2to3 utility. (Earlier attempts to fix all ws_comma in one patch did not pass centos regression, hence multiple patches to identify the change that causes the failure.) Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I6e5a2408fa1fc81e00e66d6e4a7f9f6fa1d1ed15 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-212-15/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, and https://review.gluster.org/#/c/20307/ This patch fixes more selected comma white space (ws_comma) as suggested by the 2to3 utility. Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs, so popular practices like #!/usr/bin/env python and or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I60932030813484803f73733a9b2b7b23c7a843fd updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* fuse: avoid using the which commandJohn Mulligan2018-06-211-1/+1
| | | | | | | | | | | In mount.glusterfs avoid using the which tool as it may not exist on minimal system installs. Use the "command -v" builtin as it is expected to be more portable. Remove a extra semicolon while we're at it. Change-Id: Ib682ed4955d5bad1beb94b65d10f4c44e9490767 fixes: bz#1593351 Signed-off-by: John Mulligan <jmulligan@redhat.com>
* client: remove the "connecting" state - it's not usedMichael Adam2018-06-213-6/+0
| | | | | | | | | The "connecting" state is not used anywhere really. It's only being set and printed. So remove it. Change-Id: I11fc8b0bdcda5a812d065543aa447d39957d3b38 fixes: bz#1583583 Signed-off-by: Michael Adam <obnox@samba.org>
* cluster/dht: refactor dht_lookupN Balachandran2018-06-211-211/+321
| | | | | | | | | | The dht lookup code is getting difficult to maintain due to its size. Refactoring the code will make it easier to modify it in future. Change-Id: Ic7cb5bf4f018504dfaa7f0d48cf42ab0aa34abdd updates: bz#1590385 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* glusterfs: access trusted peer group via remote-host commandMohit Agrawal2018-06-201-5/+0
| | | | | | | | | | | | | Problem: In SSL environment the user is able to access volume via remote-host command without adding node in a trusted pool Solution: Change the list of rpc program in glusterd.c at the time of initialization while SSL is enabled BUG: 1593232 Change-Id: I987e433b639e68ad17b77b6452df1e22dbe0f199 fixes: bz#1593232 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* features/shard: Perform shards deletion in the backgroundKrutika Dhananjay2018-06-204-159/+690
| | | | | | | | | | | | | | | A synctask is created that would scan the indices from .shard/.remove_me, to delete the shards associated with the gfid corresponding to the index bname and the rate of deletion is controlled by the option features.shard-deletion-rate whose default value is 100. The task is launched on two accounts: 1. when shard receives its first-ever lookup on the volume 2. when a rename or unlink deleted an inode Change-Id: Ia83117230c9dd7d0d9cae05235644f8475e97bc3 updates: bz#1568521 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* ctime: Fix self heal of symlink in EC volumeKotresh HR2018-06-205-6/+2
| | | | | | | | | | | | | | | | | | | | Since IEEE Std 1003.1-2001 does not require any association of file times with symbolic links, there is no requirement that file times be updated by readlink() states [1]. stat on symlink file was generating a readlink fop on one of the subvolumes of ec set which in turn updates atime on that subvolume. This causes mdata xattr to be different across ec set and hence self heal fails. So based on [1], atime is no longer updated by readlink fop. [1] http://pubs.opengroup.org/onlinepubs/009695399/functions/readlink.html fixes: bz#1592509 Change-Id: I08bd3ca3bdb222bd18160b1aa58fc2f7630c8083 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* posix/ctime: Fix differential ctime duing entry operationsKotresh HR2018-06-201-51/+60
| | | | | | | | | | | | | | | | | We should not be relying on backend file's time attributes to load the initial ctime time attribute structure. This is incorrect as each replica set would have witnessed the file creation at different times. For new file creation, ctime, atime and mtime should be same, hence initiate the ctime structure with the time from the frame. But for the files which were created before ctime feature is enabled, this is not accurate but still fine as the times would get eventually accurate. fixes: bz#1592275 Change-Id: I206a469c83ee7b26da2fe096ae7bf8ff5986ad67 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* cluster/dht: Minor code cleanupN Balachandran2018-06-201-15/+13
| | | | | | | | Removed extra variable. Change-Id: If43c47f6630454aeadab357a36d061ec0b53cdb5 updates: bz#1590385 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-195-79/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, and https://review.gluster.org/#/c/20227/ This patch fixes selected comma white space (ws_comma) as suggested by the 2to3 utility. Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs, so popular practices like #!/usr/bin/env python and or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I691d7f86279572642902650ed1c24840e3c4b781 Updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* afr: heal gfids when file is not present on all bricksRavishankar N2018-06-195-12/+51
| | | | | | | | | | | commit 20fa80057eb430fd72b4fa31b9b65598b8ec1265 introduced a regression wherein if a file is present in only 1 brick of replica *and* doesn't have a gfid associated with it, it doesn't get healed upon the next lookup from the client. Fix it. Change-Id: I7d1111dcb45b1b8b8340a7d02558f05df70aa599 fixes: bz#1591193 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* afr: don't update readables if inode refresh failed on all childrenRavishankar N2018-06-184-32/+50
| | | | | | | | | | | | | | | | | | | | | | | | Problem: If inode refresh failed on all children of afr due to ENOENT (say file migrated by dht), it resets the readables to zero. Any inflight txn which then later comes on the inode fails with EIO because no readable children present for the inode. Fix: Don't update readables when inode refresh fails on *all* children of afr. In that way any inflight txns will either proceed with its own inode refresh if needed and fail it with the right errno or use the old value of readables and continue with the txn. Also, add quorum checks to the beginning of afr_transaction(). Otherwise, we seem to be winding the lock and checking for quorum only in pre-op pahse. Note: This should ideally fix BZ 1329505 since the stop gap fix for it is has been reverted at https://review.gluster.org/#/c/20028. Change-Id: Ia638c092d8d12dc27afb3cdad133394845061319 updates: bz#1584483 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* performance/quick-read: provide an invalidation based on ctimeRaghavendra G2018-06-183-1/+55
| | | | | | | | | | | | | | | | | | | | Quick-read by default uses mtime to identify changes to file data. However there are applications like rsync which explicitly set mtime making it unreliable for the purpose of identifying change in file content. Since ctime also changes when content of a file changes and it cannot be set explicitly, it becomes suitable for identifying staleness of cached data. This option makes quick-read to prefer ctime over mtime to validate its cache. However, using ctime can result in false positives as ctime changes with just attribute changes like permission without changes to file data. So, use this option only when mtime is not reliable. credits to Kotresh Hiremath Ravishankar <khiremat@redhat.com> for suggestion on using ctime instead of mtime. Change-Id: Ib3ae39a3252b2876c8ffe81f471d02a87190e9b9 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Updates: bz#1591621
* protocol/client: Remove code duplicationKrutika Dhananjay2018-06-153-119/+22
| | | | | | | | | | | | | | | client_submit_vec_request() which is used by WRITEV, and PUT and client_submit_request() used by the rest of the fops have almost similar code. However, there have been some more checks - such as whether setvolume was successful or not, and one more that is send-gid-specific - that have been missed out in the vectored version of the function. This patch fixes this code duplication. Change-Id: I363a28eeead6219cb1009dc831538153e8bd7d40 fixes: bz#1591580 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* glusterd: removing the unnecessary glusterd messageSanju Rakonde2018-06-141-2/+1
| | | | | | Fixes: bz#1589253 Change-Id: I5510250a3d094e19e471b3ee47bf13ea9ee8aff5 Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* features/snapview-server: properly go through the list of snapshotsRaghavendra Bhat2018-06-141-3/+9
| | | | | | | | | | The comparison code to check whether a glfs instance is valid (i.e. whether it corresponds to one in the list of current snapshots) was not correct and was not comparing all the snapshots Change-Id: I87c58edb47bd9ebbb91d805e45df2c4baf2c8118 fixes: bz#1589842 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* storage/posix: Handle ENOSPC correctly in zero_fillPranith Kumar K2018-06-141-1/+22
| | | | | | Change-Id: Icc521d86cc510f88b67d334b346095713899087a fixes: bz#1590710 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* glusterd: Fix for shd not coming upSanju Rakonde2018-06-132-1/+6
| | | | | | | | | | | | | | Problem: After creating and starting n(n is large) distribute-replicated volumes using a script, if we create and start (n+1)th distribute-replicate volume manually self heal daemon is down. Solution: In glusterd_proc_stop after giving SIGTERM signal if the process is still running, we are giving a SIGKILL. As SIGKILL will not perform any cleanup process, we need to remove the pidfile. Fixes: bz#1589253 Change-Id: I7c114334eec74c8d0f21b3e45cf7db6b8ef28af1 Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-132-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, and https://review.gluster.org/#/c/20207/ This patch changes uses of has_key() as suggested by the 2to3 utility. Note: Fedora packaging guidelines require explicit shebangs, so popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, zip, map, and raise have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Updates: #411 Change-Id: I79bda20f1583a0a1bb0320667498f4c137de93b3 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* storage/posix: Fix excessive logging in WRITE fop pathKrutika Dhananjay2018-06-131-0/+3
| | | | | | | | | | | | I was running some write-intensive tests on my volume, and in a matter of 2 hrs, the 50GB space in my root partition was exhausted. On inspecting further, figured that excessive logging in bricks was the cause - specifically in posix write when posix_check_internal_writes() does dict_get() without a NULL-check on xdata. Change-Id: I89de57a3a90ca5c375e5b9477801a9e5ff018bbf fixes: bz#1590655 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* features/shard: Introducing ".shard/.remove_me" for atomic shard deletion ↵Krutika Dhananjay2018-06-134-421/+1040
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | (part 1) PROBLEM: Shards are deleted synchronously when a sharded file is unlinked or when a sharded file participating as the dst in a rename() is going to be replaced. The problem with this approach is it makes the operation really slow, sometimes causing the application to time out, especially with large files. SOLUTION: To make this operation atomic, we introduce a ".remove_me" directory. Now renames and unlinks will simply involve two steps: 1. creating an empty file under .remove_me named after the gfid of the file participating in unlink/rename 2. carrying out the actual rename/unlink A synctask is created (more on that in part 2) to scan this directory after every unlink/rename operation (or upon a volume mount) and clean up all shards associated with it. All of this happens in the background. The task takes care to delete the shards associated with the gfid in .remove_me only if this gfid doesn't exist in backend, ensuring that the file was successfully renamed/unlinked and its shards can be discarded now safely. Change-Id: Ia1d238b721a3e99f951a73abbe199e4245f51a3a updates: bz#1568521 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* cluster/dht: Refactor rebalance codeN Balachandran2018-06-131-309/+253
| | | | | | | | | | Created init and cleanup functions for certain functionality in order to improve readability. Removed unused code. Change-Id: Ia6a2f4ab64923b6ea8e10487227fb5621eec1488 updates: bz#1586363 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* glusterd: Coverity FixesSanju Rakonde2018-06-112-4/+7
| | | | | | | Fixes: #789278 Change-Id: I633704fab49992cac6ee9e05bc368f7da360d09e Signed-off-by: Sanju Rakonde <srakonde@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* cluster/dht: Leverage MDS subvol for dht_removexattr alsoMohit Agrawal2018-06-111-60/+190
| | | | | | | | | | | | | Problem: In a distributed volume situation can be arise when custom extended attributed are not removed from all bricks after stop/start or added newly brick. Solution: To resolve the same use MDS subvol for remove xattr also BUG: 1575587 Change-Id: I7701e0d3833e3064274cb269f26061bff9b71f50 fixes: bz#1575587 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-20 17:05:05 +0000