summaryrefslogtreecommitdiffstats
path: root/tests/features
Commit message (Collapse)AuthorAgeFilesLines
* tests: subdir-mount.t is failing for brick_mux regrssionMohit Agrawal2019-06-171-3/+8
| | | | | | | | | To avoid the failure wait to run hook script S13create-subdir-mounts.sh after executed add-brick command by test case. Change-Id: I063b6d0f86a550ed0a0527255e4dfbe8f0a8c02e fixes: bz#1720993 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* tests: keep glfsxmp in tests directoryAmar Tumballi2019-06-111-1/+1
| | | | | | | | | | | | | this is critical so all the tests will be contained in the same directory, and one can just 'cp -a tests/ <any-location>/' and run glusterfs tests. only 'glfsxmp.c' was an exception as it was just copying the file from api example directory. Now moved it to tests. updates: bz#1193929 Change-Id: I00359d64be580bffc5b3c3a090968d86c2c6952a Signed-off-by: Amar Tumballi <amarts@redhat.com>
* lcov: run more fops on translatorsAmar Tumballi2019-06-041-0/+12
| | | | | | | | | | | | | | Translators covered: * playground/template * debug/delay-gen * debug/error-gen * features/namespace * features/quiesce * meta updates: bz#1693692 Change-Id: Ic8fde8efcb309ea492d8e819241f786f7ff467a1 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* socket/ssl: fix crl handlingMilind Changire2019-03-191-2/+11
| | | | | | | | | | | | | | | | Problem: Just setting the path to the CRL directory in socket_init() wasn't working. Solution: Need to use special API to retrieve and set X509_VERIFY_PARAM and set the CRL checking flags explicitly. Also, setting the CRL checking flags is a big pain, since the connection is declared as failed if any CRL isn't found in the designated file or directory. A comment has been added to the code appropriately. Change-Id: I8a8ed2ddaf4b5eb974387d2f7b1a85c1ca39fe79 fixes: bz#1687326 Signed-off-by: Milind Changire <mchangir@redhat.com>
* fuse lock interrupt: fix flock_interrupt.tCsaba Henk2019-03-051-5/+5
| | | | | | updates: bz#1193929 Change-Id: I347de62755100cd69e3cf341434767ae23fd1ba4 Signed-off-by: Csaba Henk <csaba@redhat.com>
* fuse: SETLKW interruptCsaba Henk2018-12-141-0/+33
| | | | | | | | | Use the (f)getxattr based clearlocks interface to interrupt a pending lock request. updates: #465 Change-Id: I4e91a4d8791fc688fed400a02de4c53487e61be2 Signed-off-by: Csaba Henk <csaba@redhat.com>
* fuse: add --lru-limit optionAmar Tumballi2018-12-141-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The inode LRU mechanism is moot in fuse xlator (ie. there is no limit for the LRU list), as fuse inodes are referenced from kernel context, and thus they can only be dropped on request of the kernel. This might results in a high number of passive inodes which are useless for the glusterfs client, causing a significant memory overhead. This change tries to remedy this by extending the LRU semantics and allowing to set a finite limit on the fuse inode LRU. A brief history of problem: When gluster's inode table was designed, fuse didn't have any 'invalidate' method, which means, userspace application could never ask kernel to send a 'forget()' fop, instead had to wait for kernel to send it based on kernel's parameters. Inode table remembers the number of times kernel has cached the inode based on the 'nlookup' parameter. And 'nlookup' field is not used by no other entry points (like server-protocol, gfapi etc). Hence the inode_table of fuse module always has to have lru-limit as '0', which means no limit. GlusterFS always had to keep all inodes in memory as kernel would have had a reference to it. Again, the reason for this is, kernel's glusterfs inode reference was pointer of 'inode_t' structure in glusterfs. As it is a pointer, we could never free it (to prevent segfault, or memory corruption). Solution: In the inode table, handle the prune case of inodes with 'nlookup' differently, and call a 'invalidator' method, which in this case is fuse_invalidate(), and it sends the request to kernel for getting the forget request. When the kernel sends the forget, it means, it has dropped all the reference to the inode, and it will send the forget with the 'nlookup' parameter too. We just need to make sure to reduce the 'nlookup' value we have when we get forget. That automatically cause the relevant prune to happen. Credits: Csaba Henk, Xavier Hernandez, Raghavendra Gowdappa, Nithya B fixes: bz#1560969 Change-Id: Ifee0737b23b12b1426c224ec5b8f591f487d83a2 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* gfapi: Offload callback notifications to synctaskSoumya Koduri2018-11-262-0/+374
| | | | | | | | | | | | | | | | | | | | | | Upcall notifications are received from server via epoll and same thread is used to forward these notifications to the application. This may lead to deadlock and hang in the following scenario. Consider if as part of handling these callbacks, application has to do some operations which involve sending I/Os to gfapi stack which inturn have to wait for epoll threads to receive repsonse. Thus this may lead to deadlock if all the epoll threads are waiting to complete these callback notifications. To address it, instead of using epoll thread itself, make use of synctask to send those notificaitons to the application. Change-Id: If614e0d09246e4279b9d1f40d883a32a39c8fd90 updates: bz#1648768 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* lease: Treat unlk request as noop if lease not foundSoumya Koduri2018-11-171-2/+2
| | | | | | | | | | | | | | | | When the glusterfs server recalls the lease, it expects client to flush data and unlock the lease. If not it sets a timer (starting from the time it sends RECALL request) and post timeout, it revokes it. Here we could have a race where in client did send UNLK lease request but because of network delay it may have reached after server revokes it. To handle such situations, treat such requests as noop and return sucesss. Change-Id: I166402d10273f4f115ff04030ecbc14676a01663 updates: bz#1648768 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* glupy: remove from the buildAmar Tumballi2018-11-081-31/+0
| | | | | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removing 'glupy' translator from the build. [1] https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html This patch aims at clearing the translator from build and tests. A followup is needed to remove the code from repository. Updates: bz#1642810 Change-Id: I41d0c1956330c3bbca62c540ccf9ab01bbf3a092 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* tests/interrupt.t: remove 'stripe' volume typeAmar Tumballi2018-11-061-1/+1
| | | | | | | | | Merged the patch which introduced this testcase after the 'remove stripe' patch got merged, and hence the confusion. Updates: bz#1193929 Change-Id: Ia08552debb111292caf14e51ea6a27334fe5c788 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* fuse: diagnostic FLUSH interruptCsaba Henk2018-11-062-0/+94
| | | | | | | | | | | | | | | | | | | We add dummy interrupt handling for the FLUSH fuse message. It can be enabled by the "--fuse-flush-handle-interrupt" hidden command line option, or "-ofuse-flush-handle-interrupt=yes" mount option. It serves no other than diagnostic & demonstational purposes -- to exercise the interrupt handling framework a bit and to give an usage example. Documentation is also provided that showcases interrupt handling via FLUSH. Change-Id: I522f1e798501d06b74ac3592a5f73c1ab0590c60 updates: #465 Signed-off-by: Csaba Henk <csaba@redhat.com>
* python3: assume python3 unless building _packages_ on sys without py3Kaleb S. KEITHLEY2018-09-271-1/+0
| | | | | | | | | | | | | | | The jenkins release-new job runs on a CentOS 7 box, which does not have python3. As a result it runs (autogen.sh and) configure before producing the dist tar file, converting all the python3 shebangs to python2 shebangs in the dist tar file. Then when that tar file is "carried" to, e.g. Fedora koji build system to build packages, the shebangs are incorrect, despite having originally been correct in the git repo. Change-Id: I5154baba3f6d29d3c4823bafc2b57abecbf90e5b updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* dht: utilize the framework to pass-through xlator tasksAmar Tumballi2018-09-191-1/+1
| | | | | | | | | | | | | | | | | | | | | Also fixes the issue caused due to not converting back the fn function to after getting its address. We wanted the value of the field, not the address of the pt_fop field. With this patch, DHT will always be started in pass-through mode if the number of subvols is just 1. Fixes some tests to make sure DHT is in full config (ie, subvols > 1). - increased timeout of brick-mux test as it was bordering on 300 seconds. - Also change the volume type to supported 'replica 3' from 'replica 2'. - also no DHT tests should assume presence of DHT when there is just 1 brick in volume Credits: Nithya B <nbalacha@redhat.com> fixes: #405 Change-Id: I8e55239ce58d6ac6ae1901e2e384be1ecbd33d6e Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-122-596/+623
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* core: python3Kaleb S. KEITHLEY2018-09-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-221-2/+2
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-06-211-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, and https://review.gluster.org/#/c/20307/ This patch fixes more selected comma white space (ws_comma) as suggested by the 2to3 utility. Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs, so popular practices like #!/usr/bin/env python and or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: I60932030813484803f73733a9b2b7b23c7a843fd updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/ and https://review.gluster.org/#/c/19871/ Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, idioms, numliterals, set_literal, types, urllib, zip Note: these 2to3 fixes report no changes are necessary: exec, execfile, exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Any 2to3 fixes not in the above two lists have more extensive changes which will follow in separate patches. most unicode changes suggested by 2to3 will need to be applied at the same time as changing the shebangs from python2 to python3. Prashanth notes that unicode strings in py2 need 'u' prefix; unicode strings in py3 3.0, 3.1, and 3.2 a 'u' prefix will throw an error, but in py3 3.3+ it is legal (or just ignored). All Linux dists we care about have 3.3 or later so we can leave 'u' prefixes on unicode strings. Change-Id: I49bba2f328b0ee24b9a8115a7183be979981563e updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* tests: Add lease test casePoornima G2018-05-052-2/+704
| | | | | | | Updates: #350 Change-Id: Iee78ab4baf48c481de1e13ff2b0393bc106b7d0e Signed-off-by: Poornima G <pgurusid@redhat.com> Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-05-021-2/+3
| | | | | | | | | | see https://review.gluster.org/#/c/19788/ use print fn from __future__ Change-Id: If5075d8d9ca9641058fbc71df8a52aa35804cda4 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* cluster/dht: enable lookup-optimize by defaultN Balachandran2018-04-041-1/+1
| | | | | | | | | | | | | | Lookup-optimize has been shown to improve create performance. The code has been in the project for several years and is considered stable. Enabling this by default in order to test this in the upstream regression runs. Change-Id: Iab792979ee34f0af4713931e0b5b399c23f65313 updates: bz#1557435 BUG: 1557435 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* python: Remove all uses of find_library. Fixes #1450593Niklas Hambüchen2018-03-241-8/+2
| | | | | | | | `find_library()` doesn't consider LD_LIBRARY_PATH on Python < 3.6. Change-Id: Iee26085cb5d14061001f19f032c2664d69a378a8 BUG: 1450593 Signed-off-by: Niklas Hambüchen <mail@nh2.me>
* hooks: add a script to stat the subdirs in add-brickAmar Tumballi2018-03-061-12/+4
| | | | | | | | | | | | | | | | | | | | | | The subdirectories are expected to be present for a subdir mount to be successful. If not, the client_handshake() itself fails to succeed. When a volume is about to get mounted first time, this is easier to handle, as if the directory is not present in one brick, then its mostly not present in any other brick. In case of add-brick, the directory is not present in new brick, and there is no chance of healing it from the subdirectory mount, as in those clients, the subdir itself will be 'root' ('/') of the filesystem. Hence we need a volume mount to heal the directory before connections can succeed. This patch does take care of that by healing the directories which are expected to be mounted as subdirectories from the volume level mount point. Change-Id: I2c2ac7b7567fe209aaa720006d09b68584d0dd14 BUG: 1549915 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* features/worm: new config option to manage deletion of Worm files.Vishal Pandey2017-11-201-0/+8
| | | | | | | | | | | | | | | | | | | | Add a new configuration option worm-files-deletable to file-level Worm in order to control behaviour of Worm files upon deletion. Steps to Test: 1. Add all the configuration options to a volume to activate file-level-worm 2. Option features.worm-files-deletable is set to 1 by default. 3. Create a new file and wait for the retention time to expire. 4. After retention time expires, do an truncate, rename, unlink, link or write to send the file in Worm state. 5. After that do `rm -f filename`. 6. The file is successfully removed. 7. Repeat from step 2 by setting features.worm-files-deletable 0. This time deletion should not be successful. Change-Id: Ibc89861ee296e065330b93a9f9606be5da40af31 BUG: 1508898 Signed-off-by: Vishal Pandey <vishpandey2014@gmail.com>
* write once read many: file appendable in worm stateVishal Pandey2017-10-301-0/+7
| | | | | | | | | | | | | | | | | | | | Issue: A new file is appendable even when file level worm is 1. Fix: Do a state transition in writev function in worm.c file. Steps To Test: 1- Activate file level worm. 2- Create a new file. 3- Leave file dormant for auto commit period. 4- Try and append some content to the file. 5- check the file if new content has been appended or not. 6- check if file has been transitioned to Worm Retention state. Change-Id: I52d50ad888cb0c39ad54be9352ccb07d48b8d71a BUG: 1505807 Signed-off-by: Vishal Pandey <vishpandey2014@gmail.com>
* protocol/client: handle the subdir handshake properly for add-brickAmar Tumballi2017-10-291-5/+26
| | | | | | | | | There should be different way we handle handshake in case of subdir mount for the first time, and in case of subsequent graph changes. Change-Id: I2a7ba836433bb0a0f4a861809e2bb0d7fbc4da54 BUG: 1505323 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Disable failing NetBSD testsNigel Babu2017-10-131-0/+1
| | | | | BUG: 1501390 Change-Id: I9a04c094783ec33e617baeae3d0e0cbedb1d6c3b
* protocol-auth: use the proper validation methodAmar Tumballi2017-10-121-0/+4
| | | | | | | | | | | Currently, server protocol's init and glusterd's option validation methods are different, causing an issue. They should be same for having consistent behavior Updates #175 Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* features/worm: Adding check to newloc when doing renameluneo72017-09-181-0/+9
| | | | | | | | | | | | | | | | | | Problem: Since rename didn't check if newloc exists and it's retention state it was possible to rename a new file that wasn't in retention over a existing file that was in read-only state. Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229 BUG: 1484490 Signed-off-by: luneo7 <luneo7@gmail.com> Reviewed-on: https://review.gluster.org/18104 Tested-by: jiffin tony Thottan <jthottan@redhat.com> Tested-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Karthik U S <ksubrahm@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* debug/delay-gen: Implement delay-generation featurePranith Kumar K2017-08-311-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: I was working on a customer issue where the disks were responding some times after seconds. It was becoming very difficult to recreate the issues in our labs, so had to come up with this feature. Requirements: We need an xlator which can delay x% of ops for y micro seconds. We should be able to enable delays for specific fops. This feature is modeled after error-gen. Most of the logic is borrowed from that xlator. This is a minimum implementation of the feature which satisfied the requirements I had. May be in future with more requirements and understanding of the problem further we can improve upon this implementation. Here are the commands and what they do: Enable delay-gen: (This is similar to how err-gen is enabled on the brick side) - gluster volume set <volname> delay-gen posix Set the percentage of fops that need to be delayed - gluster volume set <volname> delay-gen.delay-percentage 50 Default is 10% Set the delay in micro seconds - gluster volume set <volname> delay-gen.delay-duration 500000 Default is 100000 Set comma separated fops to be delayed - gluster v set r2 delay-gen.enable read,write Default is all fops. Fixes #257 Change-Id: Ib547bd39cc024c9cdb63754d21e3aa62fc9d6473 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17591 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* cluster/dht: Reorder dir operations in gf_defrag_fix_layoutN Balachandran2017-08-201-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, rebalance performed a fix-layout on a directory before healing its subdirectories. If there were a lot of subdirs, it could take a while before all subdirs were created on the newly added bricks. As dht_readdirp only lists dirs from their hashed subvol, those dirs which hashed to the newly added bricks but were not yet created on them were not listed. Now, the child dirs are listed and processed before the layout of the parent is fixed. This introduces a change in behaviour where files in subdirs are migrated before those in parent directories. Credit: Shyam <srangana@redhat.com> Github issue: #239 Change-Id: I8ae7f24a510754cd8d1b31e5d608bcf1928599e2 BUG: 1248393 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18045 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* glusterfsd: allow subdir mountAmar Tumballi2017-08-041-0/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changes: 1. Take subdir mount option in client (mount.gluster / glusterfsd) 2. Pass the subdir mount to server-handshake (from client-handshake) 3. Handle subdir-mount dir's lookup in server-first-lookup and handle all fops resolution accordingly with proper gfid of subdir 4. Change the auth/addr module to handle the multiple subdir entries in option, and valid parsing. How to use the feature: `# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point` Or `# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point` Option can be set like: `# gluster volume set <volname> auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"` Updates #175 Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/17141 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* tests: disable IPC test-caseNiels de Vos2017-07-181-0/+3
| | | | | | | | | | | | | | | | | | | | | The IPC test-case is not correct, and can cause segfaults or hangs. The ipctest.py script calls glfs_ipc() with the `glfs_t` structure, but should do so with a `glfs_fd_t`. In addition, the test-case is written in a way that we do not suggest to use libgfapi. Python scripts are encouraged to use the bindings from the libgfapi-python project. It would be better to rewrite the test in C so that there is type-checking while compiling and no additional issues with portability (see `LD_PRELOAD` note in the `.t` file). Change-Id: Icb52b5b1585fbee98f2c694547c31df0aa2ba70b Updates: #269 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17786 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
* tests/lock_revocation: mark as badRaghavendra G2017-05-101-0/+1
| | | | | | | | | | | | | The test is failing in master. see gluster-devel for more details. Change-Id: I7a589ad2c54bd55d62f4e66fdf8037c19fc123ea BUG: 1448364 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: https://review.gluster.org/17234 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* worm: add check for internal processes in ftruncate()Amar Tumballi2017-04-181-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The patch fixes the recently seen issues with worm_sh.t test. RCA: - $ git log --oneline xlators/features/read-only/src/worm.c 1b01bdc worm: allow Self-heal-Daemon to perform some operations c5a4a77 features/worm: Adding implementation for ftruncate - These two patches were merged in reverse order of their submission, and hence the check added for internal processes got missed in new fop 'ftruncate()'. The worm_sh.t passed the tests as while that patch got submitted there was no ftruncate() in worm xlator. Change-Id: I81a8a45fa2679917a2c859c4f5224a2c3edbc784 BUG: 1423413 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/17048 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Zhou Zhengping <johnzzpcrystal@gmail.com> Reviewed-by: David Spisla <david.spisla@iternity.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/dht: Make rebalance honor min-free-diskSusant Palai2017-04-131-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test: Manual created files of size 1K on 2 brick(of size 1GB) setup . added a brick of size 16GB. set min-free-disk to 12GB(so that first two bricks won't receive any files). removed one of the 1st brick of size 1GB. Logs from test: [2017-04-12 08:52:08.196484] W [MSGID: 0] [dht-rebalance.c:895:__dht_check_free_space] 0-test1-dht: Write will cross min-free-disk for file - /tile32 on subvol - test1-client-1. Looking for new subvol. [2017-04-12 08:52:08.196904] I [MSGID: 0] [dht-rebalance.c:925:__dht_check_free_space] 0-test1-dht: new target found - test1-client-2 for file - /tile32 - Post migration we have two files. The new destination (/brick/1) has the data file [root@vm1 ~]# ll /brick/1/tile32 -rw-r--r--. 2 root root 0 Apr 12 14:22 /brick/1/tile32 - On the old target the linkto file is there with linkto xattr pointing to /brick/1 [root@vm1 ~]# ll /tmp/2/tile32 ---------T. 2 root root 1000 Apr 12 14:22 /tmp/2/tile32 [root@vm1 ~]# getfattr -m . -de text /tmp/2/tile32 getfattr: Removing leading '/' from absolute path names security.selinux="unconfined_u:object_r:user_tmp_t:s0" trusted.gfid="����:Aс�#�/'b2" trusted.glusterfs.dht.linkto="test1-client-2" Marking ./tests/features/worm_sh.t as bad test. Reason being, this patch failed on master branch as well and it has nothing to do with rebalance/remove-brick. BUG: 1441508 Change-Id: I90bae251cda3d957a49cdceda90cd08311a392fb Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17034 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* worm: allow Self-heal-Daemon to perform some operationsDavid Spisla2017-04-121-0/+75
| | | | | | | | | | | | | | | | | | The Self-Heal-Daemon should be allowed to trigger unlink, link, trauncate, rename and write operation. The value of frame->root->pid can be used to detect internal (by SHD) operations. Change-Id: I7526148100bef1e2837d69df5c119dc97d91fffd BUG: 1423413 Signed-off-by: David Spisla <david.spisla@iternity.com> Reviewed-on: https://review.gluster.org/16661 Tested-by: jiffin tony Thottan <jthottan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* tests: reenable trash.tJeff Darcy2017-02-101-3/+0
| | | | | | | | | | | | | | | Now that the underlying bug has been fixed (by d97e63d0) we can allow the test to run again. Change-Id: If9736d142f414bf9af5481659c2b2673ec797a4b BUG: 1420434 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/16584 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Anoop C S <anoopcs@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* core: run many bricks within one glusterfsd processJeff Darcy2017-01-305-5/+12
| | | | | | | | | | | | | | | | | | | | | | | This patch adds support for multiple brick translator stacks running in a single brick server process. This reduces our per-brick memory usage by approximately 3x, and our appetite for TCP ports even more. It also creates potential to avoid process/thread thrashing, and to improve QoS by scheduling more carefully across the bricks, but realizing that potential will require further work. Multiplexing is controlled by the "cluster.brick-multiplex" global option. By default it's off, and bricks are started in separate processes as before. If multiplexing is enabled, then *compatible* bricks (mostly those with the same transport options) will be started in the same process. Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb BUG: 1385758 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/14763 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* jbr: Sending rollback from failed fop to fdlAvra Sengupta2016-11-083-23/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In case of a failed fop, the failure is detected by the leader in the jbr-server in two places. First during a quorum check of +ve responses when it receives responses from all the followers. At this point if the fop hasn't been successfully journaled at a quorum of followers (as in there is no merit in trying the fop in the leader as the quorum will never be met), then we fail the fop. Also if this quorum is met, then the fop is tried on the leader, and after the leader completes the fop a quorum check similar to the previous one is done again, this time including the leaders outcome. If quorum is not met, then we fail the fop. In both these cases, when the fop fails we send a -ve ack to the client. With this patch, now we will also send a rollback through a GF_FOP_IPC to all the followers(and also to the leader in the second case of failure). This rollback will contain the index and term number of the fop which failed. This will be recorded in the respective journals of the bricks and will be used to rollback the fop on that brick later. A subsequent write, and it's respective rollback would look something like the following in the journal. The trusted.jbr.term and trusted.jbr.index present in the dict of both the logs, relate them, and the presence of "rollback-fop" in the dict of IPC indicates that it is a rollback fop, and the value 13(stands for GF_FOP_WRITE) indicates what kind of rollback operation it is. === GF_FOP_WRITE fd = <gfid 77f12ea2-ca56-40e3-a46e-ba2308baa035> vector = <158 bytes> offset = 0 (0x0) flags = 32769 (0x8001) xdata = dict { trusted.jbr.term = 0 <2 bytes> trusted.jbr.index = 4 <2 bytes> } === GF_FOP_IPC xdata = dict { trusted.jbr.term = 0 <2 bytes> trusted.jbr.index = 4 <2 bytes> rollback-fop = 13 <3 bytes> } Change-Id: I70b6a143d20697153d58e2f719e34ecd1ed160a5 BUG: 1349385 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/14783 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* tests: disable lock_revocation.t on NetBSDRaghavendra Talur2016-08-311-0/+1
| | | | | | | | | | | | | | | This has been consistently causing hangs in NetBSD machines. I have not been able to debug the issue and we have merge deadline for 3.9. It would be better to disable this for now. Change-Id: I8c63940aa26f78dd9994bb63293a5757835ec52b BUG: 1369401 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/15374 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* glusterd: clean up old port and allocate new one on every restartAtin Mukherjee2016-08-031-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | GlusterD as of now was blindly assuming that the brick port which was already allocated would be available to be reused and that assumption is absolutely wrong. Solution : On first attempt, we thought GlusterD should check if the already allocated brick ports are free, if not allocate new port and pass it to the daemon. But with that approach there is a possibility that if PMAP_SIGNOUT is missed out, the stale port will be given back to the clients where connection will keep on failing. Now given the port allocation always start from base_port, if everytime a new port has to be allocated for the daemons, the port range will still be under control. So this fix tries to clean up old port using pmap_registry_remove () if any and then goes for pmap_registry_alloc () Change-Id: If54a055d01ab0cbc06589dc1191d8fc52eb2c84f BUG: 1221623 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/15005 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Avra Sengupta <asengupt@redhat.com>
* features/locks: Add lock revocation functionality to posix locks translatorRichard Wareing2016-07-181-0/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: - Motivation: Prevents cluster instability by mis-behaving clients causing bricks to OOM due to inode/entry lock pile-ups. - Adds option to strip clients of entry/inode locks after N seconds - Adds option to clear ALL locks should the revocation threshold get hit - Adds option to clear all or granted locks should the max-blocked threshold get hit (can be used in combination w/ revocation-clear-all). - Options are: features.locks-revocation-secs <integer; 0 to disable> features.locks-revocation-clear-all [on/off] features.locks-revocation-max-blocked <integer> - Adds monkey-locking option to ignore 1% of unlock requests (dev only) features.locks-monkey-unlocking [on/off] - Adds logging to indicate revocation event & reason Test Plan: First you will need TWO fuse mounts for this repro. Call them /mnt/patchy1 & /mnt/patchy2. 1. Enable monkey unlocking on the volume: gluster vol set patchy features.locks-monkey-unlocking on 2. From the "patchy1", use DD or some other utility to begin writing to a file, eventually the dd will hang due to the dropped unlocked requests. This now simulates the broken client. Run: for i in {1..1000};do dd if=/dev/zero of=/mnt/patchy1/testfile bs=1k count=10;done' ...this will eventually hang as the unlock request has been lost. 3. Goto another window and setup the mount "patchy2" @ /mnt/patchy2, and observe that 'echo "hello" >> /mnt/patchy2/testfile" will hang due to the inability of the client to take out the required lock. 4. Next, re-start the test this time enabling lock revocation; use a timeout of 2-5 seconds for testing: 'gluster vol set patchy features.locks-revocation-secs <2-5>' 5. Wait 2-5 seconds before executing step 3 above this time. Observe that this time the access to the file will succeed, and the writes on patchy1 will unblock until they hit another failed unlock request due to "monkey-unlocking". BUG: 1350867 Change-Id: I814b9f635fec53834a26db634d1300d9a61057d8 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/14816 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* api: use versioned symbols for minor ABI changeKaleb S KEITHLEY2016-06-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | No need to expand the API/ABI. E.g., see how glfs_lookupat changed between 3.7.0 and 3.7.4 IIRC (I originally argued against versioning the library. I wanted to just add new functions as they were needed, as was initially done for glfs_ipc and glfs_ipc_xd in the master branch for 4.0. But others strongly wanted versioning.) Having made the decision to use versioning, I believe we should continue. At least until we have a public decision that we're no longer going to use versioning. Change-Id: I0c3b2c1cbb297ae2b2864b647c224922987d74ad Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14717 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* glusterd: volgen and volume set changes for leasesPoornima G2016-06-021-0/+22
| | | | | | | | | | | | | Add leases xlator in volgen and also add corresponding volume set options Change-Id: Ic5de50cdb87eaf6a833e739bc7e08fecbeca3de3 BUG: 1319992 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/11722 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* features/locks: Implement mandatory locksAnoop C S2016-05-022-0/+218
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Initial change to fix/enable the mandatory locking support in GlusterFS as per the following design: https://review.gluster.org/#/c/12014/ Accordingly 'locks.mandatory-locking' option is available as part of this change which will accept one among the following values: * off * file * forced * optimal See design doc for more details Change-Id: I14c489b3f8af5ebcbfa155a03f0c175e9558ac46 BUG: 762184 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/9768 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* WORM/Retention Translator: Implementation of file level WORMkarthik-us2016-05-011-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To activate the file level worm feature, the features.read-only and features.worm options should be switched "off" on the volume and the features.worm-file-level should be switched "on". Both read-only and worm or worm-file-level cannot be switched "on" together. The files which are created when the worm-file-level option is set on the volume will have their own retention profile. If both worm and worm-file-level are "on" at that time the worm which is the volume level worm will have priority over file level worm. If worm-file level is switched "off" after some time and the read-only option is switched "on" then read-only will have priority. The current implementation allows the users to manually transmit a file to a WORM-Retained state by removing all the write bits of the file using the chmod command. The file will have a retention profile which contains the state of the file, mode of retention, and the default retention time. The file will be made WORM-Retained for a default of 120 seconds during which it will be immutable and undeletable and it sets the atime of the file to the time till which it is retained. After that period if any fop request comes for that file, will make the transition from WORM-Retained state to WORM state, where the file will be immutable but deletable and, it will reset the atime to the actual atime of the file. If a WORM file needs to be made undeletable again, it can be done by using the chmod command with all the write bits removed. There are two modes of retention: 1. Relax: where the retention time of a WORM-Retained file can be increased or decreased. 2. Enterprise: where the retention time of a WORM-Retained file can be increased but not be decreased. Whenever a utime change(touch -a, -t, ...)request comes for a file it checks the mode of retention before setting the utimes. This is done only if the file is WORM-Retained but for a WORM file it will change the utimes. Lazy auto commit: Whenever a file gets created it will store the creation time of the file or if a file already exists then any of the next unlink, link, truncate or rename fops will set the current time as the start time in an xattr. The next rename/unlink/truncate/link call will check for the auto commit period and if is is expired, then it will automatically do the state transition. If it is a normal file then it gets converted to WORM-Retained state. If it is a WORM-Retained file and its retention period is expired, then it gets converted to WORM state. Added the volume set options for the WORM translator. It allows the users to change the default values of auto-commit-period, default-retention-period, retention-mode. To make use of the file-level WORM first we have to set the 'worm-file' option to 'on'. The files which are created when the worm-file option is set on the volume will get WORM-Retained. Other files will work as usual and will not be WORMed. The auto-commit-period, retention-mode, and the default-retention-period values for the file will be set to the values which are set on the volume when the file is created. Added the tests to check the basic functionalities of the WORM/Retention feature. Change-Id: I77bd9777f9395a944d76b5cc35a5b48a3c14d148 BUG: 1326308 Signed-off-by: karthik-us <ksubrahm@redhat.com> Reviewed-on: http://review.gluster.org/13429 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* glusterd: volume set changes for lock migrationSusant Palai2016-05-011-0/+34
| | | | | | | | | | | Change-Id: I48c6f9cdda47503615ba65882acd5eedf0a70c89 BUG: 1326085 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/14024 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>