summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* doc: Update release 5.0 release notesv5.0ShyamsundarR2018-10-181-3/+289
| | | | | | | | | - Added missing options - Added bugs fixed Change-Id: I3b788a093bc00fb977f792ce535ed50cc0cd9c9e Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: Bug fixes in leases processing code-pathSoumya Koduri2018-10-187-23/+64
| | | | | | | | | | | | | | | | | This patch fixes below issues in gfapi lease code-path * 'glfs_setfsleasid' should allow NULL input to be able to reset leaseid * Applications should be allowed to (un)register for upcall notifications of type GLFS_EVENT_LEASE_RECALL * APIs added to read contents of GLFS_EVENT_LEASE_RECALL argument which is of type "struct glfs_upcall_lease" This is backport of the below mainline patch - https://review.gluster.org/#/c/glusterfs/+/21391 Change-Id: I3320ddf235cc82fad561e13b9457ebd64db6c76b updates: #350 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* core: libuuid-devel breakageKaleb S. KEITHLEY2018-10-182-11/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The #include "uuid.h" left over from using .../contrib/uuid is debatably incorrect now that we use the "system header" file /usr/include/uuid/uuid.h from libuuid-devel. Unfortunately this is complicated by things like FreeBSD having its own /usr/include/uuid.h, and the e2fsprogs-libuuid uuid.h in installed - as most third-party packages in FreeBSD are - in /usr/local as /usr/local/include/uuid/uuid.h With a system header file it should at least be #include <uuid.h>, and even better as #include <uuid/uuid.h>, much like the way <sys/types.h> and <net/if.h> are included. Using #include <uuid/uuid.h> guarantees not getting the /usr/include/uuid.h on FreeBSD, but clang/cc knows to find "system" header files like this in /usr/local/include; with or without the -I/... from uuid.pc. Also using #include "uuid.h" leaves the compiler free to find a uuid.h from any -I option it might be passed. (Fortunately we don't have any at this time.) As we now require libuuid-devel or e2fsprogs-libuuid and configure will exit with an error if the uuid.pc file doesn't exist, the HAVE_LIBUUID (including the #elif FreeBSD) tests in compat-uuid.h are redundant. We are guaranteed to have it, so testing for it is a bit silly IMO. It may also break building third party configure scripts if they omit defining it. (Just how hard do we want to make things for third party developers?) Change-Id: I7317f63c806281a5d27de7d3b2208d86965545e1 updates: bz#1639688 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* debug/io-stats: io stats filenames contain garbageN Balachandran2018-10-181-4/+23
| | | | | | | | | | | | | | | | As dict_unserialize does not null terminate the value, using snprintf adds garbage characters to the buffer used to create the filename. The code also used this->name in the filename which will be the same for all bricks for a volume. The files were thus overwritten if a node contained multiple bricks for a volume. The code now uses the conf->unique instead if available. Change-Id: I2c72534b32634b87961d3b3f7d53c5f2ca2c068c fixes: bz#1640392 Signed-off-by: N Balachandran <nbalacha@redhat.com> (cherry picked from commit 219cd649fdbd7bfd6c2268a0a4f66bcc15918e31)
* api: fill out attribute information if not validRaghavendra Gowdappa2018-10-173-2/+147
| | | | | | | | | | | | | | | | | | translators like readdir-ahead selectively retain entry information of iatt (gfid and type) when rest of the iatt is invalidated (for write invalidating ia_size, (m)(c)times etc). Fuse-bridge uses this information and sends only entry information in readdirplus response. However such option doesn't exist in gfapi. This patch modifies gfapi to populate the stat by forcing an extra lookup. Thanks to Shyamsundar Ranganathan <srangana@redhat.com> and Prashanth Pai <ppai@redhat.com> for tests. Change-Id: Ieb5f8fc76359c327627b7d8420aaf20810e53000 Fixes: bz#1630804 Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com> Signed-off-by: Soumya Koduri <skoduri@redhat.com> (cherry picked from commit 6257276d9de3f15643f159b2ec627a67c84fc23d)
* afr: fix incorrect reporting of directory split-brainRavishankar N2018-10-119-17/+85
| | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/#/c/glusterfs/+/21135/ Problem: When a directory has dirty xattrs due to failed post-ops or when replace/reset brick is performed, AFR does a conservative merge as expected, but heal-info reports it as split-brain because there are no clear sources. Fix: Modify pending flag to contain information about pending heals and split-brains. For directories, if spit-brain flag is not set,just show them as needing heal and not being in split-brain. Change-Id: I09ef821f6887c87d315ae99e6b1de05103cd9383 fixes: bz#1638163 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* afr: prevent winding inodelks twice for arbiter volumesRavishankar N2018-10-112-1/+45
| | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/#/c/glusterfs/+/21380/ Problem: In an arbiter volume, if there is a pending data heal of a file only on arbiter brick, self-heal takes inodelks twice due to a code-bug but unlocks it only once, leaving behind a stale lock on the brick. This causes the next write to the file to hang. Fix: Fix the code-bug to take lock only once. This bug was introduced master with commit eb472d82a083883335bc494b87ea175ac43471ff Thanks to Pranith Kumar K <pkarampu@redhat.com> for finding the RCA. fixes: bz#1638159 Change-Id: I15ad969e10a6a3c4bd255e2948b6be6dcddc61e1 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* cliutils : python2 to python3 compatSunny Kumar2018-10-101-13/+13
| | | | | | | | | | | This patch fixes import issue in cliutils. Provided solution is to use relative import. Change-Id: I14c9a0b528ef52e7c91f6b17b569c68c2ced8912 updates: #411 Signed-off-by: Sunny Kumar <sunkumar@redhat.com> (cherry picked from commit 8d4c5e022bba1b99786ce13f407c27024beccc23)
* georep: python2 to python3 compat - syscallsKotresh HR2018-10-107-35/+88
| | | | | | | | | | | | | 1. ctypes/syscalls A) arguments is expected to be encoded B) Raw conversion of return value from bytearray into string 2. struct pack/unpack - Raw converstion of string to bytearray 3. basestring -> str Updates: #411 Change-Id: I80f939adcdec0ed0022c87c0b76d057ad5559e5a Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit fb6e8d0d0ca21b16d331fa69da9b9dadf6c5c35d)
* events: python2 to python3 compatKotresh HR2018-10-103-12/+13
| | | | | | | | | | | 1. Fix imports 2. Use universal_newlines flag in Popen 3. encode msg used in 'sendto' Updates: #411 Change-Id: Ieacdc2ba86953ff3a2e9d9b9a97157e8e0e36836 Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit b5f75e337c19d4fed85d4dcf15cf47f02fea5f6a)
* posix: Fix exporting default value for `export-statfs-size`Aravinda VK2018-10-081-1/+3
| | | | | | | | | | | | | | | | | | | No default value was specified for `export-statfs-size` in posix option table. Glusterd2 sets default value as `off` since the option type is `bool`. Posix treats `export-statfs-size=on` if not specified in volfile(That means default value is `on`) This patch sets default value as `on` > Change-Id: I5c6341183be9b62a78fdbc94621220f9284e1382 > updates: #302 > Signed-off-by: Aravinda VK <avishwan@redhat.com> (cherry picked from commit 07088d95e450f847722e5decbfa5da18a0dbd9de) Change-Id: Ib6b3accdb9921376c16040bd2312b99b0226a26f Fixes: bz#1636842 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* doc: Initial release notes for 5.0v5.0rc1ShyamsundarR2018-10-051-0/+155
| | | | | | Change-Id: Id2ed7e04fc7fe708a144ac09c69b837956aedb94 Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* cluster/afr: Make data eager-lock decision based on number of locksPranith Kumar K2018-10-053-6/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For both Virt and block workloads the file is opened multiple times leading to dynamically setting eager-lock to off for the workload. Instead of depending on the number-of-open-fds, if we change the logic to depend on number of inodelks, then it will give better performance than the earlier logic. When there is an eager-lock and number of inodelks is more than 1 we know that there is a conflicting lock, so depend on that information to decide whether to keep the current transaction go through delayed-post-op or not. Locks xlator doesn't have implementation to query number of locks in fxattrop in releases older than 3.10 so to keep things backward compatible in 3.12, data transactions will use new logic where as fxattrop transactions will use old logic. I am planning to send one more patch which makes metadata domain locks also depend on inodelk-count Profile info for a dd of 500MB to a file with another fd opened on the file using exec 250>filename Without this patch: 0.14 67.41 us 16.72 us 3870.82 us 892 FINODELK 0.59 279.87 us 95.71 us 2085.89 us 898 FXATTROP 3.46 366.43 us 81.75 us 6952.79 us 4000 WRITE 95.79 148733.99 us 50568.12 us 919127.86 us 273 FSYNC With this patch: 0.00 51.01 us 38.07 us 80.16 us 4 FINODELK 0.00 235.43 us 235.43 us 235.43 us 1 TRUNCATE 0.00 125.07 us 56.80 us 193.33 us 2 GETXATTR 0.00 135.86 us 62.13 us 209.59 us 2 INODELK 0.00 197.88 us 155.39 us 253.90 us 4 FXATTROP 0.00 450.59 us 394.28 us 506.89 us 2 XATTROP 0.00 56.96 us 19.06 us 406.59 us 23 FLUSH 37.81 273648.93 us 48.43 us 6017657.05 us 44 LOOKUP 62.18 4951.86 us 93.80 us 1143154.75 us 3999 WRITE postgresql benchmark performance changed from ~1130 TPS to ~2300TPS randio fio job inside Ovirt based VM went from ~600IOPs to ~2000IOPS fixes bz#1635972 Change-Id: If7f7388d2f08cf7f17ca517a4ea222560661dc36 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* cluster/afr: Batch writes in same lock even when multiple fds are openPranith Kumar K2018-10-051-9/+2
| | | | | | | | | | | | | | | | | | | | | | Problem: When eager-lock is disabled because of multiple-fds opened and app writes come on conflicting regions, the number of locks grows very fast leading to all the CPU being spent just in locking and unlocking by traversing huge queues in locks xlator for granting locks. Fix: Reduce the number of locks in transit by bundling the writes in the same lock and disable delayed piggy-pack when we learn that multiple fds are open on the file. This will reduce the size of queues in the locks xlator. This also reduces the number of network calls like inodelk/fxattrop. Please note that this problem can still happen if eager-lock is disabled as the writes will not be bundled in the same lock. fixes bz#1635975 Change-Id: I8fd1cf229aed54ce5abd4e6226351a039924dd91 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* mgmt/glusterd: use proper path to the volfileRaghavendra Bhat2018-10-053-9/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Till now, glusterd was generating the volfile path for the snapshot volume's bricks like this. /snaps/<snap name>/<brick volfile> But in reality, the path to the brick volfile for a snapshot volume is /snaps/<snap name>/<snap volume name>/<brick volfile> The above workaround was used to distinguish between a mount command used to mount the snapshot volume, and a brick of the snapshot volume, so that based on what is actually happening, glusterd can return the proper volfile (client volfile for the former and the brick volfile for the latter). But, this was causing problems for snapshot restore when brick multiplexing is enabled. Because, with brick multiplexing, it tries to find the volfile and sends GETSPEC rpc call to glusterd using the 2nd style of path i.e. /snaps/<snap name>/<snap volume name>/<brick volfile> So, when the snapshot brick (which is multiplexed) sends a GETSPEC rpc request to glusterd for obtaining the brick volume file, glusterd was returning the client volume file of the snapshot volume instead of the brick volume file. Change-Id: I28b2dfa5d9b379fe943db92c2fdfea879a6a594e fixes: bz#1636162 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> (cherry picked from commit 83a89296a3d12a3fc2a643c0630be5ce659204ea)
* georep: Fix python3 compatibility (configparser)Kotresh HR2018-10-051-5/+2
| | | | | | | | | | '%' needs special handling in config and also removed duplicate misspelled 'changelog-archive-format' config. Updates: #411 Change-Id: I33621a62bdf5f781ee62e6cedec0c2df3f5d70cf Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 62164ac0d242a271f19b0e0bb352af5f7df082ce)
* georep: python3 compatibility (Popen)Kotresh HR2018-10-053-12/+21
| | | | | | | | | | | | | | | The file objects for python3 by default is opened in binary mode where as in python2 it's opened as text by default. The geo-rep code parses the output of Popen assuming it as text, hence used the 'universal_newlines' flag which provides backward compatibility for the same. Change-Id: I371a03b6348af9666164cb2e8b93d47475431ad9 Updates: #411 Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 65aed1070cc2e44959cf3a0fbfde635de7e03103)
* georep: Fix python3 compatibility (os.pipe)Kotresh HR2018-10-053-7/+19
| | | | | | | | | | | | | | | 'os.pipe' returns pair of file descriptors which are non-inheritable by child processes. But geo-rep uses te inheritable nature of pipe fds to communicate between parent and child processes. Hence wrote a compatiable pipe routine which works well both with python2 and python3 with inheritable nature. Updates: #411 Change-Id: I869d7a52eeecdecf3851d44ed400e69b32a612d9 Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 173e89a6506bc8c727ce6d8e5ac84b59ad2e21de)
* georep: python2 and python3 compat - bytes-strKotresh HR2018-10-051-3/+5
| | | | | | | | | 1. Fix fdopen used for pid file 2. Fix sha256 checksum calculation Updates: #411 Change-Id: Ic173d104a73822c29aca260ba6de872cd8d23f86 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: python2 to python3 compatibility-mount writeKotresh HR2018-10-051-3/+9
| | | | | | | | | | python3 expects byte string for os.write. This works for both py2 and py3. Fixed the same for geo-rep mount testing code path. Updates: #411 Change-Id: I2dfedcb0869457707bcca4d2847ef0d52bff1987 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* georep: python2 to python3 compat - pickleKotresh HR2018-10-051-4/+12
| | | | | | | | | | | | | | | | | | | | | | Handle py2 and py3 compatibility for pickling and unpickling. Geo-rep pickles and unpickles reading sys.stdin and sys.stdout streams. py2 and py3 compatibility expects the streams to be opened in binary mode but the sys.stdout objects are different in python2 and python3 python2: >>> type(sys.stdout) <type 'file'> python3: >>> type(sys.stdout) <class '_io.TextIOWrapper'> So in order to access binary stream, using sys.stdin.buffer in python3 Updates: #411 Change-Id: I1a633ccdddff5baf0cf05a8b493add39ddf75bd7 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* mdcache: Fix asan reported potential heap buffer overflowShyamsundarR2018-10-021-0/+1
| | | | | | | | | | | | | | | The char pointer mdc_xattr_str in function mdc_xattr_list_populate is malloc'd and doing a strcat into a malloc'd region can overflow content allocated based on prior contents of the memory region. Added a NULL terimation to the malloc'd region to prevent the overflow, and treat it as an empty string. Change-Id: If0decab669551581230a8ede4c44c319ff04bac9 Updates: bz#1635373 Signed-off-by: ShyamsundarR <srangana@redhat.com> (cherry picked from commit d00a2a1b398346bbdc5ac9b3ba4b09fb1ce1e699)
* ctime: Provide noatime optionKotresh HR2018-10-0212-9/+191
| | | | | | | | | | | | | | | | | | | | | | | | Most of the applications are {c|m}time dependant and very few are atime dependant. So provide noatime option to not update atime when ctime feature is enabled. Also this option has to be enabled with ctime feature to avoid unnecessary self heal. Since AFR/EC reads data from single subvolume, atime is only updated in one subvolume triggering self heal. Backport of: > Patch: https://review.gluster.org/21073 > BUG: 1593538 > Change-Id: I085fb33c882296545345f5df194cde7b6cbc337e > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 89636be4c73b12de2e11c75d8e59527bb243f147) updates: bz#1633015 Change-Id: I085fb33c882296545345f5df194cde7b6cbc337e Signed-off-by: Kotresh HR <khiremat@redhat.com>
* python3: assume python3 unless building _packages_ on sys without py3Kaleb S. KEITHLEY2018-09-2713-24/+7
| | | | | | | | | | | | | | | The jenkins release-new job runs on a CentOS 7 box, which does not have python3. As a result it runs (autogen.sh and) configure before producing the dist tar file, converting all the python3 shebangs to python2 shebangs in the dist tar file. Then when that tar file is "carried" to, e.g. Fedora koji build system to build packages, the shebangs are incorrect, despite having originally been correct in the git repo. Change-Id: I5154baba3f6d29d3c4823bafc2b57abecbf90e5b updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* glusterd: acquire lock to update volinfo structureSanju Rakonde2018-09-273-34/+42
| | | | | | | | | | | | | | | | | | | | | | | | Problem: With commit cb0339f92, we are using a separate syntask for restart_bricks. There can be a situation where two threads are accessing the same volinfo structure at the same time and updating volinfo structure. This can lead volinfo to have inconsistent values and assertion failures because of unexpected values. Solution: While updating the volinfo structure, acquire a store_volinfo_lock, and release the lock only when the thread completed its critical section part. > BUG: bz#1627610 > Signed-off-by: Sanju Rakonde <srakonde@redhat.com> > Change-Id: I545e4e2368e3285d8f7aa28081ff4448abb72f5d (cherry picked from commit 484f417da945cf83afdbf136bb4817311862a8d2) fixes: bz#1633552 Change-Id: I545e4e2368e3285d8f7aa28081ff4448abb72f5d Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* glusterd: make sure that brickinfo->uuid is not nullSanju Rakonde2018-09-271-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: After an upgrade from the version where shared-brick-count option is not present to a version which introduced this option causes issue at the mount point i.e, size of the volume at mount point will be reduced by shared-brick-count value times. Cause: shared-brick-count is equal to the number of bricks that are sharing the file system. gd_set_shared_brick_count() calculates the shared-brick-count value based on uuid of the node and fsid of the brick. https://review.gluster.org/#/c/glusterfs/+/19484 handles setting of fsid properly during an upgrade path. This patch assumed that when the code path is reached, brickinfo->uuid is non-null. But brickinfo->uuid is null for all the bricks, as the uuid is null https://review.gluster.org/#/c/glusterfs/+/19484 couldn't reached the code path to set the fsid for bricks. So, we had fsid as 0 for all bricks, which resulted in gd_set_shared_brick_count() to calculate shared-brick-count in a wrong way. i.e, the logic written in gd_set_shared_brick_count() didn't work as expected since fsid is 0. Solution: Before control reaches the code path written by https://review.gluster.org/#/c/glusterfs/+/19484, adding a check for whether brickinfo->uuid is null and if brickinfo->uuid is having null value, calling glusterd_resolve_brick will set the brickinfo->uuid to a proper value. When we have proper uuid, fsid for the bricks will be set properly and shared-brick-count value will be caluculated correctly. Please take a look at the bug https://bugzilla.redhat.com/show_bug.cgi?id=1632889 for complete RCA Steps followed to test the fix: 1. Created a 2 node cluster, the cluster is running with binary which doesn't have shared-brick-count option 2. Created a 2x(2+1) volume and started it 3. Mouted the volume, checked size of volume using df 4. Upgrade to a version where shared-brick-count is introduced (upgraded the nodes one by one i.e, stop the glusterd, upgrade the node and start the glusterd). 5. after upgrading both the nodes, bumped up the cluster.op-version 6. At mount point, df shows the correct size for volume. > backport of https://review.gluster.org/#/c/glusterfs/+/21278/ > Change-Id: Ib9f078aafb15e899a01086eae113270657ea916b > Signed-off-by: Sanju Rakonde <srakonde@redhat.com> (cherry picked from commit f1e9b878ce2067db83a0baa5f384eda87287719d) fixes: bz#1633242 Change-Id: Ib9f078aafb15e899a01086eae113270657ea916b Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* geo-rep: Fix issues related config setKotresh HR2018-09-196-11/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | 1. '--ignore-mising-args' option for rsync is not being used even though the rsync version is greater than 3.1.0. Fixed the same. 2. '--existing' option for rsync is also not being used. Fixed the same. 3. geo-rep config fails to set rsync-options as the value contains '--'. Interestingly, python argsparse treats the value with '--' (e.g., --ignore-missing-args) as option. But when passed with something like --value=--ignore-missing-args, it succeeds. Fixed the same. Backport of: > Patch: https://review.gluster.org/21191 > Change-Id: Iaeb838acaff1c2920fee9c7f920c99edce13a0a1 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > BUG: 1629561 (cherry picked from commit b977b44dd0adfcd7a3b432844260de4b8d1c4adf) Change-Id: Iaeb838acaff1c2920fee9c7f920c99edce13a0a1 Signed-off-by: Kotresh HR <khiremat@redhat.com> fixes: bz#1630673
* gfapi: Cleanup alias filev5.0rc0ShyamsundarR2018-09-173-8/+1
| | | | | | | | | 2 extra symbols were in the alias file, that are not exported this is cleaned up with this patch. Change-Id: I1ab54b9fb6b0d455884fbbfd89820c60bb861e6f Updates: bz#1629877 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* core: remove experimental xlators and associated testsShyamsundarR2018-09-1757-5956/+2
| | | | | | | | experimental xlators removed from 5.0 Change-Id: I47219d8b95efc3d5875ec9224d1e79f8371e9f76 Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* glusterd: Update op-version from 4.2 to 5.0ShyamsundarR2018-09-175-20/+20
| | | | | | | | | | | | Post changing the max op-version to 4.2, after release 4.1 branching, the decision was to go with increasing release numbers. Thus this needs to change to 5.0. This commit addresses the above change. Fixes: bz#1628668 Change-Id: Ifcc0c6da90fdd51e4eceea40749511110a432cce Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: revert several patchs that introduced pre/post attrsShyamsundarR2018-09-1721-307/+90
| | | | | | | | | | | | | | Reverted the following: - 248152767b0599986bbb6bb35fc27197f6be6964 - 09943beb499617212f2985ca8ea9ecd1ed1b470e - d01f7244e9d9f7e3ef84e0ba7b48ef1b1b09d809 The reverts are redone by hand, due to clang format changes that made using git to revert the changes more tedious. Change-Id: I96489638a2b641fb2206a110298543225783f7be Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* gfapi: revert "gfapi: return pre/post attributes at callback for glfs api"ShyamsundarR2018-09-176-370/+93
| | | | | | | | | | | | | | This reverts commit 384562b294e9a7847403961e878a4daa0fff33eb. The revert is done manually owing to the clang format changes, and the 4.1 patch that reverts this fix is used as the source for the revert. The 4.1 patch has the commit ID: 98376e0c0a Change-Id: Ib2cbce9940f6a2a2755eb47bf332832147835e4d Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* core: move logs which are only developer relevant to DEBUG levelShyamsundarR2018-09-135-7/+7
| | | | | | | | | | | | For each release, we move certain logs on master that are meant as an indicator to contributors to correct or adapt to core infrastructure changes in libglusterfs and other places. This commit achieves the above. Change-Id: I4157a7ec7d5ec9c2948b2bbc1e4cb8317f28d6b8 Updates: bz#1628620 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* Update rfc.sh for release-5v5.0alphaShyamsundarR2018-09-131-1/+1
| | | | Signed-off-by: ShyamsundarR <srangana@redhat.com>
* build: cleanup xlator link, --no-undefined, libuuidv6devKaleb S. KEITHLEY2018-09-128-18/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While attempting to build a (pre-)5.0 of glusterfs on Ubuntu bionic and cosmic, it became apparent that there are some gremlins hiding in the combination of the xlator export-symbols, the newish addition of -Wl,--no-undefined, and the new switch to libuuid from the old contrib/uuid. Note: even though Fedora 28 (and later) and Ubuntu bionic (and later) have the same nominal version of libtool, the Fedora version appears to do a better job of recursing through dependencies to determine the libraries to link with. Examination of the build logs showed that despite appearing to work on Fedora, not all xlators and shared libs were linked with -Wl, --no-undefined, and -luuid. And in the case of the gnfs xlator, it was not only not linked with -Wl,--no-undefined but alsos not linked with -lgfxdr and -lgfrpc. Added GF_XLATOR_LDFLAGS, similar to GF_XLATOR_DEFAULT_LDFLAGS. GF_XLATOR_DEFAULT_LDFLAGS is for xlators that export/expose the default or common set of symbols. GF_XLATOR_LDFLAGS is for those remaining xlators that export/expose non-default symbols, e.g. dht and glupy. This removes the need in the future to add things like $(UUID_LIBS) to every xlator's Makefile.am. Just add it to GF_XLATOR_LDFLAGS and GF_XLATOR_DEFAULT_LDFLAGS in configure.ac and you're done. This patch was tested on Fedora 28 (build, rpmbuild), Fedora Rawhide/30 (rpmbuild), RHEL8 (rpmbuild), CentOS7 (rpmbuild), Fedora koji --scratch build for f30/rawhide, and a Launchpad build for Ubuntu cosmic/18.10. Change-Id: Ieca104fa5c5d3c094e701c8ca4a73754dd0292b0 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* template files: revert clangAmar Tumballi2018-09-125-1169/+1260
| | | | | | Change-Id: If3925191d23afe83cbbdbc3cf0554c0a9c76d043 updates: bz#1564149 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-12490-439790/+426792
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* Land clang-format changesGluster Ant2018-09-12364-31211/+29701
| | | | Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
* clang-format: add the config fileAmar Tumballi2018-09-125-1/+151
| | | | | | | | Also update the required documents and scripts to enable clang-format Change-Id: I73aae6db06c2f732a1779d59a73bc05e28beafba updates: bz#1564149 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* doc: make developer-index.md as READMEAmar Tumballi2018-09-121-0/+0
| | | | | | | | | | This helps to traverse the docs well as README.md is rendered by default in github pages, and would help to provide better indexed document for new users. updates: bz#1193929 Change-Id: I4130dfd16aed924ecab17f96326cedb86892d6a6 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* dht: Use snprintf instead of strncpyN Balachandran2018-09-121-9/+20
| | | | | | | | | | | The recent changes to use malloc instead of calloc left the new_name and new_path non-null terminated. We now use snprintf instead of strncpy to fix this. Change-Id: I1a31701ca9447efde38921be0ba2c73cde2e7976 fixes: bz#1626346 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* doc: add coding-standard and commit-msg link in READMEAmar Tumballi2018-09-111-0/+8
| | | | | | | | | Having the links in README makes people follow it if they browse through github. Updates: bz#1193929 Change-Id: I580332a41d9b52858aa9796a74a563ff57a60b4b Signed-off-by: Amar Tumballi <amarts@redhat.com>
* mount/fuse: convert ENOENT to ESTALE in open(dir)_resumeRaghavendra G2018-09-111-0/+9
| | | | | | | | | | | | | | | | | | | | | | This patch is continuation of commit fb4b914ce84bc83a5f418719c5ba7c25689a9251. <snip> mount/fuse: never fail open(dir) with ENOENT open(dir) being an operation on inode should never fail with ENOENT. If gfid is not present, the appropriate error is ESTALE. This will enable kernel to retry open after a revalidate lookup. </snip> Earlier commit failed to fix codepath where error response is sent back on gfid resolution failures in fuse_open(dir)_resume. Current patch completes that work Change-Id: Ia07e3cece404811703c8cfbac9b402ca5fe98c1e Signed-off-by: Raghavendra G <rgowdapp@redhat.com> updates: bz#1627620
* mgmt xlators: store boolean fields using integerYaniv Kaul2018-09-112-8/+11
| | | | | | | | | | | | | | | | | | | | Surprisingly, there is not set_boolean() as there is a get_boolean() In fact, it is stored as an INT dictionary type. In some occasions it was stored using a string, and this caused errors such as: key gfproxy-server, integer type asked, has string type [Invalid argument] I've fixed what I saw in some logs, I'm sure there are more. The CORRECT fix is to create a boolean set and use it, but this requires a bit more work. I'll see if I can do it later on. Only compile-tested! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I45fd0c7a0824b2f42b8ce510296c9dfa4f32ad66
* glusterd: NULL pointer dereferencing clang fixHarpreet Lalwani2018-09-111-2/+1
| | | | | | | | | | | problem: NULL point dereferencing solution: Adding a conditional statement before and then dereferencing it. Updates: bz#1622665 Change-Id: I562ca90aebf2a4882cfea10114a90364d9ef1996 Signed-off-by: Harpreet Lalwani <hlalwani@redhat.com>
* misc: fix misc. shebangsKaleb S. KEITHLEY2018-09-118-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * One #!/usr/bin/env python and three #!/usr/bin/python were overlooked in all the other python fixups. Ugh. * Two new python files missed the memo about #!/usr/bin/python3. * One #!/usr/bin/env bash. Various distribution packaging policies have strong wording about the use of #!/usr/bin/env ... Note: this patch does not change the use of #!/usr/bin/env bash in the two files extras/{clang-checker.sh,check_goto.pl} as these are not included in any packages. (Although I'm not actually sure why anyone would ever use '/usr/bin/env {sh,bash}' as I'm not aware of any version-specific differences like there are with, e.g., python.) * One #!/usr/bin/bash. On Fedora and CentOS > 6, /bin is a symlink to /usr/bin, so it makes little difference. But Debian & Ubuntu still have separate /bin and /usr/bin; and sh and bash are in /bin, not /usr/bin. (Historically, in BSD and SYSV Unix it was /bin/sh.) Note: Fedora and CentOS package build runs a script that converts all /bin/sh and /bin/bash to /usr/bin/sh and /usr/bin/bash. Change-Id: I9171265829af78dd0cd7622c22b56d22179ff8a3 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* rpm: add "--with asan" build optionNiels de Vos2018-09-112-1/+17
| | | | | | | | | | | | RHEL-6 does not have libasan, enabling the "--with asan" option is a no-op there. RHEL-7 has an earlier version of libasan, and that does not have the __asan_init symbol. Test for __asan_report_error in confiure.as instead. Change-Id: I6322e832c5cfbd7d750f5c32c84c28771674ced6 Updates: #492 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* storage/posix: Fix coverity issue - Unchecked return valueAshish Pandey2018-09-111-1/+8
| | | | | | | | | | Fixes CID: 1388886 https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=85287446&defectInstanceId=25997291&mergedDefectId=1388886 Change-Id: Ic4e558bba7e15d213c07bc31affb2e175ace5502 updates: bz#789278 Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* performance/write-behind: remove the request from wip queue in ↵Raghavendra G2018-09-111-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | wb_fulfill_request The bug is very similar to bz 1379655 and the fix too very similar to commit a8b2a981881221925bb5edfe7bb65b25ad855c04. Before this patch, a request is removed from wip queue only when ref count of request hits 0. Though, wb_fulfill_request does an unref, it need not be the last unref and hence the request may survive in wip queue till the last unref. Let, T1: the time at which wb_fulfill_request is invoked T2: the time at which last unref is done on request Let's consider a case of T2 > T1. In the time window between T1 and T2, any other request (waiter) conflicting with request in liability queue (blocker - basically a write which has been lied) is blocked from winding. If T2 happens to be when wb_do_unwinds is invoked, no further processing of request list happens and "waiter" would get blocked forever. An example imaginary sequence of events is given below: 1. A write request w1 is picked up for winding in __wb_pick_winds and w1 is moved to wip queue. Let's call this invocation of wb_process_queue by wb_writev as PQ1. Note w1 is not unwound. 2. A dependent write (w2) hits write-behind and is unwound followed by a flush (f1) request. Since the liability queue of inode is not empty, w2 and f1 are not picked for unwinding. Let's call the invocation of wb_process_queue by wb_flush as PQ2. Note that invocation of wb_process_queue by w2 doesn't wind w2 instead unwinds it after which we hit PQ2 3. PQ2 continues and picks w1 for fulfilling and invokes wb_fulfill. As part of successful wb_fulfill_cbk, wb_fulfill_request (w1) is invoked. But, w1 is not freed (and hence not removed from wip queue) as w1 is not unwound _yet_ and a ref remains (PQ1 has not invoked wb_do_unwinds _yet_). 4. wb_fulfill_cbk (triggered by PQ2) invokes a wb_process_queue (let's say PQ3). w2 is not picked up for winding in PQ3 as w1 is still in wip queue. At this time, PQ2 and PQ3 are complete. 5. PQ1 continues, unwinds w1 and does last unref on w1 and w1 is freed (and removed from wip queue). Since PQ1 didn't invoke wb_fulfill on any other write requests, there won't be any future codepaths that would invoke wb_process_queue and w2 is stuck forever. This will prevent f2 too and hence close syscall is hung With this fix, w1 is removed from liability queue in step 3 above and PQ3 winds w2 in step 4 (as there are no requests conflicting with w2 in liability queue during execution of PQ3). Once w2 is complete, f1 is resumed. Change-Id: Ia972fad0858dc4abccdc1227cb4d880f85b3b89b Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Fixes: bz#1626787
* cli: Add warning message while converting to replica 2 configurationkarthik-us2018-09-113-15/+59
| | | | | | | | | | | | | | Currently while creating replica 2 volume we display a warning message of ending up in split-brain. But while converting an existing volume from other configuration to replica 2 by add-brick or remove-brick operations we do not show any such messages. With this fix in add-brick and remove-brick cases also we will display the same warning message and prompt for confirmation if the configuration changes to replica 2. Change-Id: Ifc4ed6994a087d2403894f4e743c4eb41633276b fixes: bz#1627044 Signed-off-by: karthik-us <ksubrahm@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-18 12:58:38 +0000