glusterfs.git/tests, branch v3.12.9 features/index: Choose different base file on EMLINK error 2018-04-12T05:20:36+00:00 Pranith Kumar K pkarampu@redhat.com 2018-03-21T12:36:54+00:00 638e57e091f5c23e5c1d0c3692ecdacf85aa9bc5 Change-Id: I4648816af908539efdc2528608aa2ebf7f0d0e2f fixes: bz#1565655 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> (cherry picked from commit bb12f2109a01856e8184e13cf984210d20155b13) 
Change-Id: I4648816af908539efdc2528608aa2ebf7f0d0e2f
fixes: bz#1565655
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
(cherry picked from commit bb12f2109a01856e8184e13cf984210d20155b13)
cluster/ec: send list-node-uuids request to all subvolumes 2018-04-06T12:49:33+00:00 Xavi Hernandez xhernandez@redhat.com 2018-03-28T09:34:49+00:00 3cd3e338ea4586b772d8be0e2b798e7fd36539b0 The xattr trusted.glusterfs.list-node-uuids was only sent to a single subvolume. This was returning null uuids from the other subvolumes as if they were down. This fix forces that xattr to be requested from all subvolumes. Backport of: > BUG: 1561406 Change-Id: If62eb39a6857258923ba625e153d4ad79018ea2f BUG: 1561731 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> 
The xattr trusted.glusterfs.list-node-uuids was only sent to a single
subvolume. This was returning null uuids from the other subvolumes as
if they were down.

This fix forces that xattr to be requested from all subvolumes.

Backport of:
> BUG: 1561406

Change-Id: If62eb39a6857258923ba625e153d4ad79018ea2f
BUG: 1561731
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
cluster/ec: Change default read policy to gfid-hash 2018-04-06T12:49:09+00:00 Ashish Pandey aspandey@redhat.com 2018-03-13T08:33:20+00:00 7061fc7e37847d0674059da0e62f42bd7032707f Problem: Whenever we read data from file over NFS, NFS reads more data then requested and caches it. Based on the stat information it makes sure that the cached/pre-read data is valid or not. Consider 4 + 2 EC volume and all the bricks are on differnt nodes. In EC, with round-robin read policy, reads are sent on different set of data bricks. This way, it balances the read fops to go on all the bricks and avoid heating UP (overloading) same set of bricks. Due to small difference in clock speed, it is possible that we get minor difference for atime, mtime or ctime for different bricks. That might cause a different stat returned to NFS based on which NFS will discard cached/pre-read data which is actually not changed and could be used. Solution: Change read policy for EC as gfid-hash. That will force all the read to go to same set of bricks. >Change-Id: I825441cc519e94bf3dc3aa0bd4cb7c6ae6392c84 >BUG: 1554743 >Signed-off-by: Ashish Pandey <aspandey@redhat.com> Change-Id: I825441cc519e94bf3dc3aa0bd4cb7c6ae6392c84 BUG: 1558352 Signed-off-by: Ashish Pandey <aspandey@redhat.com> 
Problem:
Whenever we read data from file over NFS, NFS reads
more data then requested and caches it. Based on the
stat information it makes sure that the cached/pre-read
data is valid or not.

Consider 4 + 2 EC volume and all the bricks are on
differnt nodes.

In EC, with round-robin read policy, reads are sent on
different set of data bricks. This way, it balances the
read fops to go on all the bricks and avoid heating UP
(overloading) same set of bricks.

Due to small difference in clock speed, it is possible
that we get minor difference for atime, mtime or ctime
for different bricks. That might cause a different stat
returned to NFS based on which NFS will discard
cached/pre-read data which is actually not changed and
could be used.

Solution:
Change read policy for EC as gfid-hash. That will force
all the read to go to same set of bricks.

>Change-Id: I825441cc519e94bf3dc3aa0bd4cb7c6ae6392c84
>BUG: 1554743
>Signed-off-by: Ashish Pandey <aspandey@redhat.com>

Change-Id: I825441cc519e94bf3dc3aa0bd4cb7c6ae6392c84
BUG: 1558352
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
cluster/ec: avoid delays in self-heal 2018-04-06T12:48:38+00:00 Xavi Hernandez jahernan@redhat.com 2018-02-21T16:47:37+00:00 3487b309feb864d3bb8458357f7d72cbaee8b500 Self-heal creates a thread per brick to sweep the index looking for files that need to be healed. These threads are started before the volume comes online, so nothing is done but waiting for the next sweep. This happens once per minute. When a replace brick command is executed, the new graph is loaded and all index sweeper threads started. When all bricks have reported, a getxattr request is sent to the root directory of the volume. This causes a heal on it (because the new brick doesn't have good data), and marks its contents as pending to be healed. This is done by the index sweeper thread on the next round, one minute later. This patch solves this problem by waking all index sweeper threads after a successful check on the root directory. Additionally, the index sweep thread scans the index directory sequentially, but it might happen that after healing a directory entry more index entries are created but skipped by the current directory scan. This causes the remaining entries to be processed on the next round, one minute later. The same can happen in the next round, so the heal is running in bursts and taking a lot to finish, specially on volumes with many directory levels. This patch solves this problem by immediately restarting the index sweep if a directory has been healed. Backport of: > BUG: 1547662 Change-Id: I58d9ab6ef17b30f704dc322e1d3d53b904e5f30e BUG: 1555201 Signed-off-by: Xavi Hernandez <jahernan@redhat.com> 
Self-heal creates a thread per brick to sweep the index looking for
files that need to be healed. These threads are started before the
volume comes online, so nothing is done but waiting for the next
sweep. This happens once per minute.

When a replace brick command is executed, the new graph is loaded and
all index sweeper threads started. When all bricks have reported, a
getxattr request is sent to the root directory of the volume. This
causes a heal on it (because the new brick doesn't have good data),
and marks its contents as pending to be healed. This is done by the
index sweeper thread on the next round, one minute later.

This patch solves this problem by waking all index sweeper threads
after a successful check on the root directory.

Additionally, the index sweep thread scans the index directory
sequentially, but it might happen that after healing a directory entry
more index entries are created but skipped by the current directory
scan. This causes the remaining entries to be processed on the next
round, one minute later. The same can happen in the next round, so
the heal is running in bursts and taking a lot to finish, specially
on volumes with many directory levels.

This patch solves this problem by immediately restarting the index
sweep if a directory has been healed.

Backport of:
> BUG: 1547662

Change-Id: I58d9ab6ef17b30f704dc322e1d3d53b904e5f30e
BUG: 1555201
Signed-off-by: Xavi Hernandez <jahernan@redhat.com>
cluster/afr: Fail open on split-brain 2018-03-08T06:39:57+00:00 Pranith Kumar K pkarampu@redhat.com 2017-09-04T11:27:25+00:00 e304d218602f3099dc4ba9bb86fd953cee8a8e59 Problem: Append on a file with split-brain succeeds. Open is intercepted by open-behind, when write comes on the file, open-behind does open+write. Open succeeds because afr doesn't fail it. Then write succeeds because write-behind intercepts it. Flush is also intercepted by write-behind, so the application never gets to know that the write failed. Fix: Fail open on split-brain, so that when open-behind does open+write open fails which leads to write failure. Application will know about this failure. Change-Id: I4bff1c747c97bb2925d6987f4ced5f1ce75dbc15 BUG: 1544635 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> (cherry picked from commit 786343abca3474ff01aa1017210112d97cbc4843) 
Problem:
Append on a file with split-brain succeeds. Open is intercepted by open-behind,
when write comes on the file, open-behind does open+write. Open succeeds
because afr doesn't fail it. Then write succeeds because write-behind
intercepts it. Flush is also intercepted by write-behind, so the application
never gets to know that the write failed.

Fix:
Fail open on split-brain, so that when open-behind does open+write open fails
which leads to write failure. Application will know about this failure.

Change-Id: I4bff1c747c97bb2925d6987f4ced5f1ce75dbc15
BUG: 1544635
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
(cherry picked from commit 786343abca3474ff01aa1017210112d97cbc4843)
tests/dht: Non-root can delete stale linkto files 2018-02-12T10:13:57+00:00 N Balachandran nbalacha@redhat.com 2018-02-07T15:25:47+00:00 9436976cb1971f7426ba55f28f81bac294170f90 Test to check that non-root users can delete stale linkto files > Change-Id: Ic9bc76bc485cab839927af60cfce78a058eee2e4 > BUG: 1542318 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: Ic9bc76bc485cab839927af60cfce78a058eee2e4 BUG: 1543016 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
Test to check that non-root users can delete stale
linkto files

> Change-Id: Ic9bc76bc485cab839927af60cfce78a058eee2e4
> BUG: 1542318
> Signed-off-by: N Balachandran <nbalacha@redhat.com>

Change-Id: Ic9bc76bc485cab839927af60cfce78a058eee2e4
BUG: 1543016
Signed-off-by: N Balachandran <nbalacha@redhat.com>
nfs: Fixing the failure in bug-974972.t test case 2018-02-12T10:13:32+00:00 karthik-us ksubrahm@redhat.com 2018-02-08T09:17:34+00:00 23dd9f8127bd591868cd2e31e72e72b6bd3c3803 Problem: gNFS servers is restarted even before the pending marker is set becuase of eager lock being on, leading to a file in split-brain to get healed. Fix: Switch off the eager lock. This is fixed in the later versions by the patch: https://review.gluster.org/#/c/13075/ Change-Id: If59e5ede2de1cbfcdeac01cca38dc0d046f1993c BUG: 1542475 Signed-off-by: karthik-us <ksubrahm@redhat.com> 
Problem:
gNFS servers is restarted even before the pending marker is set
becuase of eager lock being on, leading to a file in split-brain
to get healed.

Fix:
Switch off the eager lock.
This is fixed in the later versions by the patch:
https://review.gluster.org/#/c/13075/

Change-Id: If59e5ede2de1cbfcdeac01cca38dc0d046f1993c
BUG: 1542475
Signed-off-by: karthik-us <ksubrahm@redhat.com>
tests: mark test bad 2018-02-07T06:10:46+00:00 N Balachandran nbalacha@redhat.com 2018-02-07T06:08:30+00:00 e19b3354a173b3e4d652a9aca5cc066eccc08332 Please remove after fixing the test. Change-Id: If6c36dc6c395730dfb17b5b4df6f24629d904926 BUG: 1542826 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
Please remove after fixing the test.

Change-Id: If6c36dc6c395730dfb17b5b4df6f24629d904926
BUG: 1542826
Signed-off-by: N Balachandran <nbalacha@redhat.com>
tests: Spurious failure multiplex-limit-issue-151.t 2018-02-06T16:34:25+00:00 N Balachandran nbalacha@redhat.com 2017-11-27T16:48:33+00:00 b94e8c60dff6284ad30ee1b2a96e4a3221768a4d A timing issue caused the remove-brick commit to fail. Replaced 'remove-brick commit' with 'remove-brick force'. > Change-Id: I69144b2f7be34095dbd3a7d182e0bf01b27fb0a4 > BUG: 1517904 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I69144b2f7be34095dbd3a7d182e0bf01b27fb0a4 BUG: 1542615 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
A timing issue caused the remove-brick commit to fail.
Replaced 'remove-brick commit' with 'remove-brick force'.

> Change-Id: I69144b2f7be34095dbd3a7d182e0bf01b27fb0a4
> BUG: 1517904
> Signed-off-by: N Balachandran <nbalacha@redhat.com>

Change-Id: I69144b2f7be34095dbd3a7d182e0bf01b27fb0a4
BUG: 1542615
Signed-off-by: N Balachandran <nbalacha@redhat.com>
tests: Fix a broken test case 2018-02-05T13:29:58+00:00 Poornima G pgurusid@redhat.com 2017-11-29T12:54:23+00:00 4389b4fd93b85814c950aeafbeda59538b70414d Issue: When using statedump command to take statedump of the gfapi process, we specify the following things: $gluster volume statedump <volname> client <host>:<pid> pid: Pid of the gfapi application host: This should be the IP/hostname as seen by the glusterd, the gfapi application is connected to. In this test case, if gfapi application is running locally, and is connected to $H1 glusterd, the <host> need not be $H1. <host> could be localhost, 127.0.0.1, 127.1.1.1 etc. based on the configuration of the system. Hence use netstat to find the right <host> value. >mainline patch : https://review.gluster.org/#/c/18893/ Change-Id: I6efb9d1ccaf9c6841a9ab7c9ebfecafc03c0bc5e BUG: 1542054 Signed-off-by: Poornima G <pgurusid@redhat.com> 
Issue: When using statedump command to take statedump of the
gfapi process, we specify the following things:

$gluster volume statedump <volname> client <host>:<pid>
   pid: Pid of the gfapi application
   host: This should be the IP/hostname as seen by the glusterd,
         the gfapi application is connected to.

In this test case, if gfapi application is running locally,
and is connected to $H1 glusterd, the <host> need not be $H1.
<host> could be localhost, 127.0.0.1, 127.1.1.1 etc. based on
the configuration of the system. Hence use netstat to find the
right <host> value.

>mainline patch : https://review.gluster.org/#/c/18893/

Change-Id: I6efb9d1ccaf9c6841a9ab7c9ebfecafc03c0bc5e
BUG: 1542054
Signed-off-by: Poornima G <pgurusid@redhat.com>