glusterfs.git/tests/basic/afr, branch v3.7.10

afr : Enable auto heal when replica count increases

2016-03-23T11:49:36+00:00

            Backport of: http://review.gluster.org/12454

    This patch is part two change to prevent data loss
    in a replicate volume on doing a add-brick operation.

    Problem: After doing add-brick, there is a chance that
    self heal might happen from the newly added brick rather
    than the source brick, leading to data loss.

    Solution: Mark pending changelogs on afr children for
    the new afr-child so that heal is performed in the
    correct direction.

            >Change-Id: I11871e55eef3593aec874f92214a2d97da229b17
            >BUG: 1276203
            >Signed-off-by: Anuradha Talur 
            >Reviewed-on: http://review.gluster.org/12454
            >Smoke: Gluster Build System 
            >Reviewed-by: Pranith Kumar Karampuri 
            >Tested-by: Pranith Kumar Karampuri 
            >CentOS-regression: Gluster Build System 
            >NetBSD-regression: NetBSD Build System 

Change-Id: Iae6af44f97e612cb3ee8c642254ec3d15ac063f5
BUG: 1320020
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/13807
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

afr: Add throttled background client-side heals

2016-03-23T02:21:35+00:00

Backport of: http://review.gluster.org/13207

If a heal is needed after inode refresh (lookup, read_txn), launch it in
the background instead of blocking the fop (that triggered refresh)
until the heal happens.

afr_replies_interpret() is modified such that the heal is
launched only if atleast one sink brick is up.

Max. no of heals that can happen in parallel is configurable via the
'background-self-heal-count' volume option. Any number greater than that
is put in a wait queue whose length is configurable via
'heal-wait-queue-leng' volume option. If the wait queue is also full,
further heals will be ignored.

Default values:  background-self-heal-count=8, heal-wait-queue-leng=128

Change-Id: I9a134b2c29d66b70b7b1278811bd504963aabacc
BUG: 1313312
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13564
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

afr: do not set arbiter as a readable subvol in inode context

2016-03-07T08:32:00+00:00

Problem:
If afr_lookup_done() or afr_read_subvol_select_by_policy() chooses the
arbiter brick to serve the stat() data, file size will be reported as
zero from the mount, despite other data bricks being available. This can
break programs like tar which use the stat info to decide how much to read.

Fix:
In the inode-context, mark arbiter as a non-readable subvol for both
data and metadata.

It it to be noted that by making this fix, we are *not* going to serve
metadata FOPS anymore from the arbiter brick despite the brick storing
the metadata. It makes sense to do this because the ever increasing
over-loaded FOPs (getxattr returning stat data etc.) and compound FOPS
in gluster will otherwise make it difficult to add checks in code to
handle corner cases.

 >Change-Id: Ic60b25d77fd05e0897481b7fcb3716d4f2101001
 >BUG: 1310171
 >Signed-off-by: Ravishankar N 
 >Reported-by: Mat Clayton 
 >Reviewed-on: http://review.gluster.org/13539
 >Reviewed-by: Anuradha Talur 
 >Reviewed-by: Krutika Dhananjay 
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Jeff Darcy 

BUG: 1313921
Change-Id: I07fc08d633ca2af48f7354454bc2ab75cedb850a
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13609
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

Revert "afr: do not set arbiter as a readable subvol in inode context"

2016-03-05T10:47:10+00:00

This reverts commit ad0b1253b9d74797620c493184818685c024f17c.

Change-Id: Id43ba8e75d58325f897e15e3f64f9389236adb40
Reviewed-on: http://review.gluster.org/13608
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

afr: do not set arbiter as a readable subvol in inode context

2016-03-05T01:28:58+00:00

Backport-of: http://review.gluster.org/#/c/13539/

Problem:
If afr_lookup_done() or afr_read_subvol_select_by_policy() chooses the
arbiter brick to serve the stat() data, file size will be reported as
zero from the mount, despite other data bricks being available. This can
break programs like tar which use the stat info to decide how much to read.

Fix:
In the inode-context, mark arbiter as a non-readable subvol for both
data and metadata.

It it to be noted that by making this fix, we are *not* going to serve
metadata FOPS anymore from the arbiter brick despite the brick storing
the metadata. It makes sense to do this because the ever increasing
over-loaded FOPs (getxattr returning stat data etc.) and compound FOPS
in gluster will otherwise make it difficult to add checks in code to
handle corner cases.

Change-Id: Ic60b25d77fd05e0897481b7fcb3716d4f2101001
BUG: 1313921
Signed-off-by: Ravishankar N 
Reported-by: Mat Clayton 
Reviewed-on: http://review.gluster.org/13582
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cli/ afr: op_ret for index heal launch

2016-02-17T09:49:28+00:00

Backport of http://review.gluster.org/#/c/13303/

Problem:
If index heal is launched when some of the bricks are down, glustershd of that
node sends a -1 op_ret to glusterd which eventually propagates it to the CLI.
Also, glusterd sometimes sends an err_str and sometimes not (depending on the
failure happening in the brick-op phase or commit-op phase). So the message that
gets displayed varies in each case:

"Launching heal operation to perform index self heal on volume testvol has been
unsuccessful"
                (OR)
"Commit failed on . Please check log file for details."

Fix:
1. Modify afr_xl_op() to return -1 even if index healing of atleast one brick
fails.
2. Ignore glusterd's error string in gf_cli_heal_volume_cbk and print a more
meaningful message.

The patch also fixes a bug in glusterfs_handle_translator_op() where if we
encounter an error in notify of one xlator, we break out of the loop instead of
sending the notify to other xlators.

Change-Id: I957f6c4b4d0a45453ffd5488e425cab5a3e0acca
BUG: 1306922
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13435
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tests: Fix sparse-file-self-heal.t

2016-01-25T11:36:32+00:00

Backport of http://review.gluster.org/13233
Psuedo Problem:
https://build.gluster.org/job/rackspace-regression-2GB-triggered/16682/consoleFull

The 'zeroedfile' disk usage comparision which is failing in this .t file
fails so only on XFS. The test passes when the backend is on a s̶a̶n̶e̶r̶
different filesystem like EXT4 or BTRFS. This is due to the speculative
preallocation in XFS which can reserve different disk space on different
XFS mounts for the same version and same file operation. See BZ 1277992
for an example of XFS behaviour.

Fix:
Don't compare the disk usage of the file on the bricks of the replica:
instead, check that the disk space consumed is atleast equal to the size
of the file.

Also remove sparse-file-self-heal.t from is_bad_test()

Change-Id: If43f59549136ebf91f17ff9d958954b3587afe56
BUG: 1300210
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13265
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Fix arbiter-statfs.t

2015-12-14T05:29:46+00:00

..and remove it from bad tests list.

Backport of  http://review.gluster.org/#/c/12936/
Problem:
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/12516/consoleFull

++ SETUP_LOOP /d/backends/brick1
++ '[' 1 '!=' 1 ']'
++ backend=/d/backends/brick1
++ case ${OSTYPE} in
+++ awk -F: '/not in use/{print $1; exit}'
+++ vnconfig -l
vnconfig: VNDIOCGET: Bad file descriptor
++ vnd=
++ '[' x = x ']'
++ echo 'no more vnd'
no more vnd
++ return 1

Fix:
TEST the return value of SETUP_LOOP.
Also added EXIT_EARLY to the test case because there is no point in
continuing the test when setting the bricks fail.

Change-Id: Idca269650385765a13be070186dc0b7eb2e5fda1
BUG: 1290658
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/12947
Reviewed-by: Michael Adam 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

afr: write zeros to sink for non-sparse files

2015-10-29T08:40:19+00:00

Backport of http://review.gluster.org/#/c/12371/
Problem: If a file is created with zeroes ('dd', 'fallocate' etc.) when
a brick is down, the self-heal does not write the zeroes to the sink
after it comes up. Consequenty, there is a mismatch in disk-usage
amongst the bricks of the replica.

Fix: If we definitely know that the file is not sparse, then write the
zeroes to the sink even if the checksums match.

Change-Id: Ic739b3da5dbf47d99801c0e1743bb13aeb3af864
BUG: 1275921
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/12436
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tests: check free space only on gluster mount

2015-09-28T06:18:13+00:00

Back-port of http://review.gluster.org/#/c/12096/

Change-Id: I06de4f555e66fac2594676572c8f8a4ee08f8131
BUG: 1259659
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/12098
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri