glusterfs.git/tests/basic, branch v3.7.11

cluster/afr: Don't delete gfid-req from lookup request

2016-04-12T11:57:36+00:00

Problem:
Afr does dict_ref of the xattr_req that comes to it and deletes "gfid-req" key.
Dht uses same dict to send lookup to other subvolumes. So in case of
directories and more than 1 dht subvolumes, second subvolume till the last
subvolume won't get a lookup request with "gfid-req". So gfid reset never
happens on the directories in distributed replicate subvolume for 2nd till last
subvolumes.

Fix:
Make a copy of lookup xattr request.

Also fixed replies_wipe possibly resetting gfid to NULL gfid

 >BUG: 1312816
 >Change-Id: Ic16260e5a4664837d069c1dc05b9e96ca05bda88
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/13545
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 
 >(cherry picked from commit 9b022c3a3f2f774904b5b458ae065425b46cc15d)

Change-Id: Ia68193b559ec1dfd841cc5a22ef1fa801b866200
BUG: 1313693
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/13574
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System

quota: check inode limits only when new file/dir is created

2016-04-06T09:26:28+00:00

This is a backport of http://review.gluster.org/#/c/13911/

When a inode limit is full, writes to any existing file
fails with disk quota exceed even if usage limit is
not set or usage limit is not full.

> BUG: 1323486
> Change-Id: I9679fe26a2839ade0b1541fa7f0a2b71ac6dcc31
> Signed-off-by: vmallika 

Change-Id: I55ec86ebecbb8490e557c61090f6fb8c6c449ec7
BUG: 1324058
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/13912
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

afr: add mtime based split-brain resolution to CLI

2016-04-04T17:33:56+00:00

Backport of http://review.gluster.org/#/c/13828/

Extended the CLI to include support for split-brain resolution based on
mtime. The command syntax is:

$:gluster volume heal  split-brain latest-mtime 

where  can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file.

Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f
BUG: 1321748
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13838
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Anuradha Talur 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tier/dht : Attach tier fix layout to run in background

2016-04-01T10:55:32+00:00

1. Spawn a thread for background fix-layout for tier process.

2. Once the fix-layout is completed a marker xttr is set on the root of
   volume to mark the completion of the background fixlayout, so that
   even if the tier process is spawned again, fixlayout will not be
   issued, if it was completed last time.

3. Please note that promotion of legacy files will happen eventually as
   the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy
   files OR the ctr lookup heal happend due to a name lookup.

4. When a detach tier is successful in evacuation data from hot tier, we remove
   the marker xattr is removed. So that next attach tier runs the background
   tier fixlayout.

what is remaining ?
1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start
   clear it during detach commit. But the issue is detach commit is a glusterd operation
   and the volume is not mounted in glusterd.
   The reason we want to do it in detach commit is that if the admin wants to attach the
   same tier again, then a background fixlayout will be triggered, which would not be needed.
2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having
   entries which will be stale when the volume is used, with ctr off (ctr is switched off only when
   we have detach commit.)

Backport of http://review.gluster.org/13491

> Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5
> BUG: 1313228
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/13491
> Smoke: Gluster Build System 
> Tested-by: Joseph Fernandes
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: Ic28affdf78d2ac0f394f3dd59f0126df7915d609
BUG: 1323016
Reviewed-on: http://review.gluster.org/13879
Smoke: Gluster Build System 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

gfapi: preserve glfd state during glfs_dup

2016-03-30T14:23:23+00:00

Following patch introduced a new state variable in glfd
to track the current status of the fd.
http://review.gluster.org/13340/

But this state was not copied in glfd_dup function.

Backport of commit 5bdfaf98904a339144bf3a237b162e8385b95085:
> BUG: 1311146
> Change-Id: I283f8944035f6defe491f81e13d7ef28fc440572
> Signed-off-by: Rajesh Joseph 
> Reviewed-on: http://review.gluster.org/13666
> Smoke: Gluster Build System 
> Reviewed-by: Prashanth Pai 
> Tested-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Niels de Vos 

Change-Id: I283f8944035f6defe491f81e13d7ef28fc440572
BUG: 1317863
Signed-off-by: Rajesh Joseph 
Reviewed-on: http://review.gluster.org/13742
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

afr : Enable auto heal when replica count increases

2016-03-23T11:49:36+00:00

            Backport of: http://review.gluster.org/12454

    This patch is part two change to prevent data loss
    in a replicate volume on doing a add-brick operation.

    Problem: After doing add-brick, there is a chance that
    self heal might happen from the newly added brick rather
    than the source brick, leading to data loss.

    Solution: Mark pending changelogs on afr children for
    the new afr-child so that heal is performed in the
    correct direction.

            >Change-Id: I11871e55eef3593aec874f92214a2d97da229b17
            >BUG: 1276203
            >Signed-off-by: Anuradha Talur 
            >Reviewed-on: http://review.gluster.org/12454
            >Smoke: Gluster Build System 
            >Reviewed-by: Pranith Kumar Karampuri 
            >Tested-by: Pranith Kumar Karampuri 
            >CentOS-regression: Gluster Build System 
            >NetBSD-regression: NetBSD Build System 

Change-Id: Iae6af44f97e612cb3ee8c642254ec3d15ac063f5
BUG: 1320020
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/13807
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

uss/gluster: generate gfid for snapshot files from snapname and gfid

2016-03-23T11:03:52+00:00

This is a backport of http://review.gluster.org/#/c/9255/

If 'a' and 'b' are hardlinks, we need to generate a virtual
gfid for these files so that the inode number for 'a' and 'b'
are same.

Generate gfid as below:
gfid_of_a = MD5(snapname + back_end_gfid(a))
if '/dir1/a' and '/dir2/b' are hardlinks, then inode number should be
same for
all below files:
/mnt/.snaps/snap1/dir1/a
/mnt/.snaps/snap1/dir2/b
/mnt/dir1/.snaps/snap1/a
/mnt/dir2/.snaps/snap1/b

> Change-Id: Ifda793455610e554f3f1e4cbb90d44c02cda4b0f
> BUG: 1171703
> Signed-off-by: vmallika 

Change-Id: I917b2fe2915d88f69700bc8c3283e9c613e13bb8
BUG: 1316099
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/13656
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph

afr: Add throttled background client-side heals

2016-03-23T02:21:35+00:00

Backport of: http://review.gluster.org/13207

If a heal is needed after inode refresh (lookup, read_txn), launch it in
the background instead of blocking the fop (that triggered refresh)
until the heal happens.

afr_replies_interpret() is modified such that the heal is
launched only if atleast one sink brick is up.

Max. no of heals that can happen in parallel is configurable via the
'background-self-heal-count' volume option. Any number greater than that
is put in a wait queue whose length is configurable via
'heal-wait-queue-leng' volume option. If the wait queue is also full,
further heals will be ignored.

Default values:  background-self-heal-count=8, heal-wait-queue-leng=128

Change-Id: I9a134b2c29d66b70b7b1278811bd504963aabacc
BUG: 1313312
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13564
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

tests/quota : improving tests for quota

2016-03-15T09:48:10+00:00

tests/basic/quota.t includes all the basic test that
needs to be tested for quota. In most of the other
tests specific to bugs(tests/bugs/quota/*), tests
such as creating and starting volume, enabling quota,
setting limit, writing data, doing list have been done
which is essential to write a individual quota test
file, but, if the specific bug just needs to test
*few* particular cases, I have moved those tests
under tests/basic itself to speedup the regressions.

Basics of inode-quota and it's enforcing, renaming
with quota are basic tests and is hence moved under
tests/basic folder.

In other files, I have removed tests which are not
needed, such as 'pidof glusterd' or checking for
'gluster volume info' or if there are any test which
is already being tested under tests/basic and is being
written again.

        Backport of http://review.gluster.org/#/c/13216/

> Change-Id: Iefd6d9529246d59829cc5bf02687a1861d8462a8
> BUG: 1294826
> Signed-off-by: Manikandan Selvaganesh 
> Reviewed-on: http://review.gluster.org/13216
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Vijaikumar Mallikarjuna 
> Reviewed-by: Jeff Darcy 

Change-Id: Iefd6d9529246d59829cc5bf02687a1861d8462a8
BUG: 1314680
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/13606
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijaikumar Mallikarjuna

Tier: making detach start fail when brick on hot tier is down

2016-03-08T15:50:59+00:00

        backport of : http://review.gluster.org/#/c/13474/6

Currently detach tier start happens even when a hot brick is down
this might lead to data loss.

This patch prevents the detach tier start from being executed
successfully if a brick in hot tier is down

>Change-Id: I3b6047a44bd01b8a6887d41f799f64de6bf075ef
>BUG: 1309999
>Signed-off-by: hari 

Change-Id: Ica0ae72f8e30156090be43e428545d684bdea36b
BUG: 1314617
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13602
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright