glusterfs.git/tests/basic/tier, branch v3.7.20

tier/cli : printing a warning instead of skipping the node

2016-05-20T18:19:49+00:00

        back-port of : http://review.gluster.org/#/c/14347/8

Problem: skipping the status of the nodes down creates confusion
to the user as one might see the status as completed for all nodes
and while performing detach commit, the operation will fail as the
node is down

Fix: Display a warning message

Note: When the last node is down (as per the peer list) then
warning message can't be displayed as the total number of peers
participating in the transaction is considered to be the total count.

>Change-Id: Ib7afbd1b26df3378e4d537db06f41f5c105ad86e
>BUG: 1324439
>Signed-off-by: hari gowtham 

Change-Id: Ie4296e932abaf163edc55b540b26dc6f5824ea85
BUG: 1328410
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/14458
Tested-by: hari gowtham 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

Tier/glusterd: Resetting the tier status value to not started

2016-05-06T03:43:30+00:00

        back-port of : http://review.gluster.org/#/c/14106/

Problem: during a volume restart or tier start force, the
value of tier status is set as started irrespective of the result.

Fix: The appropriate value of status is set during the restart of
rebalance function.

>Change-Id: I6164f0add48542a57dee059e80fa0f9bb036dbef
>BUG: 1315666
>Signed-off-by: hari 

Change-Id: Ie4345bd7ce1d458574e36b70fe8994b3d758396a
BUG: 1316808
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/14229
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

tests: Backport all changes to tests dir

2016-05-04T18:47:11+00:00

Test framework should be the same on all the branches. This is a copy
of all the files under tests dir from master branch. New tests in
master have not been backported, but changes to existing tests have
been.

Change-Id: I75747c525aabbd9247473dd29b3a0e7a7d93c827
BUG: 1316533
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/13683
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Tier: tier command fails message when any node is down

2016-04-22T15:12:39+00:00

        back-port of : http://review.gluster.org/#/c/13918/

PROBLEM: the dict doesn't get set on the node if its down.
so while printing the output on cli we get a ENOENT
which ends in a tier command failed.

FIX: this patch skips the node that wasn't available
and carrys on with the next node for both tier status
and tier detach status.

>Change-Id: I718a034b18b109748ec67f3ace56540c50650d23
>BUG: 1324439
>Signed-off-by: hari 
>Reviewed-on: http://review.gluster.org/13918
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Kaushal M 

Change-Id: Ia23df47596adb24816de4a2a1c8db875f145838e
BUG: 1328410
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/14030
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

tier/dht : Attach tier fix layout to run in background

2016-04-01T10:55:32+00:00

1. Spawn a thread for background fix-layout for tier process.

2. Once the fix-layout is completed a marker xttr is set on the root of
   volume to mark the completion of the background fixlayout, so that
   even if the tier process is spawned again, fixlayout will not be
   issued, if it was completed last time.

3. Please note that promotion of legacy files will happen eventually as
   the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy
   files OR the ctr lookup heal happend due to a name lookup.

4. When a detach tier is successful in evacuation data from hot tier, we remove
   the marker xattr is removed. So that next attach tier runs the background
   tier fixlayout.

what is remaining ?
1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start
   clear it during detach commit. But the issue is detach commit is a glusterd operation
   and the volume is not mounted in glusterd.
   The reason we want to do it in detach commit is that if the admin wants to attach the
   same tier again, then a background fixlayout will be triggered, which would not be needed.
2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having
   entries which will be stale when the volume is used, with ctr off (ctr is switched off only when
   we have detach commit.)

Backport of http://review.gluster.org/13491

> Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5
> BUG: 1313228
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/13491
> Smoke: Gluster Build System 
> Tested-by: Joseph Fernandes
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: Ic28affdf78d2ac0f394f3dd59f0126df7915d609
BUG: 1323016
Reviewed-on: http://review.gluster.org/13879
Smoke: Gluster Build System 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

Tier: making detach start fail when brick on hot tier is down

2016-03-08T15:50:59+00:00

        backport of : http://review.gluster.org/#/c/13474/6

Currently detach tier start happens even when a hot brick is down
this might lead to data loss.

This patch prevents the detach tier start from being executed
successfully if a brick in hot tier is down

>Change-Id: I3b6047a44bd01b8a6887d41f799f64de6bf075ef
>BUG: 1309999
>Signed-off-by: hari 

Change-Id: Ica0ae72f8e30156090be43e428545d684bdea36b
BUG: 1314617
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13602
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cli: output of rebalance to show run time in proper format

2016-03-01T04:54:20+00:00

Backport of http://review.gluster.org/#/c/10544/

Modified tests to parse the new rebalance time format.

> Change-Id: I775f13c8046dd2aeb9d4b86a737dcebb396778b4
> BUG: 1223625
> Signed-off-by: Sakshi Bansal 
> Reviewed-on: http://review.gluster.org/10544
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 

BUG: 1311822
Change-Id: I3c8106f7a1309f154861ce1b3efa202fa6c47c57
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/13514
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Tested-by: Raghavendra Talur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd/rebalance: initialize defrag variable after glusterd restart

2016-02-25T03:49:27+00:00

During reblance restart after glusterd restarted, we are not
connecting to rebalance process from glusterd, because the
defrag variable in volinfo will be null.

Initializing the variable will connect the rpc

Back port of>
>Change-Id: Id820cad6a3634a9fc976427fbe1c45844d3d4b9b
>BUG: 1303028
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/13319
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Dan Lambright 
>CentOS-regression: Gluster Build System 

(cherry picked from commit a67331f3f79e827ffa4f7a547f6898e12407bbf9)

Change-Id: Ieec82a798da937002e09fb9325c93678a5eefca8
BUG: 1311041
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13494
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

tier:unlink during migration

2016-02-22T14:50:54+00:00

files deleted during promotion were not deleting as the
files are moving from hashed to non-hashed

On deleting a file that is undergoing promotion,
the unlink call is not sent to the dst file as the
hashed subvol == cached subvol. This causes
the file to reappear once the migration is complete.

This patch also fixes a problem with stale linkfile
deleting.

Backport of>
>Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5
>BUG: 1282390
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12829
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

(cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1)

Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d
BUG: 1282388
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12991
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier: allow db queries to be interruptable

2016-02-18T13:54:58+00:00

A query to the database may take a long time if the database
has many entries. The tier daemon also sends IPC calls to the
bricks which can run slowly, espcially in RHEL6. While it is
possible to track down each such instance, the snapshot
feature should not be affected by database operations. It requires
no migration be underway. Therefore it is okay to pause tiering
at any time except when DHT is moving a file.  This fix implements
this strategy by monitoring when control passes to DHT to
migrate a file using the GF_XATTR_FILE_MIGRATE_KEY trigger. If it
is not, the pause operation is successful.

> Change-Id: I21f168b1bd424077ad5f38cf82f794060a1fabf6
> BUG: 1287842
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/13104
> Reviewed-by: Joseph Fernandes
> Tested-by: Gluster Build System 
Signed-off-by: Dan Lambright 

Change-Id: I667e0af24eaa66afefa860c4d73b324e4f39b997
BUG: 1288352
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/13199
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System