glusterfs.git/tests/basic/tier, branch v3.7.11

tier/dht : Attach tier fix layout to run in background

2016-04-01T10:55:32+00:00

1. Spawn a thread for background fix-layout for tier process.

2. Once the fix-layout is completed a marker xttr is set on the root of
   volume to mark the completion of the background fixlayout, so that
   even if the tier process is spawned again, fixlayout will not be
   issued, if it was completed last time.

3. Please note that promotion of legacy files will happen eventually as
   the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy
   files OR the ctr lookup heal happend due to a name lookup.

4. When a detach tier is successful in evacuation data from hot tier, we remove
   the marker xattr is removed. So that next attach tier runs the background
   tier fixlayout.

what is remaining ?
1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start
   clear it during detach commit. But the issue is detach commit is a glusterd operation
   and the volume is not mounted in glusterd.
   The reason we want to do it in detach commit is that if the admin wants to attach the
   same tier again, then a background fixlayout will be triggered, which would not be needed.
2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having
   entries which will be stale when the volume is used, with ctr off (ctr is switched off only when
   we have detach commit.)

Backport of http://review.gluster.org/13491

> Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5
> BUG: 1313228
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/13491
> Smoke: Gluster Build System 
> Tested-by: Joseph Fernandes
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: Ic28affdf78d2ac0f394f3dd59f0126df7915d609
BUG: 1323016
Reviewed-on: http://review.gluster.org/13879
Smoke: Gluster Build System 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

Tier: making detach start fail when brick on hot tier is down

2016-03-08T15:50:59+00:00

        backport of : http://review.gluster.org/#/c/13474/6

Currently detach tier start happens even when a hot brick is down
this might lead to data loss.

This patch prevents the detach tier start from being executed
successfully if a brick in hot tier is down

>Change-Id: I3b6047a44bd01b8a6887d41f799f64de6bf075ef
>BUG: 1309999
>Signed-off-by: hari 

Change-Id: Ica0ae72f8e30156090be43e428545d684bdea36b
BUG: 1314617
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13602
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cli: output of rebalance to show run time in proper format

2016-03-01T04:54:20+00:00

Backport of http://review.gluster.org/#/c/10544/

Modified tests to parse the new rebalance time format.

> Change-Id: I775f13c8046dd2aeb9d4b86a737dcebb396778b4
> BUG: 1223625
> Signed-off-by: Sakshi Bansal 
> Reviewed-on: http://review.gluster.org/10544
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 

BUG: 1311822
Change-Id: I3c8106f7a1309f154861ce1b3efa202fa6c47c57
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/13514
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Tested-by: Raghavendra Talur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd/rebalance: initialize defrag variable after glusterd restart

2016-02-25T03:49:27+00:00

During reblance restart after glusterd restarted, we are not
connecting to rebalance process from glusterd, because the
defrag variable in volinfo will be null.

Initializing the variable will connect the rpc

Back port of>
>Change-Id: Id820cad6a3634a9fc976427fbe1c45844d3d4b9b
>BUG: 1303028
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/13319
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Dan Lambright 
>CentOS-regression: Gluster Build System 

(cherry picked from commit a67331f3f79e827ffa4f7a547f6898e12407bbf9)

Change-Id: Ieec82a798da937002e09fb9325c93678a5eefca8
BUG: 1311041
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13494
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

tier:unlink during migration

2016-02-22T14:50:54+00:00

files deleted during promotion were not deleting as the
files are moving from hashed to non-hashed

On deleting a file that is undergoing promotion,
the unlink call is not sent to the dst file as the
hashed subvol == cached subvol. This causes
the file to reappear once the migration is complete.

This patch also fixes a problem with stale linkfile
deleting.

Backport of>
>Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5
>BUG: 1282390
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12829
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

(cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1)

Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d
BUG: 1282388
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12991
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier: allow db queries to be interruptable

2016-02-18T13:54:58+00:00

A query to the database may take a long time if the database
has many entries. The tier daemon also sends IPC calls to the
bricks which can run slowly, espcially in RHEL6. While it is
possible to track down each such instance, the snapshot
feature should not be affected by database operations. It requires
no migration be underway. Therefore it is okay to pause tiering
at any time except when DHT is moving a file.  This fix implements
this strategy by monitoring when control passes to DHT to
migrate a file using the GF_XATTR_FILE_MIGRATE_KEY trigger. If it
is not, the pause operation is successful.

> Change-Id: I21f168b1bd424077ad5f38cf82f794060a1fabf6
> BUG: 1287842
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/13104
> Reviewed-by: Joseph Fernandes
> Tested-by: Gluster Build System 
Signed-off-by: Dan Lambright 

Change-Id: I667e0af24eaa66afefa860c4d73b324e4f39b997
BUG: 1288352
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/13199
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Tier: "tier start force" command implementation

2015-12-23T01:45:00+00:00

	back port of : http://review.gluster.org/#/c/12983/

The start command doesnt restart the tier deamon if the deamon
is running at one node. hence to bring up the tierd on the nodes
where the deamon is down, the force command is implemented.
It skips the check for tierd running.

>Change-Id: I0037d3e5ecfe56637d0da201a97903c435d26436
>BUG: 1292112
>Signed-off-by: hari gowtham 

Change-Id: Idaca442c1a41ded8bf555a6e34eed0ebb9ea4034
BUG: 1293698
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13069
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/glusterd: Only positive values for freq-thresholds

2015-12-17T21:13:58+00:00

Fixed error handling for validation for freq-thresholds

Backport of http://review.gluster.org/12970

> Change-Id: Ibe3a9752ac0b525b0c8c0d6c4b4e4d694bd91b88
> BUG: 1291603
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/12970
> Reviewed-by: Dan Lambright 
> Tested-by: Gluster Build System 
> Tested-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: I044284b5134a685e505a377028bc9a11563b2665
BUG: 1292359
Reviewed-on: http://review.gluster.org/12992
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/glusterd : making new tier detach command throw warning

2015-12-16T17:28:19+00:00

        back port of : http://review.gluster.org/#/c/12883/

For detach tier, the validation was done using the string "detach-tier"
but the new commands used has the string "tier". Making the string use
"tier" to compare, creates problem as the tier status and tier detach
have the keyword "tier". So tier detach and tier status were separated.
and strtok was used to prevent the condition from passing when the
volume name has a substring of "tier". (only the second word from the
string is got and checked if the feature is tier).

Problem: new detach tier command doesnt throw warnings like
"not a tier volume" or " detach tier not started" respectively
instead it prints empty output.

Fix: while validate the volume is checked if its a tiered volume
if yes it is checked if the detach tier is started, else a warning
is thrown respectively.

>Change-Id: I94246d53b18ab0e9406beaf459eaddb7c5b766c2
>BUG: 1288517
>Signed-off-by: hari gowtham 
>Reviewed-on: http://review.gluster.org/12883
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Atin Mukherjee 

Change-Id: I1ac3b6baaec644dbc2025085a7f17abd56ba169d
BUG: 1291970
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/12976
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/libgfdb/sql: Correcting logic in sql query for read

2015-12-05T00:46:51+00:00

Correcting query in the  where clause

from "a & b | c"

to "a & (b | c)"

where "a" is the condition to join the gf_file_tb and gf_flink_tb
through gfids

"b" is the condition for the write heat
and "c" is the condition for read heat

Backport of http://review.gluster.org/12823

> Change-Id: I99226d82b0efb68fbef3a40f02b215bb2b4370d6
> BUG: 1286656
> Signed-off-by: Joseph Fernandes 
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12823
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
Signed-off-by: Joseph Fernandes 

Change-Id: I70533c8589a6cd67e8737e110af0a23abed9013d
BUG: 1287583
Reviewed-on: http://review.gluster.org/12868
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright