glusterfs.git/tests/basic/tier, branch v3.7.7

Tier: "tier start force" command implementation

2015-12-23T01:45:00+00:00

	back port of : http://review.gluster.org/#/c/12983/

The start command doesnt restart the tier deamon if the deamon
is running at one node. hence to bring up the tierd on the nodes
where the deamon is down, the force command is implemented.
It skips the check for tierd running.

>Change-Id: I0037d3e5ecfe56637d0da201a97903c435d26436
>BUG: 1292112
>Signed-off-by: hari gowtham 

Change-Id: Idaca442c1a41ded8bf555a6e34eed0ebb9ea4034
BUG: 1293698
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13069
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/glusterd: Only positive values for freq-thresholds

2015-12-17T21:13:58+00:00

Fixed error handling for validation for freq-thresholds

Backport of http://review.gluster.org/12970

> Change-Id: Ibe3a9752ac0b525b0c8c0d6c4b4e4d694bd91b88
> BUG: 1291603
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/12970
> Reviewed-by: Dan Lambright 
> Tested-by: Gluster Build System 
> Tested-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: I044284b5134a685e505a377028bc9a11563b2665
BUG: 1292359
Reviewed-on: http://review.gluster.org/12992
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/glusterd : making new tier detach command throw warning

2015-12-16T17:28:19+00:00

        back port of : http://review.gluster.org/#/c/12883/

For detach tier, the validation was done using the string "detach-tier"
but the new commands used has the string "tier". Making the string use
"tier" to compare, creates problem as the tier status and tier detach
have the keyword "tier". So tier detach and tier status were separated.
and strtok was used to prevent the condition from passing when the
volume name has a substring of "tier". (only the second word from the
string is got and checked if the feature is tier).

Problem: new detach tier command doesnt throw warnings like
"not a tier volume" or " detach tier not started" respectively
instead it prints empty output.

Fix: while validate the volume is checked if its a tiered volume
if yes it is checked if the detach tier is started, else a warning
is thrown respectively.

>Change-Id: I94246d53b18ab0e9406beaf459eaddb7c5b766c2
>BUG: 1288517
>Signed-off-by: hari gowtham 
>Reviewed-on: http://review.gluster.org/12883
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Atin Mukherjee 

Change-Id: I1ac3b6baaec644dbc2025085a7f17abd56ba169d
BUG: 1291970
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/12976
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/libgfdb/sql: Correcting logic in sql query for read

2015-12-05T00:46:51+00:00

Correcting query in the  where clause

from "a & b | c"

to "a & (b | c)"

where "a" is the condition to join the gf_file_tb and gf_flink_tb
through gfids

"b" is the condition for the write heat
and "c" is the condition for read heat

Backport of http://review.gluster.org/12823

> Change-Id: I99226d82b0efb68fbef3a40f02b215bb2b4370d6
> BUG: 1286656
> Signed-off-by: Joseph Fernandes 
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12823
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
Signed-off-by: Joseph Fernandes 

Change-Id: I70533c8589a6cd67e8737e110af0a23abed9013d
BUG: 1287583
Reviewed-on: http://review.gluster.org/12868
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier: fix loading tier.so into glusterd

2015-12-04T13:44:32+00:00

The glusterd process loads the shared libraries of client translators.
This failed for tiering due to a reference to dht_methods which is
defined as a global variable which is not necessary.
The global variable has been removed and this is now a member of
dht_conf and is now initialised in the *_init calls.

> Change-Id: Ifa0a21e3962b5cd8d9b927ef1d087d3b25312953
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/12863
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 
(cherry picked from commit 96fc7f64da2ef09e82845a7ab97574f511a9aae5)

Change-Id: If3cc908ebfcd1f165504f15db2e3079d97f3132e
BUG: 1288352
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12877
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/glusterd : Validation for frequency thresholds and record-counters

2015-12-02T19:47:17+00:00

1) if record-counters is set to off
   check if both the frequency thresholds are non-zero, then pop
   an error message, with volume set failed.
2) if record-counters is set to on
   check if both the frequency thresholds are zero, then pop
   an note, but volume set is not failed.
3) If any of the frequency thresholds are set to a non-zero value,
   switch record-counters on, if not already on
4) If both the frequency thresholds are set to zero,
   switch record-counters off, if not already off

NOTE: In this fix we have
1) removed unnecessary ctr vol set options.
2) changed  ctr_hardlink_heal_expire_period to ctr_lookupheal_link_timeout

Backport of  http://review.gluster.org/12780

> Change-Id: Ie7ccfd3f6e021056905a79de5a3d8f199312f315
> BUG: 1286346
> Signed-off-by: Joseph Fernandes 
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12780
> Tested-by: Gluster Build System 
> Tested-by: NetBSD Build System 
> Signed-off-by: Joseph Fernandes 

Change-Id: I35621ed3cc1ad20a75ef16e0b95453b8c6120852
BUG: 1287560
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12849
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/ec: Create copy of dict for setting internal xattrs

2015-12-01T17:21:47+00:00

           Backport of http://review.gluster.com/12831

Problem:
Ec takes a ref of the request xdata and sets trusted.ec.version/algo etc xattrs
as part of it. But this request xdata could be using same dictionary to do the
operation on multiple subvolumes, due to which other subvolumes will have
internal xattrs of ec in it and will be created on subvols where they are not
supposed to appear.

Fix:
Take a copy of the request xdata/dict to prevent this from happening.

Most of the debugging work and test script is contributed by Nitya.

BUG: 1286985
Change-Id: Ie9b7d9f063434789f6c5902c3a68ececdc3c7efa
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12835
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

tier/ctr: Correcting rename logic

2015-11-26T17:19:35+00:00

Problem: When a file with old_file_name and GFID_1 is renamed with a new_file_name
which already exists and with GFID_2, this is what happens in linux internaly.
   a. "new_file_name" is unlinked for GFID_2
   b. a hardlink "new_file_name" is created to GFID_1
   c. "old_file_name" hardlink is unlinked for GFID_2.
Well this is all internal to linux, and gluster just issues a rename system call
at POSIX layer. But CTR Xlator doesn't delete the entries corresponding to the
"new_file_name" and GFID_2. Thus leaving the stale entry in the DB.
The following are the implications.
a. Promotion are tried on these stale entries which will fail and show
false results in the status of migration,
b. GFID_2 Files with 2 hardlinks, which will have only one hardlink
after the rename will not be promoted or demoted as the DB shows 2 entries.

Solution: Delete the older database entry for the replaced hardlink

Backport of http://review.gluster.org/12711

> Change-Id: I4eafa0872253e29ff1f0bec4283bcfc579ecf0e2
> BUG: 1284090
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/12711
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 

Change-Id: Ic35348303ec21f9bd19f20a48f3141449349668b
BUG: 1285688
Reviewed-on: http://review.gluster.org/12762
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

glusterd: copy snapshot object during duplication of volfile

2015-11-26T11:41:36+00:00

Back port of > http://review.gluster.org/#/c/12734/

When creating duplicate volfile for hot/cold tier, we need to
copy the snapshot object in to volfile as it requires to generate
snapshot brick volfile.

>Change-Id: I39ccfa20cd1c16ef2801901e3cd3a31c76f8995d
>BUG: 1284789
>Signed-off-by: Mohammed Rafi KC 

Change-Id: Ia0892dfc3af24ee428e0aa0a3e23063a91049a57
BUG: 1285629
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12756
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Avra Sengupta 
Reviewed-by: Rajesh Joseph

cluster/tier: readdirp to cold tier only

2015-11-24T12:13:53+00:00

It is possible a file would get migrated in the middle
of a readdir operation. If there are four subvolumes A,B,C,D,
and if readdir reads them in order and reaches subvol B,
then, if a file is moved from D to A, it will not be included
in the readdir output.

This phenonema has pre-existed in DHT migration but is more
apparent in tiering.

When a file is moved off the hashed subvolume a T file is created.
For tiering, we will make the cold subvolume the hashed subvolume.
This will ensure the creation of a T file. Readdir will not skip T
files in the tier translator.

Making the cold subvolume the hashed subvolume ensures the T
files created on promotions or creates will be less likely to
fill the volume.

Creates still put the data on the hot subvolume.

This is a backport of 12530

> Change-Id: Ifde557d3d0e94a4570ca9f115adee3db2ee75407
> BUG:  1281598
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12530
> Tested-by: Gluster Build System 
> Tested-by: NetBSD Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 
Signed-off-by: Dan Lambright 

Signed-off-by: Dan Lambright 

Conflicts:
	xlators/cluster/dht/src/tier.c

Change-Id: I5720a4cd04ae5088e5d7d23439b0f90d6bbc6265
BUG: 1283923
Reviewed-on: http://review.gluster.org/12722
Tested-by: NetBSD Build System 
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright