glusterfs.git/tests/basic/tier, branch v3.10.0alpha1

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

tier/cli : printing a warning instead of skipping the node

2016-05-20T14:25:37+00:00

Problem: skipping the status of the nodes down creates confusion
to the user as one might see the status as completed for all nodes
and while performing detach commit, the operation will fail as the
node is down

Fix: Display a warning message

Note: When the last node is down (as per the peer list) then
warning message can't be displayed as the total number of peers
participating in the transaction is considered to be the total count.

Change-Id: Ib7afbd1b26df3378e4d537db06f41f5c105ad86e
BUG: 1324439
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/14347
Tested-by: hari gowtham 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
NetBSD-regression: NetBSD Build System

Tier/glusterd: Resetting the tier status value to not started

2016-05-05T14:45:54+00:00

Problem: during a volume restart or tier start force, the
value of tier status is set as started irrespective of the result.

Fix: The appropriate value of status is set during the restart of
rebalance function.

Change-Id: I6164f0add48542a57dee059e80fa0f9bb036dbef
BUG: 1315666
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/14106
Tested-by: mohammed rafi  kc 
Tested-by: hari gowtham 
Smoke: Gluster Build System 
Reviewed-by: mohammed rafi  kc 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

Tier: tier command fails message when any node is down

2016-04-19T10:13:17+00:00

PROBLEM: the dict doesn't get set on the node if its down.
so while printing the output on cli we get a ENOENT
which ends in a tier command failed.

FIX: this patch skips the node that wasn't available
and carrys on with the next node for both tier status
and tier detach status.

Change-Id: I718a034b18b109748ec67f3ace56540c50650d23
BUG: 1324439
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13918
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

tier/dht : Attach tier fix layout to run in background

2016-03-28T16:17:59+00:00

1. Spawn a thread for background fix-layout for tier process.

2. Once the fix-layout is completed a marker xttr is set on the root of
volume to mark the completion of the background fixlayout, so that
even if the tier process is spawned again, fixlayout will not be
issued, if it was completed last time.

3. Please note that promotion of legacy files will happen eventually as
the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy
files OR the ctr lookup heal happend due to a name lookup.

4. When a detach tier is successful in evacuation data from hot tier, we remove
the marker xattr is removed. So that next attach tier runs the background
tier fixlayout.

what is remaining ?
1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start
clear it during detach commit. But the issue is detach commit is a glusterd operation
and the volume is not mounted in glusterd.
The reason we want to do it in detach commit is that if the admin wants to attach the
same tier again, then a background fixlayout will be triggered, which would not be needed.
2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having
entries which will be stale when the volume is used, with ctr off (ctr is switched off only when
we have detach commit.)

Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5
BUG: 1313228
Signed-off-by: Joseph Fernandes
Reviewed-on: http://review.gluster.org/13491
Smoke: Gluster Build System
Tested-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System
CentOS-regression: Gluster Build System
Reviewed-by: Dan Lambright

cluster/tier: add tunable to migrate files based on size

2016-03-17T06:12:04+00:00

This fix adds a paramater "tier-max_promote_size" to control wether
a file is migrated or not based on its size. By default the value
is 0, meaning all files are migrated. If set to a non-zero
value, files larger than the parameter won't be moved
in tiered volumes.

Change-Id: Ia6b88e9b2508935bef500d956f9192e59670fe00
BUG: 1313495
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/13570
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Joseph Fernandes

tests: Add mechanism for disabled tests

2016-03-10T07:19:40+00:00

Requirements:
Should be able to skip tests from run-tests.sh run.
Should be granular enough to disable on subset of OSes.

Solution:
Tests can have special comment lines with some comma separated values
within them.

Key names used to determine test status are
G_TESTDEF_TEST_STATUS_CENTOS6
G_TESTDEF_TEST_STATUS_NETBSD7
Some examples:
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
You can change status of test to enabled or delete the line only if all the
bugs are closed or modified or if the patch fixes it.

Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
BUG: 1295704
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/13393
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Kaushal M

Tier: making detach start fail when brick on hot tier is down

2016-03-03T11:51:14+00:00

Currently detach tier start happens even when a hot brick is down
this might lead to data loss.

This patch prevents the detach tier start from being executed
successfully if a brick in hot tier is down

Change-Id: I3b6047a44bd01b8a6887d41f799f64de6bf075ef
BUG: 1309999
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13474
Smoke: Gluster Build System 
Tested-by: hari gowtham 
Reviewed-by: Gaurav Kumar Garg 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

tests/glusterd: Parse the new time format

2016-02-26T01:43:52+00:00

A recent change in cli changed elapsed time format
that broke a test.

This patch will fix the issue with parsing.

Change-Id: I9a4a4b28f654cf2ac223e25abfc9df6570607d74
BUG: 1312036
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13524
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Raghavendra Talur

glusterd/rebalance: initialize defrag variable after glusterd restart

2016-02-23T05:42:03+00:00

During reblance restart after glusterd restarted, we are not
connecting to rebalance process from glusterd, because the
defrag variable in volinfo will be null.

Initializing the variable will connect the rpc

Change-Id: Id820cad6a3634a9fc976427fbe1c45844d3d4b9b
BUG: 1303028
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13319
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
CentOS-regression: Gluster Build System