glusterfs.git/tests/basic/tier/tier.t, branch v8.0

tiering: remove the translator from build and glusterd

2018-11-02T02:39:35+00:00

Based on the proposal to remove few features as they are not
actively maintained [1], removing tier translator from the
build. Also make sure there are no regression tests involving
tiering feature are present.

[1] https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html

Change-Id: I2c177f711f9b54b7b24e1a13525ff3132bd9a9c5
updates: bz#1642807
Signed-off-by: Amar Tumballi

Tier: Watermark check for hi and low value being equal

2017-05-08T03:58:18+00:00

Problem: Both low and hi watermark can be set to same value
as the check missed the case for being equal.

Fix: Add the check to both the hi and low values being equal
along with the low value being higher than hi value.

Change-Id: Ia235163aeefdcb2a059e2e58a5cfd8fb7f1a4c64
BUG: 1447960
Signed-off-by: hari gowtham 
Reviewed-on: https://review.gluster.org/17175
Smoke: Gluster Build System 
Tested-by: hari gowtham 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Milind Changire 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

TIER: watermark check during low watermark reset

2017-03-18T22:19:45+00:00

PROBLEM: during a low watermark reset, checking of whether
the low watermark is lower than hi watermark is not done.

FIX: This patch checks if the hi watermark value is higher
the default low watermark. Else throws an failure of the reset
command

Change-Id: I8b49090c6bccce6d45c2e8076ab766047a2a6162
BUG: 1328342
Signed-off-by: hari gowtham 
Reviewed-on: https://review.gluster.org/14028
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

tier/dht : Attach tier fix layout to run in background

2016-03-28T16:17:59+00:00

1. Spawn a thread for background fix-layout for tier process.

2. Once the fix-layout is completed a marker xttr is set on the root of
volume to mark the completion of the background fixlayout, so that
even if the tier process is spawned again, fixlayout will not be
issued, if it was completed last time.

3. Please note that promotion of legacy files will happen eventually as
the ctr lookup heal in the fixlayout slowly heals the ctr db for legacy
files OR the ctr lookup heal happend due to a name lookup.

4. When a detach tier is successful in evacuation data from hot tier, we remove
the marker xattr is removed. So that next attach tier runs the background
tier fixlayout.

what is remaining ?
1. Instead of clearing the marker xattr of tiering fix layout at the end of detach start
clear it during detach commit. But the issue is detach commit is a glusterd operation
and the volume is not mounted in glusterd.
The reason we want to do it in detach commit is that if the admin wants to attach the
same tier again, then a background fixlayout will be triggered, which would not be needed.
2. Clearing the CTR DB of the cold bricks when there is a detach commit, as it will be having
entries which will be stale when the volume is used, with ctr off (ctr is switched off only when
we have detach commit.)

Change-Id: Ibe343572e95865325cd0eef4d0b976b626a3c0c5
BUG: 1313228
Signed-off-by: Joseph Fernandes
Reviewed-on: http://review.gluster.org/13491
Smoke: Gluster Build System
Tested-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System
CentOS-regression: Gluster Build System
Reviewed-by: Dan Lambright

cluster/tier: add tunable to migrate files based on size

2016-03-17T06:12:04+00:00

This fix adds a paramater "tier-max_promote_size" to control wether
a file is migrated or not based on its size. By default the value
is 0, meaning all files are migrated. If set to a non-zero
value, files larger than the parameter won't be moved
in tiered volumes.

Change-Id: Ia6b88e9b2508935bef500d956f9192e59670fe00
BUG: 1313495
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/13570
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Joseph Fernandes

tests: Add mechanism for disabled tests

2016-03-10T07:19:40+00:00

Requirements:
Should be able to skip tests from run-tests.sh run.
Should be granular enough to disable on subset of OSes.

Solution:
Tests can have special comment lines with some comma separated values
within them.

Key names used to determine test status are
G_TESTDEF_TEST_STATUS_CENTOS6
G_TESTDEF_TEST_STATUS_NETBSD7
Some examples:
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456
G_TESTDEF_TEST_STATUS_NETBSD7=KNOWN_ISSUE,BUG=4444444
G_TESTDEF_TEST_STATUS_CENTOS6=BAD_TEST,BUG=123456;555555
You can change status of test to enabled or delete the line only if all the
bugs are closed or modified or if the patch fixes it.

Change-Id: Idee21fecaa5837fd4bd06e613f5c07a024f7b0c2
BUG: 1295704
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/13393
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Kaushal M

glusterd/rebalance: initialize defrag variable after glusterd restart

2016-02-23T05:42:03+00:00

During reblance restart after glusterd restarted, we are not
connecting to rebalance process from glusterd, because the
defrag variable in volinfo will be null.

Initializing the variable will connect the rpc

Change-Id: Id820cad6a3634a9fc976427fbe1c45844d3d4b9b
BUG: 1303028
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13319
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
CentOS-regression: Gluster Build System

tier/glusterd: Only positive values for freq-thresholds

2015-12-16T14:45:26+00:00

Fixed error handling for validation for freq-thresholds

Change-Id: Ibe3a9752ac0b525b0c8c0d6c4b4e4d694bd91b88
BUG: 1291603
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12970
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Tested-by: Dan Lambright

cluster/tier correct promotion cycle calculation

2015-11-07T18:30:22+00:00

The tier translator should only choose candidate files for promotion
from the most recent cycle, not a multiple of the most recent cycles.
Otherwise user observed behavior can be inconsistent. Remove related
test in tier.t that is subject to race condition.

Change-Id: I9ad1523cac00f904097ce468efa6ddd515857024
BUG: 1275524
Signed-off-by: root 
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12480
Reviewed-by: Joseph Fernandes
Tested-by: Gluster Build System