glusterfs.git/tests/basic/tier/tier.t, branch v3.7.6

cluster/tier correct promotion cycle calculation

2015-11-07T21:44:38+00:00

This is a backport of 12480

The tier translator should only choose candidate files for promotion
from the most recent cycle, not a multiple of the most recent cycles.
Otherwise user observed behavior can be inconsistent. Remove related
test in tier.t that is subject to race condition.

> Change-Id: I9ad1523cac00f904097ce468efa6ddd515857024
> BUG: 1275524
> Signed-off-by: root 
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12480
> Reviewed-by: Joseph Fernandes
> Tested-by: Gluster Build System 
Signed-off-by: Dan Lambright 

Signed-off-by: Dan Lambright 

Conflicts:
	tests/basic/tier/tier.t
	xlators/cluster/dht/src/tier.c

Change-Id: Ic4587bf1b5d26ba377a12a4ce8e329362988a33b
BUG: 1275483
Reviewed-on: http://review.gluster.org/12536
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tests/tier: Move common functions to tier.rc

2015-11-03T18:44:29+00:00

Move common functions in tier .t files to tier.rc

> Change-Id: Ibc312d987be9d93e7cc7fc47d0bf598bb1c944c2
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/12404
> Tested-by: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 
(cherry picked from commit 594a03b030577bf0ed6960199e920cc5fa7e7afc)

Change-Id: Ie13bb86f85575164e62120e29bbbedb7789b0af5
BUG: 1277590
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12497
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier enable CTR on attach tier

2015-10-31T03:20:45+00:00

CTR is currently disabled by default, and must be manually enabled
for tiering to start. This is an overhead on the administrator and
easy to overlook. Enable it automatically when a tier is attached.

This is a backport of 12420

> Change-Id: I0c29de8762faec1bfe6d1376a57eeef3357ad15a
> BUG: 1274847
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12420
> Tested-by: Gluster Build System 
> Reviewed-by: mohammed rafi  kc 
Signed-off-by: Dan Lambright 

Change-Id: I43a32ac0c88b44d10aa0f40b18b0564ae1e17321
BUG: 1276671
Reviewed-on: http://review.gluster.org/12474
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier: add watermarks and policy driver

2015-10-10T21:01:44+00:00

Backport fix 12039

This fix introduces infrastructure to support different
policies for promotion and demotion.

Currently the tier feature automatically promotes and demotes
files periodically based on access. This is good for testing
but too stringent for most real workloads. It makes it
difficult to fully utilize a hot tier- data will be demoted
before it is touched- its unlikely a 100GB hot SSD will have
all its data touched in a window of time.

A new parameter "mode" allows the user to pick promotion/demotion
polcies.

The "test mode" will be used for *.t and other general testing.
This is the current mechanism.

The "cache mode" introduces watermarks. The watermarks
represent levels of data residing on the hot tier.

"cache mode" policy:

The % the hot tier is full is called P.

Do not promote or demote more than D MB or F files.

A random number [0-100] is called R.

Rules for migration:

if (P < watermark_low) don't demote, always promote.

if (P >= watermark_low) && (P < watermark_hi) demote if R < P; promote if R > P.

if (P > watermark_hi) always demote, don't promote.

gluster volume set {vol} cluster.watermark-hi %
gluster volume set {vol} cluster.watermark-low %
gluster volume set {vol} cluster.tier-max-mb {D}
gluster volume set {vol} cluster.tier-max-files {F}
gluster volume set {vol} cluster.tier-mode {test|cache}

> Change-Id: I157f19667ec95aa1d53406041c1e3b073be127c2
> BUG: 1257911
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12039
> Tested-by: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
Signed-off-by: Dan Lambright 

Signed-off-by: Dan Lambright 

Conflicts:
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/tier.c

Change-Id: Ibfe6b89563ceab98708325cf5d5ab0997c64816c
BUG: 1270527
Reviewed-on: http://review.gluster.org/12330
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier re-enable tier.t in automatic tests

2015-09-30T02:48:22+00:00

This is a backport of 12208

> Re-enable tier.t in automatic tests. Disable check
> for BSD until recurring problem with SQLlite on it is understood.

> Change-Id: Ib13b269ab841a59a0a41d8478c8627b180b16c61
> BUG: 1231268
> Signed-off-by: Dan Lambright 

Change-Id: Id462e9f67cc90c79aa7a230f2baae87c0746ff6f
BUG: 1262335
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12253
Tested-by: Gluster Build System 
Reviewed-by: mohammed rafi  kc

cluster/tier: add gluster v tier

2015-09-10T17:59:11+00:00

This is a backport of 11984.

> Currently the tier feature piggy backs off the rebalance command
> syntax to obtain status and this is clumsy. Introduce a new
> tier command that can do tier specific operations, starting
> with volume status to display counters.

> Old commands:
> gluster volume attach-tier  [replica count] {bricklist..}
> gluster volume detach-tier  {start|stop|commit}

> New commands:
> gluster volume tier  attach [replica count] {bricklist} |
>                           detach {start|stop|commit} |
>                           status

> Change-Id: Ic07b3c6260588162de7d34380f8cbd3d8a7f35d3
> BUG: 1255693
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/11984
> Reviewed-by: Gaurav Kumar Garg 
> Tested-by: Gluster Build System 
Signed-off-by: Dan Lambright 

Change-Id: Id45bd0fa6b8606dd47863de83a694908da393229
BUG: 1261664
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/12143
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg

tiering/rebalance: tier daemon stopped with out updating status

2015-06-26T15:36:44+00:00

When a subvol goes down, tier daemon stopped immediately, and
the status shows as "Progressing".

With this change, with respect to tier xlator, when a subvol
goes offline it will update the status as failed.

Back port of:
>Change-Id: I9f722ed0d35cda8c7fc1a7e75af52222e2d0fdb7
>BUG: 1227803
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/11068
>Tested-by: NetBSD Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

(cherry picked from commit d3714f252d91f4d1d5df05c4dcc8bc7c2ee75326)


Change-Id: I268b7e2631779d15db5d9aec495dfd00ded94c5c
BUG: 1235203
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/11414
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/volume set: Validate volume set option for tier

2015-06-17T05:13:18+00:00

Volume set option related to tier volume can only be set
for tier volume, also currently all volume set i for tier
option accepts a non-negative integer. This patch validate
both condition.

Back port of:
>Change-Id: I3611af048ff4ab193544058cace8db205ea92336
>BUG: 1216960
>Signed-off-by: Mohammed Rafi KC 
>Signed-off-by: Dan Lambright 
>Reviewed-on: http://review.gluster.org/10751
>Tested-by: Gluster Build System 
>Tested-by: NetBSD Build System 
>Reviewed-by: Joseph Fernandes


(cherry picked from commit f6a062044a3447bea5bf0fcf21a3f85c00fb6c7d)

Change-Id: Ic6081f0ce7ae7effac69ba192bd35c8d382a11d5
BUG: 1230560
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/11173
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes

tiering/nfs: duplication of nodes in client graph

2015-06-01T04:40:47+00:00

        Back port of http://review.gluster.org/10820

When creating client volfiles, xlator tier-dht will
be loaded for each volume. So for services like nfs
have one or more volumes . So for each volume in the
graph a tier-dht xlator will be created. So the graph
parser will fail because of the redundant node in
graph.

By this change tier-dht will be renamed as volname-tier-dht

 >Change-Id: I3c9b9c23ddcb853773a8a02be7fd8a5d09a7f972
 >BUG: 1222840
 >Signed-off-by: Mohammed Rafi KC 
 >Reviewed-on: http://review.gluster.org/10820
 >Reviewed-by: Atin Mukherjee 
 >Tested-by: Gluster Build System 
 >Tested-by: NetBSD Build System
 >Reviewed-by: Kaushal M 

Change-Id: I5629d48d4d1dbec8790f75e2fee66729aa2f6eed
BUG: 1226029
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10981
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Joseph Fernandes
Reviewed-by: Kaushal M

tiering: Send both attach-tier and tier-start together

2015-05-10T04:51:36+00:00

        Back port of http://review.gluster.org/10363

After attaching tier, we have to start tier rebalance process.
This patch is to trigger tier start along with attch-tier.

>Change-Id: I39380f95123f0087a82213ef263f9f33adcc5adc
>BUG: 1214222
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/10363
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 

Change-Id: Ia27e97a9dc0772f3f56b991ce4019281ab7bf1b5
BUG: 1219848
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10711
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur