glusterfs.git/tests, branch v3.7.0

features/bit-rot-stub: versioning of objects in write/truncate fop instead of open

2015-05-10T15:14:33+00:00

* This patch brings in the changes where object versioning is done in write and
  truncate fops instead of tracking them in open and create fops. This model
  works for both regular and anonymous fds. It also removes the race associated
  with open calls, create and lookups.

  This patch follows the below method for object versioning and notifications:

  Before sending writev on the fd, increase the ongoing
  version first. This makes anonymous fd write similar to the regular
  fd write by having the ongoing version increased before doing the
  write.

  Do following steps to do versioning:
  1) For anonymous fds set the fd context (so that release is invoked) and add
     the fd context to the list maintained in the inode context.
     For regular fds the above think would have been done in open itself.
  2) Increase the on-disk ongoing version
  3) Increase the in memory ongoing version and mark inode as non-dirty
  3) Once versioning is successfully done send write operation. If
     versioning fails, then fail the write fop.
  5) In writev_cbk mark inode as modified.

> Change-Id: I7104391bbe076d8fc49b68745d2ec29a6e92476c
> BUG: 1207979
> Signed-off-by: Raghavendra Bhat 
> Reviewed-on: http://review.gluster.org/10233
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I4bb86989b5fab02b9ed2950798b1a80e566f1024
BUG: 1220041
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/10722
Reviewed-by: Gaurav Kumar Garg 
Tested-by: NetBSD Build System
Tested-by: Gluster Build System

features/bitrot: Follow xattr naming conventions

2015-05-10T12:28:43+00:00

Instead of "trusted.glusterfs.bit-rot.*" use "trusted.bit-rot.*"

NOTE:
With this patch, data on existing volumes would be resigned
(which should be OK as of now since we do not expect many
users as of now :-))

> Change-Id: I926c7bca266a9c8f2cb35d57c4d0359aa5cecfa0
> BUG: 1170075
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10181
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I3c18d7dc2db4beaca6e8d8d231b4171a7b18795f
Signed-off-by: Venky Shankar 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10718
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg

bitrot: Volfile generation should not proceed if node doesn't have any brick.

2015-05-10T08:56:37+00:00

glusterd crashes when bitrot is enabled on a distributed volume from a node
which doesn't host a brick.

While generating volfile glusterd should check number of brick on that node. If
node doesn't have any brick then graph generation for bitrot and scrubber should
not proceed further.

Change-Id: I2158113e20e93738cde2a22fd73f0ae6b22aae9e
BUG: 1219785
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/10680
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

dht: make lookup-unhashed=auto do something actually useful

2015-05-10T04:55:09+00:00

The key concept here is to determine whether a directory is "clean" by
comparing its last-known-good topology to the current one for the
volume.  These are stored as "commit hashes" on the directory and the
volume root respectively.  The volume's commit hash changes whenever a
brick is added or removed, and a fix-layout is done.  A directory's
commit hash changes only when a full rebalance (not just fix-layout)
is done on it.  If all bricks are present and have a directory
commit hash that matches the volume commit hash, then we can assume
that every file is in its "proper" place. Therefore, if we look for
a file in that proper place and don't find it, we can assume it's not
on any other subvolume and *safely* skip the global (broadcast to all)
lookup.

Change-Id: Id6ce4593ba1f7daffa74cfab591cb45960629ae3
BUG: 1220064
Reviewed-on-master: http://review.gluster.org/#/c/7702/
Signed-off-by: Jeff Darcy 
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/10729
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Vijay Bellur

tiering: Send both attach-tier and tier-start together

2015-05-10T04:51:36+00:00

        Back port of http://review.gluster.org/10363

After attaching tier, we have to start tier rebalance process.
This patch is to trigger tier start along with attch-tier.

>Change-Id: I39380f95123f0087a82213ef263f9f33adcc5adc
>BUG: 1214222
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/10363
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 

Change-Id: Ia27e97a9dc0772f3f56b991ce4019281ab7bf1b5
BUG: 1219848
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10711
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

tiering/cli: Check replica count and bricks are proper or not

2015-05-10T03:25:13+00:00

        Back port of http://review.gluster.org/10428

Right now, attach-tier calls parsing function for add-brick.
Add-brick does not have any check for brick count and replca
count compatibility.

>Change-Id: I44ec13eadffc003a9ebf8c4eb0193df559933a68
>BUG: 1215122
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/10428
>Reviewed-by: Dan Lambright 
>Tested-by: Gluster Build System 
>Reviewed-by: Vijay Bellur 

Change-Id: Ie86d5b61b044386a062bfc9782178853e3af0f11
BUG: 1219850
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10710
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System

tests: fix bitrot spurious failures

2015-05-09T21:50:15+00:00

Change-Id: I55bd62480b7ee38cf7b29aeba67b19b0c5bbe2fb
BUG: 1220016
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/10702
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/10705
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg

cli/tiering: Enhance cli output for tiering

2015-05-09T17:26:36+00:00

        back port of http://review.gluster.org/10284

Fix for handling cli output for attach-tier and
detach-tier

>Change-Id: I4d17f4b09612754fe1b8cec6c2e14927029b9678
>BUG: 1211562
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/10284
>Reviewed-by: Dan Lambright 
>Tested-by: Gluster Build System 
>Tested-by: NetBSD Build System
>Reviewed-by: Vijay Bellur 

Change-Id: Ic30c8f0a104d89bf98f5d0069937a34674ee3f2d
BUG: 1220052
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10713
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: support for tier volumes 'detach start' and 'detach commit'

2015-05-09T15:35:14+00:00

        Back port of http://review.gluster.org/10108

These commands work in a manner analagous to rebalancing when removing a
brick. The existing migration daemon detects "detach start" and switches
to moving data off the hot tier. While in this state all lookups are
directed to the cold tier.

gluster v detach-tier  start
gluster v detach-tier  commit

The status and stop cli commands shall be submitted separately.

>Change-Id: I24fda5cc3ba74f5fb8aa9a3234ad51f18b80a8a0
>BUG: 1205540
>Signed-off-by: Dan Lambright 
>Signed-off-by: root 
>Signed-off-by: Dan Lambright 
>Reviewed-on: http://review.gluster.org/10108
>Reviewed-by: Kaleb KEITHLEY 

Change-Id: I212d748d077fb5870ee84b316c653acbafbea3f7
BUG: 1220047
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10708
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

tests : workaround fix for volume-snapshot-clone spurious failure

2015-05-09T09:16:49+00:00

Backport of http://review.gluster.org/#/c/10619/

Snippet of the existing test:

TEST kill_glusterd 2;
TEST $glusterd_2;
sleep 5
EXPECT_WITHIN $PROCESS_UP_TIMEOUT 'Started' volinfo_field ${V0}_clone 'Status';
EXPECT_WITHIN $PROCESS_UP_TIMEOUT 'Started' volinfo_field ${V1}_clone 'Status';
stop_force_volumes 2

The above EXPECT_WITHINs will not wait as the volumes were never stopped which
results into immediate trigger of volume stop where handshaking is not
completed. A workaround would be to check whether handshaking is completed and
then proceed.

Change-Id: Id3468b56f801212fc8e19ff196bc99e5e2358471
BUG: 1220012
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/10619
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Justin Clift 
Reviewed-by: Niels de Vos 
(cherry picked from commit 0950e6c29fc51ddcc5bb7e1e4187d54ff7c171d2)
Reviewed-on: http://review.gluster.org/10700
Reviewed-by: Vijay Bellur