glusterfs.git/xlators/cluster/dht, branch v3.7.2

cluster/dht: Prevent use after free bug

2015-06-19T05:16:25+00:00

        Backport of http://review.gluster.org/11209

BUG: 1233042
Change-Id: If3685c9ed84a6720d8696d11773005e9786b503f
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11305
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

tier/volume set: Validate volume set option for tier

2015-06-17T05:13:18+00:00

Volume set option related to tier volume can only be set
for tier volume, also currently all volume set i for tier
option accepts a non-negative integer. This patch validate
both condition.

Back port of:
>Change-Id: I3611af048ff4ab193544058cace8db205ea92336
>BUG: 1216960
>Signed-off-by: Mohammed Rafi KC 
>Signed-off-by: Dan Lambright 
>Reviewed-on: http://review.gluster.org/10751
>Tested-by: Gluster Build System 
>Tested-by: NetBSD Build System 
>Reviewed-by: Joseph Fernandes


(cherry picked from commit f6a062044a3447bea5bf0fcf21a3f85c00fb6c7d)

Change-Id: Ic6081f0ce7ae7effac69ba192bd35c8d382a11d5
BUG: 1230560
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/11173
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes

features/changelog: Avoid setattr fop logging during rename

2015-06-12T05:26:16+00:00

Problem:
        When a file is renamed and the (renamed)file's Hashing
falls into a different brick, DHT creates a special file(linkto file)
in the brick(Hashed subvolume) and carries out setattr operation
on that file.

Currently, Changelog records this(setattr) operation in Hashed
subvolume. glusterfind in turn records this operation
as MODIFY operation.

So, there is a NEW entry in Cached subvolume and MODIFY entry
in Hashed subvolume for the same file.

Solution:
        Avoid logging setattr operation carried out, by
marking the operation as internal fop using xdata.

In changelog translator, check whether setattr is set
as internal fop and skip accordingly.

Change-Id: I21b09afb5a638b88a4ccb822442216680b7b74fd
BUG: 1230687
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/11183
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

dht: Add lookup-optimize configuration option for DHT

2015-06-05T09:16:13+00:00

Currently with commit 4eaaf5 a mixed version cluster would
have issues if lookup-uhashed is set to auto, as older clients
would fail to validate the layouts if newer clients (i.e 3.7 or
upwards) create directories. Also, in a mixed version cluster
rebalance daemon would set commit hash for some subvolumes and
not for the others.

This commit fixes this problem by moving the enabling of the
functionality introduced in the above mentioned commit to a
new dht option. This option also has a op_version of 3_7_1
thereby preventing it from being set in a mixed version
cluster. It brings in the following changes,
- Option can be set only if min version of the cluster is
3.7.1 or more
- Rebalance and mkdir update the layout with the commit hashes
only if this option is set, hence ensuring rebalance works in a
mixed version cluster, and also directories created by newer
clients do not cause layout errors when read by older clients
- This option also supersedes lookup-unhased, to enable the
optimization for lookups more deterministic and not conflict
with lookup-unhashed settings.

Option added is cluster.lookup-optimize, which is a boolean.

Usage: # gluster volume set VOLNAME cluster.lookup-optimize on

Change-Id: Ifd1d4ce3f6438fcbcd60ffbfdbfb647355ea1ae0
BUG:  1225940
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/10976
Tested-by: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

cluster/dht: fix incorrect dst subvol info in inode_ctx

2015-06-04T16:44:24+00:00

Stashing additional information in the inode_ctx to help
decide whether the migration information is stale, which could
happen if a file was migrated several times but FOPs only detected
the P1 migration phase. If no FOP detects the P2 phase, the inode
ctx1 is never reset.
We now save the src subvol as well as the dst subvol in the
inode ctx. The src subvol is the subvol on which the FOP was sent
when the mig info was set in the inode ctx. This information is
considered stale if:
1. The subvol on which the current FOP is sent is the same as
the dst subvol in the ctx
2. The subvol on which the current FOP is sent is not the same
as the src subvol in the ctx

This does not handle the case where the same file might have been
renamed such that the src subvol is the same but the dst subvol
is different. However, that is unlikely to happen very often.

Change-Id: I05a2e9b107ee64750c7ca629aee03b03a02ef75f
BUG: 1225809
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/10967
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G 
Tested-by: NetBSD Build System

dht/rebalance : Fixed rebalance failure

2015-06-04T12:23:04+00:00

The rebalance process determines the local subvols for the
node it is running on and only acts on files in those subvols.
If a dist-rep or dist-disperse volume is created on 2 nodes by
dividing the bricks equally across the nodes, one process might
determine it has no local_subvols.

When trying to update the commit hash, the function attempts to
lock all local subvols. On the node with no local_subvols the dht
inode lock operation fails, in turn causing the rebalance to fail.

In a dist-rep volume with 2 nodes, if brick 0 of each replica
set is on node1 and brick 1 is on node2, node2 will find that it has
no local subvols.

Change-Id: I7d73b5b4bf1c822eae6df2e6f79bd6a1606f4d1c
BUG:  1221656
Signed-off-by: Nithya Balachandran 
Reviewed-on-master: http://review.gluster.org/10786
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Susant Palai 
Reviewed-on: http://review.gluster.org/10788
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

cluster/dht: Fix dht_setxattr to follow files under migration

2015-06-04T12:03:59+00:00

If a file is under migration, then any xattrs created on it
are lost post migration of the file. This is because
the xattrs are set only on the cached subvol of the source
and as the source is under migration, it becomes a linkto file
post migration.

Change-Id: Ib8e233b519cf954e7723c6e26b38fa8f9b8c85c0
BUG: 1225839
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/10968
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

cluster/dht: pass a destination subvol to fop2 variants to avoid races.

2015-06-04T08:34:27+00:00

The destination subvol used in the fop2 variants is either stored in
inode-ctx1 or local->cached_subvol. However, it is not guaranteed that
a value stored in these locations before invocation of fop2 is still
present after the invocation as these locations are shared among
different concurrent operations. So, to preserve the atomicity of
"check dst-subvol and invoke fop2 variant if dst-subvol found", we
pass down the dst-subvol to fop2 variant.

This patch also fixes error handling in some fop2 variants.

Change-Id: Icc226228a246d3f223e3463519736c4495b364d2
BUG: 1225809
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/10966
Tested-by: Gluster Build System

cluster/dht: Don't rely on linkto xattr to find destination subvol

2015-06-04T05:19:11+00:00

during phase 2 of migration.

linkto xattr on source file cannot be relied to find where the data
file currently resides. This can happen if there are multiple
migrations before phase 2 detection by a client. For eg.,

* migration (M1, node1, node2) starts.
* application writes some data. DHT correctly stores the state in
  inode context that phase-1 of migration is in progress
* migration M1 completes
* migration (M2, node2, node3) is triggered and completed
* application resumes writes to the file. DHT identifies it as phase-2
  of migration. However, linkto xattr on node1 points to node2, but
  the file is on node3. A lookup correctly identifies node3 as cached
  subvol

TBD:
   When we identify phase-2 of a previous migration (say M1), there
   might be a migration in progress - say (M3, node3, node4). In this
   case we need to send writes to both (node3, node4) not just
   node3. Also, the inode state needs to correctly indicate that its in
   phase-1 of migration. I'll send this as a different patch.

Change-Id: I1a861f766258170af2f6c0935468edb6be687b95
BUG: 1225809
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/10805
Reviewed-on: http://review.gluster.org/10965
Tested-by: Gluster Build System

dht/rebalance: Change log_level to DEBUG

2015-06-01T09:27:35+00:00

BUG: 1221503
Change-Id: I4ac87bb69e05e2dd445fc0a5bcf48d5bd3d0020b
Reviewed-on: http://review.gluster.org/10756
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/10778