glusterfs.git/xlators/cluster, branch v3.8.8

afr: Ignore event_generation checks post inode refresh for write txns

2017-01-08T11:15:27+00:00

Backport of http://review.gluster.org/#/c/16205/

Before http://review.gluster.org/#/c/16091/, after inode refresh, we
failed read txns in case of EIO or event_generation being zero. For
write transactions, the check was only for EIO. 16091 re-factored the
code to fail both read and write when event_generation=0. This seems to
have caused a regression as explained in the BZ.

This patch restores that behaviour in afr_txn_refresh_done().

Change-Id: Id763ed2d420b6d045d4505893a18959d998c91a3
BUG: 1378547
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16322
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

dht/rebalance: remove errno check for failure detection

2017-01-08T11:03:50+00:00

> BUG: 1410355
> Change-Id: I867419ca36a81ef7209e6911a46c1c2c898b8eab
> Signed-off-by: Susant Palai 
> Reviewed-on: http://review.gluster.org/16328
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
>Reviewed-by: Raghavendra G 
(cherry picked from commit 451ca272d12f2d49522c845d53585520f71525f8)

Change-Id: If05094346fc1fc48f4982db6a195740171ae2fad
BUG: 1410764
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/16348
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/dht: Fix dict_leak in migration check tasks

2017-01-08T10:53:49+00:00

Fixed a memleak where dict was not being unrefed
in the dht_migration_complete_check_task and
dht_rebalance_inprogress_task functions.

BUG: 1410369
> Change-Id: I3d42e9a2e5c8596c985bf6431a68fd3905227383
> BUG: 1409186
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/16308
> Smoke: Gluster Build System 
> Reviewed-by: MOHIT AGRAWAL 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> (cherry picked from commit 11b6a2c9fc5232b58774cab29873406c0fbfef19)

Change-Id: I4e92e4c87b52f6f5ffb859005aa3d525225b4bda
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/16331
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

afr: allow I/O when favorite-child-policy is enabled

2017-01-08T10:48:49+00:00

Problem:
Currently, I/O on a split-brained file fails even when the
favorite-child-policy is set until the self-heal is complete.

Fix:
If a valid 'source' is found using the set favorite-child-policy, inspect
and reset the afr pending xattrs on the 'sinks' (inside appropriate locks),
refresh the inode and then proceed with the read or write transaction.

The resetting itself happens in the self-heal code and hence can also
happen in the client side background-heal or by the shd's index-heal in
addition to the txn code path explained above. When it happens in via
heal, we also add checks in undo-pending to not reset the sink xattrs
again.

> Reviewed-on: http://review.gluster.org/15673
> Tested-by: Pranith Kumar Karampuri 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ic8c1317720cb26bd114b6fe6af4e58c73b864626
BUG: 1378547
Signed-off-by: Ravishankar N 
Reported-by: Simon Turcotte-Langevin 
Reviewed-on: http://review.gluster.org/16091
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/dht: Fix memory corruption while accessing regex stored in

2017-01-04T03:41:24+00:00

private

If reconfigure is executed parallely (or concurrently with dht_init),
there are races that can corrupt memory. One such race is modification
of regexes stored in conf (conf->rsync_regex_valid and
conf->extra_regex_valid) through dht_init_regex. With change [1],
reconfigure codepath can get executed parallely (with itself or with
dht_init) and this fix is needed.

Also, a reconfigure can race with any thread doing dht_layout_search,
resulting in dht_layout_search accessing regex freed up by reconfigure
(like in bz 1399134).

[1] http://review.gluster.org/15046

>Change-Id: I039422a65374cf0ccbe0073441f0e8c442ebf830
>BUG: 1399134
>Signed-off-by: Raghavendra G 
>Reviewed-on: http://review.gluster.org/15945
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: N Balachandran 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Shyamsundar Ranganathan 

Change-Id: I039422a65374cf0ccbe0073441f0e8c442ebf830
BUG: 1399423
Signed-off-by: Raghavendra G 
(cherry picked from commit 64451d0f25e7cc7aafc1b6589122648281e4310a)
Reviewed-on: http://review.gluster.org/15793
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

dht/rename : Incase of failure remove linkto file properly

2017-01-02T13:07:25+00:00

Generally linkto file is created using root user. Consider following
case, a user is trying to rename a file which he is not permitted.
So the rename fails with EACESS and when rename tries to cleanup the
linkto file, it fails.

The above issue happens when rename/00.t test executed on nfs-ganesha
clients :
Steps executed in script
* create a file "abc" using root
* rename the file "abc" to "xyz" using a non root user, it fails with EACESS
* delete "abc"
* create directory "abc" using root
* again try ot rename "abc" to "xyz" using non root user, test hungs here
which slowly leds to OOM kill of ganesha process

RCA put forwarded by Du for OOM kill of ganesha
Note that when we hit this bug, we've a scenario of a dentry being
present as:
    * a linkto file on one subvol
    * a directory on rest of subvols

When a lookup happens on the dentry in such a scenario, the control flow
goes into an infinite loop of:

    dht_lookup_everywhere
    dht_lookup_everywhere_cbk
    dht_lookup_unlink_cbk
    dht_lookup_everywhere_done
    dht_lookup_directory (as local->dir_count > 0)
    dht_lookup_dir_cbk (sets to local->need_selfheal = 1 as the entry is a linkto file on one of the subvol)
    dht_lookup_everywhere (as need_selfheal = 1).

This infinite loop can cause increased consumption of memory due to:
1) dht_lookup_directory assigns a new layout to local->layout unconditionally
2)  Most of the functions in this loop do a stack_wind of various fops.

This results in growing of call stack (note that call-stack is destroyed only after lookup response is
received by fuse - which never happens in this case)

Thanks Du for root causing the oom kill and Sushant for suggesting the fix

Upstream reference :
>Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d
>BUG: 1397052
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/15894
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Raghavendra G 
>(cherry picked from commit 57d59f4be205ae0c7888758366dc0049bdcfe449)

Change-Id: I1e16bc14aa685542afbd21188426ecb61fd2689d
BUG: 1401029
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/16015
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/dht: Check for null inode

2017-01-02T13:04:39+00:00

Check for NULL inode before attempting to
set dht inode ctx.

> Change-Id: I7693c18445f138221d8417df5e95b118cedb818a
> BUG: 1395261
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15847
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
(cherry picked from commit 8313d53accaa22feb14d284fb91245be0a32e16e)

Change-Id: I7607d32d38d707dd5d71b98efffd1a458ffe90d7
BUG: 1395510
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15850
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Niels de Vos

cluster/afr: Fix missing name indices due to EEXIST error

2016-12-29T15:09:10+00:00

        Backport of: http://review.gluster.org/16286

PROBLEM:
Consider a volume with  granular-entry-heal and sharding enabled. When
a replica is down and a shard is created as part of a write, the name
index is correctly created under indices/entry-changes/.
Now when a read on the same region triggers another MKNOD, the fop
fails on the online bricks with EEXIST. By virtue of this being a
symmetric error, the failed_subvols[] array is reset to all zeroes.
Because of this, before post-op, the GF_XATTROP_ENTRY_OUT_KEY will be
set, causing the name index, which was created in the previous MKNOD
operation, to be wrongly deleted in THIS MKNOD operation.

FIX:
The ideal fix would have been for a transaction to delete the name
index ONLY if it knows it is the one that created the index in the first
place. This would involve gathering information as to whether THIS xattrop
created the index from individual bricks, aggregating their responses and
based on the various posisble combinations of responses, decide whether to
delete the index or not. This is rather complex. Simpler fix would be
for post-op to examine local->op_ret in the event of no failed_subvols
to figure out whether to delete the name index or not. This can occasionally
lead to creation of stale name indices but they won't be affecting the IO path
or mess with pending changelogs in any way and self-heal in its crawl of
"entry-changes" directory would take care to delete such indices.

Change-Id: Icc642a987d1b6a5097562315aecf1263ed35ceb6
BUG: 1408786
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/16293
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

afr: use accused matrix instead of readable matrix for deciding heals

2016-12-28T09:13:23+00:00

Problem:
afr_replies_interpret() used the 'readable' matrix to trigger client
side heals after inode refresh. But for arbiter, readable is always
zero. So when `dd` is run with a data brick down, spurious data heals
are are triggered. These heals open an fd, causing eager lock to be
disabled (open fd count >1) in afr transactions, leading to extra FXATTROPS

Fix:
Use the accused matrix (derived from interpreting the afr pending
xattrs) to decide whether we can start heal or not.

> Reviewed-on: http://review.gluster.org/16277
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> Tested-by: Pranith Kumar Karampuri 
(cherry picked from commit 5a7c86e578f5bbd793126a035c30e6b052177a9f)

Change-Id: Ibbd56c9aed6026de6ec42422e60293702aaf55f9
BUG: 1408772
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16291
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

dht/rebalance: reverify lookup failures

2016-12-28T05:54:41+00:00

race: readdirp has read one entry, and doing a lookup on
that entry, but user might have renamed/removed that entry just
after readdirp but before lookup.

Since remove-brick is a costly opertaion,will ingore any
ENOENT/ESTALE failures and move on.

> Change-Id: I62c7fa93c0b9b7e764065ad1574b97acf51b5996
> BUG: 1408115
> Signed-off-by: Susant Palai 
> Reviewed-on: http://review.gluster.org/15846
> Reviewed-by: Raghavendra G 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 

Change-Id: I62c7fa93c0b9b7e764065ad1574b97acf51b5996
BUG: 1408414
Reviewed-on: http://review.gluster.org/15846
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
(cherry picked from commit c20febcb1ffaef3fa29563987e7a3b554aea27b3)
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/16278