glusterfs.git/xlators/cluster/ec, branch v3.8.6

afr,ec: Heal device files with correct major, minor numbers

2016-10-27T06:23:45+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

 >BUG: 1384297
 >Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15728
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >CentOS-regression: Gluster Build System 

Change-Id: I7646adc3771ff76cdf9c979b575bbcd0b3bc1b9a
BUG: 1388948
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15735
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Use locks for opendir

2016-10-14T20:20:15+00:00

Problem:
In some cases we see that readdir keeps winding to the brick that doesn't have
any blocked locks i.e. first brick. This is leading to the client assuming that
there are no blocking locks on the inode so it won't give away the lock. Other
clients end up blocked on the lock as if the command hung.

Fix:
Proper way to fix this issue is to use infra present in
http://review.gluster.org/14736 This is a stop gap fix where we start taking
inodelks in opendir which goes to all the bricks, this will detect if there is
any contention.

cherry picked from commit f013335400d033a9677797377b90b968803135f4:
>BUG: 1346719
>Change-Id: I91109107a26f6535b945ac476338e9f21dc31eb9
>Signed-off-by: Pranith Kumar K 
>Reviewed-on: http://review.gluster.org/15309
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Ashish Pandey 
>Signed-off-by: Ashish Pandey 

Change-Id: I91109107a26f6535b945ac476338e9f21dc31eb9
BUG: 1371397
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15405
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T10:22:36+00:00

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

> Change-Id: I9ca634b026ac830b172bac076437cc3bf1ae7d8a
> BUG: 1363721
> Signed-off-by: Krutika Dhananjay 
> Reviewed-on: http://review.gluster.org/15080
> Tested-by: Pranith Kumar Karampuri 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Oleksandr Natalenko 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit fcb5b70b1099d0379b40c81f35750df8bb9545a5)

Change-Id: I157f1025aebd6624fa3d412abc69a4ae6f2fe9e0
BUG: 1367272
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15221
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Unlock stale locks when inodelk/entrylk/lk fails

2016-07-29T10:50:49+00:00

Thanks to Rafi for hinting a while back that this kind of
problem he saw once. I didn't think the theory was valid.
Could have caught it earlier if I had tested his theory.

 >Change-Id: Iac6ffcdba2950aa6f8cf94f8994adeed6e6a9c9b
 >BUG: 1344836
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/14703
 >Reviewed-by: Xavier Hernandez 
 >Smoke: Gluster Build System 
 >Tested-by: mohammed rafi  kc 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 

BUG: 1360576
Change-Id: If9ccf0b3db7159b87ddcdc7b20e81cde8c3c76f0
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15025
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System

cluster/ec: Handle absence of keys in some callback dict

2016-07-27T05:51:20+00:00

Problem: This issue arises when we do a rolling update
from 3.7.5 to 3.7.9.
For 4+2 volume running 3.7.5, if we update 2 nodes
and after heal completion  kill 2 older nodes, this
problem can be seen. After update and killing of
bricks, 2 nodes will return inodelk count key in dict
while other 2 nodes will not have inodelk count in dict.
This is also true for get-link-count.
During dictionary match , ec_dict_compare, this will
lead to mismatch of answers and the file operation
on mount point will fail with IO error.

Solution:
Don't match inode, entry and link count keys while
comparing two dictionaries. However, while combining the
data in ec_dict_combine, go through all the dictionaries
and select the maximum values received in different dicts
for these keys.

master -
http://review.gluster.org/#/c/14761/

Change-Id: I33546e3619fe8f909286ee48fb0df2009cd3d22f
BUG: 1360174
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/14761
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15013

cluster/ec: Fix race in timer cancellation

2016-07-04T13:28:08+00:00

A race in timer cancellation for delayed unlock could cause a crash
if the cancelling thread fails to cancel the timer because it has
already been fired but not executed, and the callback is scheduled
out of the CPU, delaying it until the thread has released important
resources needed by the callback.

This patch improves the handling of this case to make it robust.

Backport of:
> Change-Id: I5c8a8c6610c5136f71b938aa78b5878ba05238d4
> BUG: 1345855
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: http://review.gluster.org/14712
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

Change-Id: I5c8a8c6610c5136f71b938aa78b5878ba05238d4
BUG: 1346158
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/14723
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Pass xdata to dht in case of error

2016-06-13T10:17:37+00:00

Problem: In case of mkdir failure, dht expects
error information so that it can act accordingly.
Aftre adding bricks and re balance, layout gets
changed. Fop "mkdir" with old layout returns EIO.
EC gets this error in xdata but does not pass it
back to dht. In this case dht will not be able to
take corrective action.

Solution: Return xdata back to dht

master -
http://review.gluster.org/#/c/14679/

Change-Id: I24def8038e6880607689b7b046dc6428f564c6ab
BUG: 1344594
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/14690
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/ec: Fix invalid __fd_unref() call

2016-06-13T10:16:17+00:00

__fd_unref() doesn't do any cleanup, so it cannot be called to release
fd references, specially if it's the last reference.

The code has been changed to avoid a call to this function.

In the previous version we always tried to keep the newest fd in the
ec_lock_t structure. However this is not necessary. We'll always keep
one reference to an open file on the same inode. It's irrelevant if
the reference is new or old.

The function __fd_unref() has also been removed from fd.h to avoid being
used in the future since it's useless as it's defined now.

Backport of http://review.gluster.org/14683

Change-Id: Ia728777fc8e464758d5ea4d3bf020f0603919039
BUG: 1344421
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/14684
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Niels de Vos

cluster/ec: Restrict the launch of replace brick heal

2016-06-13T10:12:12+00:00

Problem: When features.cache-invalidation is ON, a lot of
ec_notify function gets called which leads to launch of
too many heals. This leads to no heal completion,
which causes accumulation of heals.

Solution: ec_launch_replace_heal should not be launch
for every event. Replace brick will trigger a child up
event and then only this heal function should be called.

master -
http://review.gluster.org/#/c/14649/

Change-Id: I57b44c6a279d57230daea1d93229be6069245b7d
BUG: 1342954
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/14651
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/ec: Use correct log levels

2016-05-30T14:38:00+00:00

Problem :
Misleading messages are getting logged in mount logs
and bricks log.
"Mismatching xdata" and "Heal failed" are getting logged

Solution :
Reduce the level of logs from INFO, WARNING and NOTICE
to DEBUG level wherever applicable OR use fop_log_level
to get proper log level.

Backport of commit 02b2750ecc35f88c3262015b401dda962381f9da:
> Change-Id: Ia824c71e75ab683d3cb8949e1966ea09c9ccce72
> BUG: 1231224
> Signed-off-by: Ashish Pandey 
> Reviewed-on: http://review.gluster.org/13266
> Smoke: Gluster Build System 
> Reviewed-by: Xavier Hernandez 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

Change-Id: Ia824c71e75ab683d3cb8949e1966ea09c9ccce72
BUG: 1254934
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/14520
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Niels de Vos