glusterfs.git/xlators, branch release-3.9

features/shard: Fix EIO error on add-brick

2017-02-27T05:45:54+00:00

        Backport of: https://review.gluster.org/14419

DHT seems to link inode during lookup even before initializing
inode ctx with layout information, which comes after
directory healing.

Consider two parallel writes. As part of the first write,
shard sends lookup on .shard which in its return path would
cause DHT to link .shard inode. Now at this point, when a
second write is wound, inode_find() of .shard succeeds and
as a result of this, shard goes to create the participant
shards by issuing MKNODs under .shard. Since the layout is
yet to be initialized, mknod fails in dht call path with EIO,
leading to VM pauses.

The fix involves shard maintaining a flag to denote whether
a fresh lookup on .shard completed one network trip. If it
didn't, all inode_find()s in fop path will be followed by a
lookup before proceeding with the next stage of the fop.

Big thanks to Raghavendra G and Pranith Kumar K for the RCA
and subsequent inputs and feedback on the patch.

Change-Id: Id0d160157ad8f6bcd52801a2173c5869517d0a96
BUG: 1426512
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16752
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/shard: Put onus of choosing the inode to resolve on individual fops

2017-02-27T05:00:35+00:00

        Backport of: https://review.gluster.org/16709

... as opposed to adding checks in "common" functions to choose the inode
to resolve based local->fop, which is rather ugly and prone to errors.

Change-Id: I84c5b26160150f2fd87e7f245190c500a4b36bd8
BUG: 1426512
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16751
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Change log level of messages to DEBUG

2017-02-20T08:37:06+00:00

Heal failed or passed should not be logged as info. These can be
observed from heal info if the heal is happening or not. If we require
to debug a case where heal is not happening, we can set the level to
DEBUG.

>Change-Id: I062668eadd145ef809b25e818e6bca1094f54cd6
>BUG: 1420619
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/16580
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Ashish Pandey 

Change-Id: I91700a1960b5feb03ef186e4d0ddba1338152469
BUG: 1422783
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/16636
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/client: Fix double free of client fdctx destroy

2017-02-20T05:13:52+00:00

This patch fixes the race between fd re-open code and fd release code,
both of which free the fd context due to a race in certain variable
checks as explained below:

1. client process (shd in the case of this BZ) sends an opendir to its
children (client xlators) which send the fop to the bricks to get a valid fd.

2. Client xlator loses connection to the brick. fdctx->remotefd is -1

3. Client re-establishes connection. After handshake, it reopens the dir
and sets fdctx->remotefd to a valid fd in client3_3_reopendir_cbk().

4. Meanwhile, shd sends a fd unref after it is done with the opendir.
This triggers a releasedir (since fd->refcount becomes 0).

5. client3_3_releasedir() sees that fdctx-->remotefd is a valid number
(i.e not -1), sets fdctx->released=1 and calls  client_fdctx_destroy()

6. As a continuation of step3, client_reopen_done() is called by
client3_3_reopendir_cbk(), which sees that fdctx->released==1 and
again calls client_fdctx_destroy().

Depending on when step-5 does GF_FREE(fdctx), we may crash at any place in
step-6 in client3_3_reopendir_cbk() when it tries to access
fdctx->{whatever}.

> Reviewed-on: https://review.gluster.org/16521
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 25fc74f9d1f2b1e7bab76485a99f27abadd10b7b)

Change-Id: Ia50873d11763e084e41d2a1f4d53715438e5e947
BUG: 1422351
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/16620
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

nfs: revalidate lookup converted to fresh lookup

2017-02-16T18:53:43+00:00

Backport of http://review.gluster.org/15580

when an inode ctx is missing for a linked inode the revalidate
lookups are converted to fresh.

This could result in sending ESTALE when the gfid are recreated

We are not able to reproduce the issue with normal setup, most part of
RCA was done with code reading.

Possible scenario in which  this bug can reproduce,
Delete a file and recreate a new file with same name, at the same time
from another client process try to list/or access the file.

In this case the second client may throw an ESTALE error for such files

Thanks to Soumya and Pranith for doing the complete RCA

>Change-Id: I73992a65844b09a169cefaaedc0dcfb129d66ea1
>BUG: 1379720
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/15580
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: soumya k 
>Reviewed-by: Kaleb KEITHLEY 
Signed-off-by: Mohammed Rafi KC 

Change-Id: I44c3770fb07e84183f8bc6eceb533efbc67fb67f
BUG: 1394634
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: https://review.gluster.org/15839
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System

afr: all children of AFR must be up to resolve s-brain

2017-02-15T15:07:32+00:00

Problem:
The various split-brain resolution policies (favorite-child-policy based,
CLI based and mount (get/setfattr) based) attempt to resolve split-brain
even when not all bricks of replica are up. This can be a problem when
say in a replica 3, the only good copy is down and the other 2 bricks
are up and blame each other (i.e. split-brain). We end up healing the
file in such a  case and allow I/O on it.

Fix:
A decision on whether the file is in split-brain or not must be taken
only if we are able to examine the afr xattrs of *all* bricks of a given
replica.

> Reviewed-on: https://review.gluster.org/16476
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 0e03336a9362e5717e561f76b0c543e5a197b31b)

Change-Id: Icddb1268b380005799990f5379ef957d84639ef9
BUG: 1420983
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/16588
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

snapshot: Fix restore rollback to reassign snap volume ids to bricks

2017-02-03T06:25:56+00:00

Added further checks to ensure we do not go beyond prevalidate
when trying to restore a snapshot which has a nfs-gansha conf
file, in a cluster when nfs-ganesha is not enabled

The error message for the particular scenario is:
"Snapshot() has a nfs-ganesha export conf
file. cluster.enable-shared-storage and nfs-ganesha
should be enabled before restoring this snapshot."

> Reviewed-on: http://review.gluster.org/16116
> Reviewed-by: Rajesh Joseph 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: I1b87e9907e0a5e162f26ef1ca89fe76e8da8610f
BUG: 1405909
Signed-off-by: Avra Sengupta 
Reviewed-on: https://review.gluster.org/16195
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: jiffin tony Thottan 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

debug/trace: Print entries' iatts in readdirp cbk

2017-02-01T10:01:35+00:00

        Backport of: http://review.gluster.org/16432

Change-Id: I9d0b80cbdc3146b15ed39c8832ea39593803d1c3
BUG: 1415560
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/16453
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

upcall: Resolve dict leak from up_(f)removexattr in upcall code path

2017-01-31T19:51:59+00:00

Problem: In up_(f)removexattr() dict_for_key_value() is used to create a
         new dict. This dict is not correctly unref'd and gets leaked.

Solution: To avoid the leak up_(f)removexattr() now also does a
          dict_unref() on the newly created dict.

While reviewing the code in up_(f)setxattr() for a similar problem, it
was noticed that there is an extra dict created. There is no need for
this copy, upcall_local_init() can just take the dict that was passed as
argument to the FOP.

> BUG: 1412917
> Change-Id: I5bb9a7d99f5087af11c19ae722de62bdb5ad1498
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/16392
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Niels de Vos 
> Smoke: Gluster Build System 
> (cherry picked from commit afdd83a9b69573b854e732795c0bcba0a00d6c0f)

Change-Id: I0a53545528c43c09b88d360d3a12c460476647ba
BUG: 1417606
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/16480
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

features/changelog: Fix htime xattr during brick crash

2017-01-31T09:31:53+00:00

The htime file contains the path of all the changelogs
that is rolloved over till now. It also maintains xattr
which tracks the latest changelog file rolloved over
and the number of changelogs. The path and and xattr
update happens in two different system calls. If the
brick is crashed between them, the xattr value becomes
stale and can lead to the failure of gf_history_changelog.
To identify this, the total number of changelogs is being
calculated based on htime file size and the record
length. The above value is used in case of mismatch.

> Change-Id: Ia1c3efcfda7b74227805bb2eb933c9bd4305000b
> BUG: 1413967
> Signed-off-by: Kotresh HR 
> Reviewed-on: http://review.gluster.org/16420
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Aravinda VK 

Change-Id: Ia1c3efcfda7b74227805bb2eb933c9bd4305000b
BUG: 1415065
Signed-off-by: Kotresh HR 
(cherry picked from commit 6f4811ca9331eee8c00861446f74ebe23626bbf8)
Reviewed-on: https://review.gluster.org/16438
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Aravinda VK