glusterfs.git/xlators, branch v3.12.1

afr: heal metadata in discover code path

2017-09-11T10:54:39+00:00

Combined backport of https://review.gluster.org/17850 and
https://review.gluster.org/18187

During graph switch, if fuse sends nameless (gfid) lookups, afr takes
the discover code path to serve it. If there are pending metadata heals,
they do not happen unless an inode refresh happens as a part of
discover (which is not guaranteed to happen always).

This patch fixes it by attempting metadata heal as a part of discover,
just like how it is done in lookup code path.

Change-Id: I87c493045b9225741cad173bf3f645848697032e
BUG: 1488168
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18202
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Karthik U S 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: jiffin tony Thottan

arbiter: return ENOSYS for 'non readable' FOPs

2017-09-10T18:34:38+00:00

AFR marks the arbiter as 'non readable'. This has been introduced with
commit 8ab87137 (afr: do not set arbiter as a readable subvol in inode
context). arbiter_readv() should not get called anymore, so it could be
removed. However, it is a good defensive approach to have all the inode
read FOPs that can not be handled by the arbiter to return ENOSYS.

> Reviewed-on: https://review.gluster.org/18103
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit b1352d0974328b367afa7360e9523585efb7178d)

Change-Id: I6ea41680832859bd6790dc8d7440ee98d38205fc
BUG: 1489511
Signed-off-by: Niels de Vos 
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18227
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

dht: add FOP check to dht_file_setattr_cbk

2017-09-10T18:32:02+00:00

Problem:
bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a
non fd-based fop, namely setattr. This caused
dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL.

Fix:
Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk
only for dht_fsetattr and not dht_setattr or dht_setattr2

> Reviewed-on: https://review.gluster.org/18208
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 47188e9eac59de416a5c86c7ec7540ed6aaa1c98)

Signed-off-by: Ravishankar N 
Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99
BUG: 1489260
Reviewed-on: https://review.gluster.org/18222
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

cluster/dht: Log files skipped by rebalance

2017-09-07T07:09:20+00:00

There was no easy way to find out which files were
skipped during a rebalance.
Rebalance now logs a message for every skipped file
using msgid 109126, making it easier to find
all files that were skipped.

> BUG: 1480445
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18021
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: hari gowtham 
> Reviewed-by: Raghavendra G 

Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
BUG: 1486557
(cherry picked from commit a4c43ba9374b8f75a48d38a032353a0c7d311a73)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18149
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

cluster/dht: Aggregate xattrs only for dirs in dht_discover_cbk

2017-09-07T07:07:55+00:00

If dht_discover finds data files on more than one subvol,
racing calls to dht_discover_cbk could end up calling
dht_aggregate_xattr which could delete dictionary data
that is being accessed by higher layer translators.
Fixed to call dht_aggregate_xattr only for directories and
consider only the first file to be found.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18137
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I4f3d2a405ec735d4f1bb33a04b7255eb2d179f8a
BUG: 1486538
(cherry picked from commit 9420022df0962b1fa4f3ea3774249be81bc945cc)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18146
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System

perf/qr: Use a ref-ed data to extract content

2017-09-07T06:57:43+00:00

qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.

Fix: Use dict_get_with_ref to retrieve the file contents.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18115
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Tested-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1486538
(cherry picked from commit 414d3e92fc56f08e320a3aa65b6b18e65b384551)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18145
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: MOHIT AGRAWAL

rpc: destroy transport after client_t

2017-09-07T06:49:51+00:00

Problem:
1. Ref counting increment on the client_t object is done in
   rpcsvc_request_init() which is incorrect.
2. Ref not taken when delegating to grace_time_handler()

Solution:
1. Only fop requests which require processing down the graph via
   stack 'frames' now ref count the request in get_frame_from_request()
2. Take ref on client_t object in server_rpc_notify() but avoid
   dropping in RPCSVC_EVENT_TRANSPORT_DESRTROY. Drop the ref
   unconditionally when exiting out of grace_time_handler().
   Also, avoid dropping ref on client_t in
   RPCSVC_EVENT_TRANSPORT_DESTROY when ref mangement as been
   delegated to grace_time_handler()

mainline:
> BUG: 1481600
> Reported-by: Raghavendra G 
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/17982
> Tested-by: Raghavendra G 
> Reviewed-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
(cherry picked from commit 24b95089a18a6a40e7703cb344e92025d67f3086)

Change-Id: Ic16246bebc7ea4490545b26564658f4b081675e4
BUG: 1487033
Reported-by: Raghavendra G 
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/18156
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G 
CentOS-regression: Gluster Build System

afr: check validity of afr_reply

2017-09-07T06:46:19+00:00

...in various self-heal code paths.

Originally found by Pranith in __afr_selfheal_name_impunge ()

Also change __afr_selfheal_assign_gfid() to send lookup only on those
bricks that don't have a gfid matching that of the source.


> Reviewed-on: https://review.gluster.org/18065
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit d594900dbca92c356152be65fce16f77c402117c)

Change-Id: I70a2ccd750a2af92c5fc36e0eefb2b6125404b4a
BUG: 1487319
Signed-off-by: Pranith Kumar K 
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18173
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

features/shard: Return aggregated size in stbuf of LINK fop

2017-09-07T06:43:43+00:00

    Backport of:
    > Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0
    > Reviewed-on: https://review.gluster.org/18209
    > BUG: 1488546
    > cherry-picked from 91430817ce5bcbeabf057e9c978485728a85fb2b

Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0
BUG: 1488718
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/18211
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

features/shard: Increment counts in locks

2017-09-06T07:48:11+00:00

       Backport of https://review.gluster.org/18203

Problem:
Because create_count/eexist_count are incremented without locks, all the shards may not
be created because call_count will be lesser than what it needs to be. This can lead
to crash in shard_common_inode_write_do() because inode on which we want to do
fd_anonymous() is NULL

Fix:
Increment the counts in frame->lock

 >Change-Id: Ibc87dcb1021e9f4ac2929f662da07aa7662ab0d6
 >BUG: 1488354
 >Signed-off-by: Pranith Kumar K 

Change-Id: Ibc87dcb1021e9f4ac2929f662da07aa7662ab0d6
BUG: 1488387
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/18204
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
CentOS-regression: Gluster Build System