glusterfs.git, branch v3.12.1

doc : release-notes for GlusterFS-3.12.1

2017-09-11T16:10:23+00:00

Change-Id: Id852cec1a68966356d85856fb93cabcc122bb7ee
BUG: 1488560
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: https://review.gluster.org/18258
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

afr: heal metadata in discover code path

2017-09-11T10:54:39+00:00

Combined backport of https://review.gluster.org/17850 and
https://review.gluster.org/18187

During graph switch, if fuse sends nameless (gfid) lookups, afr takes
the discover code path to serve it. If there are pending metadata heals,
they do not happen unless an inode refresh happens as a part of
discover (which is not guaranteed to happen always).

This patch fixes it by attempting metadata heal as a part of discover,
just like how it is done in lookup code path.

Change-Id: I87c493045b9225741cad173bf3f645848697032e
BUG: 1488168
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18202
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Karthik U S 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: jiffin tony Thottan

event/epoll: don't call handler for events received after a pollerr

2017-09-11T04:59:34+00:00

we register socket with EPOLLONESHOT, which means it has to be
explicitly added back through epoll_ctl to receive more
events. Normally we do this once the handler completes processing of
current event. But event_select_on_epoll is one asynchronous codepath
where socket can be added back for polling while an event on the same
socket is being processed. event_select_on_epoll has a check whether
an event is being processed in the form of slot->in_handler. But this
check is not sufficient enough to prevent parallel events as
slot->in_handler is not atomically incremented with respect to
reception of the event. This means following imaginary sequence of
events can happen:

* epoll_wait returns with a POLLERR - say POLLERR1 - on a socket
  (sock1) associated with slot s1. socket_event_handle_pollerr is yet
  to be invoked.
* an event_select_on called from __socket_ioq_churn which was called
  in request/reply/msg submission codepath (as opposed to
  __socket_ioq_churn called as part of POLLOUT handling - we cannot
  receive a POLLOUT due to EPOLLONESHOT) adds back sock1 for polling.
* since sock1 was added back for polling in step 2 and our polling is
  level-triggered, another thread picks up another POLLERR event - say
  POLLERR2. socket_event_handler is invoked as part of processing
  POLLERR2 and it completes execution setting priv->sock to -1.
* event_unregister_epoll called as part of __socket_reset due to
  POLLERR1 would receive fd as -1 resulting in assert failure.

Also, since the first pollerr event has done rpc_transport_unref,
subsequent parallel events (not just pollerr, but other events too)
could be acting on a freed up transport too.

>Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
>BUG: 1486134
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/18129
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: mohammed rafi  kc 

(cherry picked from commit b1b49997574eeb7c6a42e6e8257c81ac8d2d7578)
Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
BUG: 1489296
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/18223
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan

arbiter: return ENOSYS for 'non readable' FOPs

2017-09-10T18:34:38+00:00

AFR marks the arbiter as 'non readable'. This has been introduced with
commit 8ab87137 (afr: do not set arbiter as a readable subvol in inode
context). arbiter_readv() should not get called anymore, so it could be
removed. However, it is a good defensive approach to have all the inode
read FOPs that can not be handled by the arbiter to return ENOSYS.

> Reviewed-on: https://review.gluster.org/18103
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit b1352d0974328b367afa7360e9523585efb7178d)

Change-Id: I6ea41680832859bd6790dc8d7440ee98d38205fc
BUG: 1489511
Signed-off-by: Niels de Vos 
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18227
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

dht: add FOP check to dht_file_setattr_cbk

2017-09-10T18:32:02+00:00

Problem:
bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a
non fd-based fop, namely setattr. This caused
dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL.

Fix:
Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk
only for dht_fsetattr and not dht_setattr or dht_setattr2

> Reviewed-on: https://review.gluster.org/18208
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 47188e9eac59de416a5c86c7ec7540ed6aaa1c98)

Signed-off-by: Ravishankar N 
Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99
BUG: 1489260
Reviewed-on: https://review.gluster.org/18222
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

cluster/dht: Log files skipped by rebalance

2017-09-07T07:09:20+00:00

There was no easy way to find out which files were
skipped during a rebalance.
Rebalance now logs a message for every skipped file
using msgid 109126, making it easier to find
all files that were skipped.

> BUG: 1480445
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18021
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: hari gowtham 
> Reviewed-by: Raghavendra G 

Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
BUG: 1486557
(cherry picked from commit a4c43ba9374b8f75a48d38a032353a0c7d311a73)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18149
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

cluster/dht: Aggregate xattrs only for dirs in dht_discover_cbk

2017-09-07T07:07:55+00:00

If dht_discover finds data files on more than one subvol,
racing calls to dht_discover_cbk could end up calling
dht_aggregate_xattr which could delete dictionary data
that is being accessed by higher layer translators.
Fixed to call dht_aggregate_xattr only for directories and
consider only the first file to be found.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18137
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I4f3d2a405ec735d4f1bb33a04b7255eb2d179f8a
BUG: 1486538
(cherry picked from commit 9420022df0962b1fa4f3ea3774249be81bc945cc)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18146
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System

perf/qr: Use a ref-ed data to extract content

2017-09-07T06:57:43+00:00

qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.

Fix: Use dict_get_with_ref to retrieve the file contents.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18115
> Smoke: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Tested-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1486538
(cherry picked from commit 414d3e92fc56f08e320a3aa65b6b18e65b384551)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18145
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: MOHIT AGRAWAL

rpc: destroy transport after client_t

2017-09-07T06:49:51+00:00

Problem:
1. Ref counting increment on the client_t object is done in
   rpcsvc_request_init() which is incorrect.
2. Ref not taken when delegating to grace_time_handler()

Solution:
1. Only fop requests which require processing down the graph via
   stack 'frames' now ref count the request in get_frame_from_request()
2. Take ref on client_t object in server_rpc_notify() but avoid
   dropping in RPCSVC_EVENT_TRANSPORT_DESRTROY. Drop the ref
   unconditionally when exiting out of grace_time_handler().
   Also, avoid dropping ref on client_t in
   RPCSVC_EVENT_TRANSPORT_DESTROY when ref mangement as been
   delegated to grace_time_handler()

mainline:
> BUG: 1481600
> Reported-by: Raghavendra G 
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/17982
> Tested-by: Raghavendra G 
> Reviewed-by: Raghavendra G 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
(cherry picked from commit 24b95089a18a6a40e7703cb344e92025d67f3086)

Change-Id: Ic16246bebc7ea4490545b26564658f4b081675e4
BUG: 1487033
Reported-by: Raghavendra G 
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/18156
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G 
CentOS-regression: Gluster Build System

afr: check validity of afr_reply

2017-09-07T06:46:19+00:00

...in various self-heal code paths.

Originally found by Pranith in __afr_selfheal_name_impunge ()

Also change __afr_selfheal_assign_gfid() to send lookup only on those
bricks that don't have a gfid matching that of the source.


> Reviewed-on: https://review.gluster.org/18065
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
(cherry picked from commit d594900dbca92c356152be65fce16f77c402117c)

Change-Id: I70a2ccd750a2af92c5fc36e0eefb2b6125404b4a
BUG: 1487319
Signed-off-by: Pranith Kumar K 
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18173
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System