glusterfs.git/xlators, branch v3.8.14

rpc/clnt: remove locks while notifying CONNECT/DISCONNECT

2017-07-11T13:25:31+00:00

Locking during notify was introduced as part of commit
aa22f24f5db7659387704998ae01520708869873 [1]. The fix was introduced
to fix out-of-order CONNECT/DISCONNECT events from rpc-clnt to parent
xlators [2]. However as part of handling DISCONNECT protocol/client
does unwind saved frames (with failure) waiting for responses. This
saved_frames_unwind can be a costly operation and hence ideally
shouldn't be included in the critical section of notifylock, as it
unnecessarily delays the reconnection to same brick. Also, its not a
good practise to pass control to other xlators holding a lock as it
can lead to deadlocks. So, this patch removes locking in rpc-clnt
while notifying parent xlators.

To fix [2], two changes are present in this patch:

* notify DISCONNECT before cleaning up rpc connection (same as commit
  a6b63e11b7758cf1bfcb6798, patch [3]).
* protocol/client uses rpc_clnt_cleanup_and_start, which cleans up rpc
  connection and does a start while handling a DISCONNECT event from
  rpc. Note that patch [3] was reverted as rpc_clnt_start called in
  quick_reconnect path of protocol/client didn't invoke connect on
  transport as the connection was not cleaned up _yet_ (as cleanup was
  moved post notification in rpc-clnt). This resulted in clients never
  attempting connect to bricks.

Note that one of the neater ways to fix [2] (without using locks) is
to introduce generation numbers to map CONNECT and DISCONNECTS across
epochs and ignore DISCONNECT events if they don't belong to current
epoch. However, this approach is a bit complex to implement and
requires time. So, current patch is a hacky stop-gap fix till we come
up with a more cleaner solution.

[1] http://review.gluster.org/15916
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1386626
[3] http://review.gluster.org/15681

Cherry picked from commit 773f32caf190af4ee48818279b6e6d3c9f2ecc79:
> Change-Id: I62daeee8bb1430004e28558f6eb133efd4ccf418
> Signed-off-by: Raghavendra G 
> BUG: 1427012
> Reviewed-on: https://review.gluster.org/16784
> Smoke: Gluster Build System 
> Reviewed-by: Milind Changire 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: I62daeee8bb1430004e28558f6eb133efd4ccf418
Reported-by: Markus Stockhausen 
Signed-off-by: Niels de Vos 
BUG: 1462447
Reviewed-on: https://review.gluster.org/17733
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Raghavendra G

posix: use the correct op_errno

2017-06-20T04:47:53+00:00

Problem:
If readdir/fstat was performed on a directory that was removed,
posix_fd_ctx_get() fails with ENOENT but we incorrectly use the ret
value (-1 in this case) as op_errno, logging "Operation not permitted"
messages in the brick logs. Also in case of fstat, the -1 op_errno was
also propagated to the client via stack unwind, causing the message to
appear in protocol/client logs as well.

Fix:
Use the right op_errno in readdir, fstat and writev.
A̶l̶s̶o̶,̶ ̶i̶f̶ p̶o̶s̶i̶x̶_̶f̶d̶_̶c̶t̶x̶_̶g̶e̶t̶(̶)̶ ̶f̶a̶i̶l̶e̶d̶ ̶w̶i̶t̶h̶ ̶E̶N̶O̶E̶N̶T̶,̶ ̶c̶o̶n̶v̶e̶r̶t̶ ̶i̶t̶ ̶i̶n̶t̶o̶ ̶E̶B̶A̶D̶F̶ ̶b̶e̶c̶a̶u̶s̶e̶ ̶E̶N̶O̶E̶N̶T̶
i̶s̶ ̶n̶o̶t̶ ̶a̶ ̶v̶a̶l̶i̶d̶ ̶e̶r̶r̶o̶r̶ ̶f̶o̶r̶ ̶a̶n̶ ̶f̶d̶ ̶o̶p̶e̶r̶a̶t̶i̶o̶n̶.̶ Don't do this as it breaks
DHT.See 17565

> Reviewed-on: https://review.gluster.org/17414
> Smoke: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 
> Tested-by: Pranith Kumar Karampuri 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Amar Tumballi 
> CentOS-regression: Gluster Build System 
(cherry picked from commit de92c363c95d16966dbcc9d8763fd4448dd84d13)

Change-Id: Ie43c0789d5040ec73b7cf885d015a183b8c64d70
BUG: 1460661
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/17517
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Fixes quota aux mount failure

2017-06-19T04:56:43+00:00

The aux mount is created on the first limit/remove_limit/list command
and it remains until volume is stopped / deleted / (quota is disabled)
, where we do a lazy unmount. If the process is uncleanly terminated,
then the mount entry remains and we get (Transport disconnected) error
on subsequent attempts to run quota list/limit-usage/remove commands.

Second issue, There is also a risk of inadvertent rm -rf on the
/var/run/gluster causing data loss for the user. Ideally, /var/run is
a temp path for application use and should not cause any data loss to
persistent storage.

Solution:
1) unmount the aux mount after each use.
2) clean stale mount before mounting, if any.

One caveat with doing mount/unmount on each command is that we cannot
use same mount point for both list and limit commands.
The reason for this is that list command needs mount to be accessible
in cli after response from glusterd, So it could be unmounted by a
limit command if executed in parallel (had we used same mount point)
Hence we use separate mount points for list and limit commands.

> Reviewed-on: https://review.gluster.org/16938
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 2ae4b4058691b324535d802f4e6d24cce89a10e5)

Change-Id: I4f9e39da2ac2b65941399bffb6440db8a6ba59d0
BUG: 1449782
Signed-off-by: Sanoj Unnikrishnan 
Reviewed-on: https://review.gluster.org/17242
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

afr: add errno to afr_inode_refresh_done()

2017-06-19T04:54:40+00:00

Backport of https://review.gluster.org/17413 and
https://review.gluster.org/17436

Problem:
When parellel `rm -rf`s were being done from cifs clients, opendir might
fail on some replicas with ENOENT. DHT ignores partial opendir failures
in dht_fd_cbk() and winds readdirs on those replicas. Afr inode refresh
(as a part of readdirp read_txn) sees in its fd context that the state
of the fds is *not* AFR_FD_OPENED and bails out to
afr_inode_refresh_done() without doing a refresh. When this happens, the
errno is set as EIO due to lack of readable subvols, logging split-brain
messages in the logs.

Fix:
Introduce an errno argument to afr_inode_refresh_do() to bail out with
the right error value when inode refresh is not performed.

Change-Id: I8eed4d6e6c85332c1f5813c74cb54ae73693a369
BUG: 1460661
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/17518
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

posix-acl: Whitelist virtual ACL xattrs

2017-06-19T04:54:02+00:00

Similar to system.posix_acl_* xattrs, all users should get
permission to be able to read glusterfs.posix.acl* xattrs too.

This is backport of below mainline patch -
https://review.gluster.org/17493

>BUG: 1459971
>Signed-off-by: Soumya Koduri 
>Reviewed-on: https://review.gluster.org/17493
>Smoke: Gluster Build System 
>Reviewed-by: jiffin tony Thottan 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra Talur 
>Reviewed-by: Niels de Vos 
>(cherry picked from commit 68f2192df570b5ee615d440c2e0c88d49a75a34f)

BUG: 1460650
Change-Id: I1fc2b67c8a12113910e4ec57cd114e4baefe0d38
Signed-off-by: Soumya Koduri 
Reviewed-on: https://review.gluster.org/17514
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: jiffin tony Thottan

nfs/nlm: remove lock request from the list after cancel

2017-06-16T09:57:48+00:00

Once an NLM client cancels a lock request, it should be removed from the
list. The list can also be cleaned of unneeded entries once the client
does not have any outstanding lock/share requests/granted.

Cherry picked from commit 71cb7f3eb4fb706aab7f83906592942a2ff2e924:
> Change-Id: I2f2b666b627dcb52cddc6d5b95856e420b2b2e26
> BUG: 1381970
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17188
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Kaleb KEITHLEY 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 

Change-Id: I2f2b666b627dcb52cddc6d5b95856e420b2b2e26
BUG: 1450380
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17278
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY

nfs/nlm: free the nlm_client upon RPC_DISCONNECT

2017-06-16T09:57:25+00:00

When an NLM client disconnects, it should be removed from the list and
free'd.

> Cherry picked from commit 6897ba5c51b29c05b270c447adb1a34cb8e61911:
> Change-Id: Ib427c896bfcdc547a3aee42a652578ffd076e2ad
> BUG: 1381970
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17189
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Kaleb KEITHLEY 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 

Change-Id: Ib427c896bfcdc547a3aee42a652578ffd076e2ad
BUG: 1450380
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17277
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY

nfs/nlm: log the caller_name if nlm_client_t can be found

2017-06-16T09:57:10+00:00

In order to help tracking possible misbehaving clients down, log the
'caller_name' (hostname of the NFS client) that does not have a matching
nlm_client_t structure.

Cherry picked from commit 9bfb74a39954a7e63bfd762c816efc7e64b9df65:
> Change-Id: Ib514a78d1809719a3d0274acc31ee632727d746d
> BUG: 1381970
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17186
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: Jeff Darcy 

Change-Id: Ib514a78d1809719a3d0274acc31ee632727d746d
BUG: 1450380
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17276
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

nfs/nlm: ignore notify when there is no matching rpc request

2017-06-16T09:56:52+00:00

In certain (unclear) occasions it seems to happen that there are
notifications sent to the Gluster/NFS NLM service, but no call-state can
be found. Instead of segfaulting, log an error but keep on running.

Cherry picked from commit e997d752ba08f80b1b00d2c0035874befafe5200:
> Change-Id: I0f186e56e46a86ca40314d230c1cc7719c61f0b5
> BUG: 1381970
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17185
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Jeff Darcy 

Change-Id: I0f186e56e46a86ca40314d230c1cc7719c61f0b5
BUG: 1450380
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17275
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY

nfs/nlm: unref rpc-client after nlm4svc_send_granted()

2017-06-16T09:56:39+00:00

nlm4svc_send_granted() uses the rpc_clnt by getting it from the
call-state structure. It is safer to unref the rpc_clnt after the
function is done with it.

Cherry picked from commit 52c28c0c04722a9ffaa7c39c49ffebdf0a5c75e1:
> Change-Id: I7cb7c4297801463d21259c58b50d7df7c57aec5e
> BUG: 1381970
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/17187
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: soumya k 
> Reviewed-by: Jeff Darcy 

Change-Id: I7cb7c4297801463d21259c58b50d7df7c57aec5e
BUG: 1450380
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17274
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY