glusterfs.git/xlators/mgmt/glusterd, branch release-3.8

Fixes quota aux mount failure

2017-06-19T04:56:43+00:00

The aux mount is created on the first limit/remove_limit/list command
and it remains until volume is stopped / deleted / (quota is disabled)
, where we do a lazy unmount. If the process is uncleanly terminated,
then the mount entry remains and we get (Transport disconnected) error
on subsequent attempts to run quota list/limit-usage/remove commands.

Second issue, There is also a risk of inadvertent rm -rf on the
/var/run/gluster causing data loss for the user. Ideally, /var/run is
a temp path for application use and should not cause any data loss to
persistent storage.

Solution:
1) unmount the aux mount after each use.
2) clean stale mount before mounting, if any.

One caveat with doing mount/unmount on each command is that we cannot
use same mount point for both list and limit commands.
The reason for this is that list command needs mount to be accessible
in cli after response from glusterd, So it could be unmounted by a
limit command if executed in parallel (had we used same mount point)
Hence we use separate mount points for list and limit commands.

> Reviewed-on: https://review.gluster.org/16938
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 2ae4b4058691b324535d802f4e6d24cce89a10e5)

Change-Id: I4f9e39da2ac2b65941399bffb6440db8a6ba59d0
BUG: 1449782
Signed-off-by: Sanoj Unnikrishnan 
Reviewed-on: https://review.gluster.org/17242
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd/geo-rep: Fix snapshot create in geo-rep setup

2017-04-28T09:39:20+00:00

glusterd persists geo-rep sessions in glusterd
info file which is represented by dictionary
'volinfo->gsync_slaves' in memory. Glusterd also
maintains in memory active geo-rep sessions in
dictionary 'volinfo->gsync_active_slaves' whose key
is "::".

When glusterd is restarted while the geo-rep sessions
are active, it builds the 'volinfo->gsync_active_slaves'
from persisted glusterd info file. Since slave volume
uuid is added to "voinfo->gsync_slaves" with the commit
"http://review.gluster.org/13111", it builds it with key
":::" which is
wrong. So during snapshot pre-validation which checks
whether geo-rep is active or not, it always says it is
ACTIVE, as geo-rep stop would not deleted this key.
Fixed the same in this patch.


> BUG: 1443977
> Signed-off-by: Kotresh HR 
> Reviewed-on: https://review.gluster.org/17093
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
(cherry picked from commit f071d2a285ea4802fe8f328f9f275180983fbbba)

Change-Id: I185178910b4b8a62e66aba406d88d12fabc5c122
BUG: 1445213
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17109
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

glusterd: Fix snapshot failure in non-root geo-rep setup

2017-04-28T09:38:22+00:00

Geo-replication session directory name has the form
'__'. But in non-root
geo-replication setup, while preparing geo-replication
session directory name, glusterd is including 'user@'
resulting in "__".
Hence snapshot is failing to copy geo-rep specific
session files. Fixing the same.

> BUG: 1442760
> Signed-off-by: Kotresh HR 
> Reviewed-on: https://review.gluster.org/17067
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Amar Tumballi 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Aravinda VK 
(cherry picked from commit cc839523364e47dea715cd7241772cd68f05f76c)

Change-Id: Id214d3186e40997d2827a0bb60d3676ca2552df7
BUG: 1443012
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17071
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System

rpcsvc: Add rpchdr and proghdr to iobref before submitting to transport

2017-04-07T12:05:09+00:00

Backport of https://review.gluster.org/16613

Issue:
When fio is run on multiple clients (each client writes to its own files),
and meanwhile the clients does a readdirp, thus the client which did
a readdirp will now recieve the upcalls. In this scenario the client
disconnects with rpc decode failed error.

RCA:
Upcall calls rpcsvc_request_submit to submit the request to socket:
rpcsvc_request_submit currently:
rpcsvc_request_submit () {
   iobuf = iobuf_new
   iov = iobuf->ptr
   fill iobuf to contain xdrised upcall content - proghdr
   rpcsvc_callback_submit (..iov..)
   ...
   if (iobuf)
       iobuf_unref (iobuf)
}

rpcsvc_callback_submit (... iov...) {
   ...
   iobuf = iobuf_new
   iov1 = iobuf->ptr
   fill iobuf to contain xdrised rpc header - rpchdr
   msg.rpchdr = iov1
   msg.proghdr = iov
   ...
   rpc_transport_submit_request (msg)
   ...
   if (iobuf)
       iobuf_unref (iobuf)
}

rpcsvc_callback_submit assumes that once rpc_transport_submit_request()
returns the msg is written on to socket and thus the buffers(rpchdr, proghdr)
can be freed, which is not the case. In especially high workload,
rpc_transport_submit_request() may not be able to write to socket immediately
and hence adds it to its own queue and returns as successful. Thus, we have
use after free, for rpchdr and proghdr. Hence the clients gets garbage rpchdr
and proghdr and thus fails to decode the rpc, resulting in disconnect.

To prevent this, we need to add the rpchdr and proghdr to a iobref and send
it in msg:
   iobref_add (iobref, iobufs)
   msg.iobref = iobref;
The socket layer takes a ref on msg.iobref, if it cannot write to socket and
is adding to the queue. Thus we do not have use after free.

Thank You for discussing, debugging and fixing along:
Prashanth Pai 
Raghavendra G 
Rajesh Joseph 
Kotresh HR 
Mohammed Rafi KC 
Soumya Koduri 

> Reviewed-on: https://review.gluster.org/16613
> Reviewed-by: Prashanth Pai 
> Smoke: Gluster Build System 
> Reviewed-by: soumya k 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Ifa6bf6f4879141f42b46830a37c1574b21b37275
BUG: 1422788
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16638
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

glusterd: support filesystems with dynamic inode sizes

2017-04-07T11:57:22+00:00

btrfs and zfs are two filesystems that do not have fixed sizes for
inodes. Instead of logging an error, skip checking and mark the size as
"N/A" like other properties that can not be reported.

The error message that was reported by users on the mailinglist shows up
like:

  [glusterd-utils.c:5458:glusterd_add_inode_size_to_dict] 0-management: could not find (null) to getinode size for /dev/vdb (btrfs): (null) package missing?

Cherry picked from commit 12921693b572f642156d3167d1c92d3449dfc8ec:
> Change-Id: Ib10b7a3669f2f4221075715d9fd44ce1ffc35324
> Reported-by: Arman Khalatyan 
> URL: http://lists.gluster.org/pipermail/gluster-users/2017-March/030189.html
> BUG: 1433425
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/16867
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 

Change-Id: Ib10b7a3669f2f4221075715d9fd44ce1ffc35324
Reported-by: Arman Khalatyan 
BUG: 1436412
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16960
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Zhou Zhengping

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-21T19:04:27+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

master change https://review.gluster.org/16880
master BZ: 1430841
release-3.10 change https://review.gluster.org/16881
release-3.10 BZ: 1430845

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430845
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16882
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire

posix: Fix creation of files with S_ISVTX on FreeBSD

2017-02-26T21:47:33+00:00

On FreeBSD the S_ISVTX flag is completely ignored when creating a
regular file. Since gluster needs to create files with this flag set,
specialy for DHT link files, it's necessary to force the flag.

This fix does this by calling fchmod() after creating a file that
must have this flag set.

> Change-Id: I51eecfe4642974df6106b9084a0b144835a4997a
> BUG: 1411228
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: https://review.gluster.org/16417
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: Icaf46ebb440a3a722fd2fd771dd9d2f765b35ef4
BUG: 1424974
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16687
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

glusterd: ignore return code of glusterd_restart_bricks

2017-02-21T03:33:53+00:00

When GlusterD is restarted on a multi node cluster, while syncing the
global options from other GlusterD, it checks for quorum and based on
which it decides whether to stop/start a brick. However we handle the
return code of this function in which case if we don't want to start any
bricks the ret will be non zero and we will end up failing the import
which is incorrect.

Fix is just to ignore the ret code of glusterd_restart_bricks ()

>Reviewed-on: https://review.gluster.org/16574
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Samikshan Bairagya 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 55625293093d485623f3f3d98687cd1e2c594460)

Change-Id: I37766b0bba138d2e61d3c6034bd00e93ba43e553
BUG: 1420993
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16594
Smoke: Gluster Build System 
Reviewed-by: Samikshan Bairagya 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

glusterd/geo-rep: Fix geo-rep config issue

2017-01-08T11:00:57+00:00

Problem:
Geo-replication config commands fail when geo-rep
status is in 'Created'.

Cause:
During staging phase of geo-rep config, it checks
for the existence of 'monitor.pid' file. But the
pid file gets created only on start of geo-rep.
Hence it fails.

Fix:
Do not check for the existence of pid-file during
staging for config commands. It is not required.

> Change-Id: I626d368b249cf0423c7f49b4284465508371f566
> BUG: 1404678
> Signed-off-by: Kotresh HR 
> Reviewed-on: http://review.gluster.org/16132
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Aravinda VK 
> Reviewed-by: Atin Mukherjee 

Change-Id: I626d368b249cf0423c7f49b4284465508371f566
BUG: 1410699
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16342
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos

glusterd (geo-rep): fix unused variable warnings/errors

2017-01-08T10:59:21+00:00

This patch fixes the fd leak when geo-rep config
command is run while geo-rep is running.

NOTE:
The patch is backport of http://review.gluster.org/15263
which was one of the collection of patches to fix the
bug https://bugzilla.redhat.com/show_bug.cgi?id=1369124

> Change-Id: I2edacd3d0f3924b1be0f0398ba9ac076459c5a61
> BUG: 1369124
> Signed-off-by: Kaleb S. KEITHLEY 
> Reviewed-on: http://review.gluster.org/15263
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Saravanakumar Arumugam 

Change-Id: I2edacd3d0f3924b1be0f0398ba9ac076459c5a61
BUG: 1410708
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16344
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System