glusterfs.git/cli, branch release-3.8-fb

event: Idle connection management

2017-09-12T15:12:21+00:00

Summary:
- This diff adds support for detecting and tracking idle client connections.
- It allows *service translators* (server, nfs) to opt-in to detect and close idle client connections.
- Right now it explicitly restricts the service to NFS as a safety.

Here are the debug logs when a client connection gets closed:

  [2016-03-29 17:27:06.154232] W [socket.c:2426:socket_timeout_handler] 0-socket: Shutting down idle client connection (idle=20s,fd=20,conn=[2401:db00:11:d0af:face:0:3:0:957]->[2401:db00:11:d0af:face:0:3:0:2049])!
  [2016-03-29 17:27:06.154292] D [event-epoll.c:655:__event_epoll_timeout_slot] 0-epoll: Connection on slot->fd=9 was idle for 20 seconds!
  [2016-03-29 17:27:06.163282] D [socket.c:629:__socket_rwv] 0-socket.nfs-server: EOF on socket
  [2016-03-29 17:27:06.163298] D [socket.c:2474:socket_event_handler] 0-transport: disconnecting now
  [2016-03-29 17:27:06.163316] D [event-epoll.c:614:event_dispatch_epoll_handler] 0-epoll: generation bumped on idx=9 from gen=4 to slot->gen=5, fd=20, slot->fd=20

Test Plan: - Used stuck NFS mounts to create idle clients and unstuck them.

Reviewers: kvigor, rwareing

Reviewed By: rwareing

Subscribers: dld, moox, dph

Differential Revision: https://phabricator.fb.com/D3112099

Change-Id: Ic06c89e03f87daabab7f07f892390edd1a1fcc20
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18265
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Merge remote-tracking branch 'origin/release-3.8' into release-3.8-fb

2017-08-31T19:33:59+00:00

Change-Id: Ie35cd1c8c7808949ddf79b3189f1f8bf0ff70ed8

cli/xml: fix return handling

2017-08-10T16:45:29+00:00

The return code of xmlTextWriter* APIs says it returns either the bytes
written (may be 0 because of buffering) or -1 in case of error. Now if the
volume of the xml data payload is not huge then most of the time the
data to be written gets buffered, however when this grows sometimes this
APIs will return the total number of bytes written and then it becomes
absolutely mandatory that every such call is followed by
XML_RET_CHECK_AND_GOTO otherwise we may end up returning a non zero ret
code which would result into the overall xml generation to fail.

>Reviewed-on: https://review.gluster.org/17702
>Smoke: Gluster Build System 
>Reviewed-by: Amar Tumballi 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Gaurav Yadav 

Change-Id: I02ee7076e1d8c26cf654d3dc3e77b1eb17cbdab0
BUG: 1470495
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17766
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Samikshan Bairagya 
Reviewed-by: Niels de Vos

Fixes quota aux mount failure

2017-06-19T04:56:43+00:00

The aux mount is created on the first limit/remove_limit/list command
and it remains until volume is stopped / deleted / (quota is disabled)
, where we do a lazy unmount. If the process is uncleanly terminated,
then the mount entry remains and we get (Transport disconnected) error
on subsequent attempts to run quota list/limit-usage/remove commands.

Second issue, There is also a risk of inadvertent rm -rf on the
/var/run/gluster causing data loss for the user. Ideally, /var/run is
a temp path for application use and should not cause any data loss to
persistent storage.

Solution:
1) unmount the aux mount after each use.
2) clean stale mount before mounting, if any.

One caveat with doing mount/unmount on each command is that we cannot
use same mount point for both list and limit commands.
The reason for this is that list command needs mount to be accessible
in cli after response from glusterd, So it could be unmounted by a
limit command if executed in parallel (had we used same mount point)
Hence we use separate mount points for list and limit commands.

> Reviewed-on: https://review.gluster.org/16938
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit 2ae4b4058691b324535d802f4e6d24cce89a10e5)

Change-Id: I4f9e39da2ac2b65941399bffb6440db8a6ba59d0
BUG: 1449782
Signed-off-by: Sanoj Unnikrishnan 
Reviewed-on: https://review.gluster.org/17242
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-21T19:04:27+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

master change https://review.gluster.org/16880
master BZ: 1430841
release-3.10 change https://review.gluster.org/16881
release-3.10 BZ: 1430845

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430845
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16882
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire

Increase CLI command timeout to 600 seconds for all commands

2017-03-05T20:48:04+00:00

Summary:
- Increase CLI command timeout to 600 seconds from 120 for use with large cluster creations
- My first attempt at this patch made an exception for create's only, but this proved to be a bad idea as then other commands would fail (such as start for example); so let's increase them all globally.
- The reason this timeout comes into play for large vol creates, is that the create flow is rather serial in nature and takes proportionately longer as the number of bricks increases.
- Parallelizing the CLI commands better may be some worthy future work

Test Plan:
- Run gluster vol create command w/ 264 bricks, prior to this patch it timed @ 120 seconds, afterwards it completes successfully
- Run gluster vol start command w/264 bricks, observe that it completes successfully

Example output:
https://phabricator.fb.com/P18037176

Reviewers: jackl, dph

Reviewed By: dph

Conflicts:
	cli/src/cli.h
FB-commit-id: e1f729e

Change-Id: I22c420fe17ad57b772e0061ad707cc2eeed1e01d
Signed-off-by: Kevin Vigor 
Reviewed-on: https://review.gluster.org/16848
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shreyas Siravara 
Smoke: Gluster Build System

Merge remote-tracking branch 'origin/release-3.8' into merge-3.8

2017-02-16T17:09:39+00:00

cli: keep 'gluster volume status detail' consistent

2017-01-25T04:50:01+00:00

The output of the command 'gluster volume status  detail' is
not consistent between operating systems. On linux hosts it shows the
file system type, the device name, mount options and inode size of each
brick. However the same command executed on a FreeBSD host doesn't show
all this information, even for bricks stored on a linux.

Additionally, for hosts other than linux, this information is shown as
'N/A' many times. This has been fixed to show as much information as it
can be retrieved from the operating system.

The file contrib/mount/mntent.c has been mostly rewriten because it
contained many errors that caused mount information to not be retrieved
on some operating systems.

> Change-Id: Icb6e19e8af6ec82255e7792ad71914ef679fc316
> BUG: 1411334
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: http://review.gluster.org/16371
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I01f99562dddff5d5cb6974df44d1b6366faf05dc
BUG: 1415132
Signed-off-by: Xavier Hernandez 
Reviewed-on: https://review.gluster.org/16442
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

Merge remote-tracking branch 'origin/release-3.8' into merge-3.8-again

2017-01-05T20:21:20+00:00

Change-Id: I844adf2aef161a44d446f8cd9b7ebcb224ee618a
Signed-off-by: Kevin Vigor

gluster: IPv6 single stack support

2016-12-08T04:02:02+00:00

Summary:
- This diff changes all locations in the code to prefer inet6 family
  instead of inet.  This will allow change GlusterFS to operate
  via IPv6 instead of IPv4 for all internal operations while still
  being able to serve (FUSE or NFS) clients via IPv4.
- The changes apply to NFS as well.
- This diff ports D1892990, D1897341 & D1896522 to the 3.8 branch.

Test Plan: Prove tests!

Reviewers: dph, rwareing

Signed-off-by: Shreyas Siravara 

Change-Id: I34fdaaeb33c194782255625e00616faf75d60c33
Reviewed-on: http://review.gluster.org/16059
Reviewed-by: Shreyas Siravara 
Tested-by: Shreyas Siravara 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System