glusterfs.git/cli, branch v3.11.0rc0

Halo Replication feature for AFR translator

2017-05-08T05:37:07+00:00

	Backport of https://review.gluster.org/16177
		    https://review.gluster.org/17174

Merged both these patches to make sure IPV6 changes don't make it to 3.11 at all.

Summary:
Halo Geo-replication is a feature which allows Gluster or NFS clients to write
locally to their region (as defined by a latency "halo" or threshold if you
like), and have their writes asynchronously propagate from their origin to the
rest of the cluster.  Clients can also write synchronously to the cluster
simply by specifying a halo-latency which is very large (e.g. 10seconds) which
will include all bricks.

In other words, it allows clients to decide at mount time if they desire
synchronous or asynchronous IO into a cluster and the cluster can support both
of these modes to any number of clients simultaneously.

There are a few new volume options due to this feature:
  halo-shd-latency:  The threshold below which self-heal daemons will
  consider children (bricks) connected.

  halo-nfsd-latency: The threshold below which NFS daemons will consider
  children (bricks) connected.

  halo-latency: The threshold below which all other clients will
  consider children (bricks) connected.

  halo-min-replicas: The minimum number of replicas which are to
  be enforced regardless of latency specified in the above 3 options.
  If the number of children falls below this threshold the next
  best (chosen by latency) shall be swapped in.

New FUSE mount options:
  halo-latency & halo-min-replicas: As descripted above.

This feature combined with multi-threaded SHD support (D1271745) results in
some pretty cool geo-replication possibilities.

Operational Notes:
- Global consistency is gaurenteed for synchronous clients, this is provided by
  the existing entry-locking mechanism.
- Asynchronous clients on the other hand and merely consistent to their region.
  Writes & deletes will be protected via entry-locks as usual preventing
  concurrent writes into files which are undergoing replication.  Read operations
  on the other hand should never block.
- Writes are allowed from _any_ region and propagated from the origin to all
  other regions.  The take away from this is care should be taken to ensure
  multiple writers do not write the same files resulting in a gfid split-brain
  which will require resolution via split-brain policies (majority, mtime &
  size).  Recommended method for preventing this is using the nfs-auth feature to
  define which region for each share has RW permissions, tiers not in the origin
  region should have RO perms.

TODO:
- Synchronous clients (including the SHD) should choose clients from their own
  region as preferred sources for reads.  Most of the plumbing is in place for
  this via the child_latency array.
- Better GFID split brain handling & better dent type split brain handling
  (i.e. create a trash can and move the offending files into it).
- Tagging in addition to latency as a means of defining which children you wish
  to synchronously write to

Test Plan:
- The usual suspects, clang, gcc w/ address sanitizer & valgrind
- Prove tests

Reviewers: jackl, dph, cjh, meyering

Reviewed By: meyering

Subscribers: ethanr

Differential Revision: https://phabricator.fb.com/D1272053

Tasks: 4117827

 >Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
 >BUG: 1428061
 >Signed-off-by: Kevin Vigor 
 >Reviewed-on: http://review.gluster.org/16099
 >Reviewed-on: https://review.gluster.org/16177
 >Tested-by: Pranith Kumar Karampuri 
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Pranith Kumar Karampuri 

BUG: 1448416
Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17192
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

cli: Adding warning message while creating a replica 2 volume

2017-04-25T15:21:59+00:00

Warn the CLI about ending up in split-brain situation with a
replica 2 volume. Display arbiter and replica 3 are recommended
option to avoid this and point to the document on split-brain
and ways to deal with it.

Change-Id: I7f31f3c74818d440a684b3130bc5ccdc72258f01
BUG: 1431963
Signed-off-by: karthik-us 
Reviewed-on: https://review.gluster.org/16899
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: Add client details to get-state output

2017-04-13T03:43:08+00:00

This commit optionally adds client details corresponding to the
locally running bricks to the get-state output. Since getting
the client details involves sending RPC requests to the respective
local bricks, this is a relatively more costly operation. These
client details would be added to the get-state output only if the
get-state command is invoked with the 'detail' option.

This commit therefore also changes the get-state CLI usage. The
modified usage is as follows:

 # gluster get-state [] [[odir ] \
[file ]] [detail]

Change-Id: I42cd4ef160f9e96d55a08a10d32c8ba44e4cd3d8
BUG: 1431183
Signed-off-by: Samikshan Bairagya 
Reviewed-on: https://review.gluster.org/17003
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

DHT: Spacing issue in fix-layout status output

2017-04-11T04:33:06+00:00

There is a spacing issue in the status output of the rebalance 
fix-layout operations. If the local host name is big then we will
have spacing issue.
To fix this we need to increase the space for node column.

Change-Id: Ic22e8017b55b05680db851a81f9b43c20489db79
BUG: 1437748
Signed-off-by: ankit 
Signed-off-by: AnkitRaj 
Reviewed-on: https://review.gluster.org/16972
Tested-by: ankitraj
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

build: miscellaneous spelling fixes

2017-04-03T00:26:59+00:00

Debian builds detected spelling issues with GlusterFS 3.10.1. Instead of
carrying the patch in the Debian sources, let's include the fixes here
too.

Change-Id: I38db6adf142f7ec247bffd47aa1e6ff1a0c49e00
Reported-by: Patrick Matthäi 
BUG: 1437853
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16973
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: (storhaug) remove ganesha

2017-03-21T17:13:44+00:00

remove all vestiges of ganesha

The storhaug CLI is used to manage ganesha and Samba. Also any setup
and teardown of the ganesha HA is initiated using storhaug to preserve
the proper layering.

Change-Id: I0eec0016a1b7802a36e7b2d92896b86fdf8607d5
BUG: 1420713
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16504
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:18:07+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430841
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Joe Julian 
Reviewed-by: Shyamsundar Ranganathan

Do not use va_end before cli_rl_out, since it use ap

2017-03-01T18:20:15+00:00

Trigger a cppcheck error

Change-Id: I8919fed47ef562319ac6baff38ed993e7cd54708
BUG: 789278
Signed-off-by: Michael Scherer 
Reviewed-on: https://review.gluster.org/16741
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Michael Scherer 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

dnt: Add a GF_DEFRAG_CMD_NONE to gf_defrag_type

2017-02-26T16:31:25+00:00

Coverity complain about enum mismatch since we assign GF_OP_CMD_NONE
to a variable holding gf_defrag_type.

Change-Id: I63e71f552b3cc752c26c1b8705420f38908e17e6
BUG: 789278
Signed-off-by: Michael Scherer 
Reviewed-on: https://review.gluster.org/16756
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Michael Scherer 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cli: fix error when HAVE_READLINE is false

2017-02-25T12:29:41+00:00

When HAVE_READLINE is false, Gluster doesn't compile with -Werror flag.
This will fix it up so it is possible to compile without having readline
present

Change-Id: Ia6620b5a372318d10f0e908e3d45f4090f99d118
BUG: 1426052
Reviewed-on: https://review.gluster.org/16734
Smoke: Gluster Build System 
Tested-by: Nigel Babu 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee