glusterfs.git/xlators/protocol, branch release-3.11

glusterfsd: process attach and detach request inside lock

2017-05-29T13:56:08+00:00

With brick multiplexing, there is a high possibility that attach and
detach requests might be parallely processed and to avoid a concurrent
update to the same graph list, a mutex lock is required.

Please note this backport defines the volfile_lock mutex which was done
as part of a different patch https://review.gluster.org/15036 in
mainline but is not available in release-3.11 branch.

Credits : Rafi (rkavunga@redhat.com) for the RCA of this issue

>Reviewed-on: https://review.gluster.org/17374
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit 3ca5ae2f3bff2371042b607b8e8a218bf316b48c)

Change-Id: Ic8e6d1708655c8a143c5a3690968dfa572a32a9c
BUG: 1455907
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17402
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

Halo Replication feature for AFR translator

2017-05-08T05:37:07+00:00

	Backport of https://review.gluster.org/16177
		    https://review.gluster.org/17174

Merged both these patches to make sure IPV6 changes don't make it to 3.11 at all.

Summary:
Halo Geo-replication is a feature which allows Gluster or NFS clients to write
locally to their region (as defined by a latency "halo" or threshold if you
like), and have their writes asynchronously propagate from their origin to the
rest of the cluster.  Clients can also write synchronously to the cluster
simply by specifying a halo-latency which is very large (e.g. 10seconds) which
will include all bricks.

In other words, it allows clients to decide at mount time if they desire
synchronous or asynchronous IO into a cluster and the cluster can support both
of these modes to any number of clients simultaneously.

There are a few new volume options due to this feature:
  halo-shd-latency:  The threshold below which self-heal daemons will
  consider children (bricks) connected.

  halo-nfsd-latency: The threshold below which NFS daemons will consider
  children (bricks) connected.

  halo-latency: The threshold below which all other clients will
  consider children (bricks) connected.

  halo-min-replicas: The minimum number of replicas which are to
  be enforced regardless of latency specified in the above 3 options.
  If the number of children falls below this threshold the next
  best (chosen by latency) shall be swapped in.

New FUSE mount options:
  halo-latency & halo-min-replicas: As descripted above.

This feature combined with multi-threaded SHD support (D1271745) results in
some pretty cool geo-replication possibilities.

Operational Notes:
- Global consistency is gaurenteed for synchronous clients, this is provided by
  the existing entry-locking mechanism.
- Asynchronous clients on the other hand and merely consistent to their region.
  Writes & deletes will be protected via entry-locks as usual preventing
  concurrent writes into files which are undergoing replication.  Read operations
  on the other hand should never block.
- Writes are allowed from _any_ region and propagated from the origin to all
  other regions.  The take away from this is care should be taken to ensure
  multiple writers do not write the same files resulting in a gfid split-brain
  which will require resolution via split-brain policies (majority, mtime &
  size).  Recommended method for preventing this is using the nfs-auth feature to
  define which region for each share has RW permissions, tiers not in the origin
  region should have RO perms.

TODO:
- Synchronous clients (including the SHD) should choose clients from their own
  region as preferred sources for reads.  Most of the plumbing is in place for
  this via the child_latency array.
- Better GFID split brain handling & better dent type split brain handling
  (i.e. create a trash can and move the offending files into it).
- Tagging in addition to latency as a means of defining which children you wish
  to synchronously write to

Test Plan:
- The usual suspects, clang, gcc w/ address sanitizer & valgrind
- Prove tests

Reviewers: jackl, dph, cjh, meyering

Reviewed By: meyering

Subscribers: ethanr

Differential Revision: https://phabricator.fb.com/D1272053

Tasks: 4117827

 >Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
 >BUG: 1428061
 >Signed-off-by: Kevin Vigor 
 >Reviewed-on: http://review.gluster.org/16099
 >Reviewed-on: https://review.gluster.org/16177
 >Tested-by: Pranith Kumar Karampuri 
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Pranith Kumar Karampuri 

BUG: 1448416
Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17192
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M

cli/auth : auth.allow and auth.reject does not accept FQDN/host name

2017-04-10T05:10:28+00:00

Problem : At the time of set FQDN name to "auth.allow/auth.reject" through
           gluster cli,it does not accept FQDN/host name.

Solution: Condition needs to be update in verify_host_name and gf_auth
           to accept FQDN/host name.

Fix : Change the condition to accept FQDN/host Name.
      To verify the patch followed below procedure
      1) Try to set FQDN name for auth.allow or auth.reject parameter
         gluster v set myvol auth.reject 
         It gives error "fqdn-name" is not a valid internet-address-list
      2) After apply the patch it does not give any error.
      3) To verify auth.allow/reject try to mount volume on some client.

Change-Id: Ieb76cbb93d43323fd29c7ca04efe3790edb4281b
BUG: 1321578
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/15086
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

protocol: corect parameter state->size in server3_3_writev

2017-04-02T04:34:22+00:00

Change-Id: I1a3d558670e4320fe8508b5ab97deff9aacca5c7
BUG: 1436936
Signed-off-by: Zhou Zhengping 
Reviewed-on: https://review.gluster.org/16962
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

protocol : fix auth-allow regression

2017-03-30T05:56:59+00:00

One of the brick multiplexing patches (commit 1a95fc3) had some changes
in gf_auth () & server_setvolume () functions which caused auth-allow
feature to be broken. mount doesn't succeed even if it's part of the
auth-allow list. This fix does the following:

1. Reintroduce the peer-info data back in gf_auth () so that fnmatch has
valid input and it can decide on the result.

2. config-params dict should capture key values pairs for all the bricks
in case brick multiplexing is on. In case brick multiplexing isn't
enabled, then config-params should carry attributes from protocol/server
such that all rpc auth related attributes stay in tact in the
dictionary.

Change-Id: I007c4c6d78620a896b8858a29459a77de8b52412
BUG: 1433815
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16920
Tested-by: Jeff Darcy 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: MOHIT AGRAWAL

protocol-client: Initialize the list_head before using

2017-03-27T00:56:19+00:00

In client3_3_readdir(p)_cbk, in case of error conditions,
it is possible that the list_head is used before initializing.
Hence move the initialization before usage.

Change-Id: Ie58902d079fdc58416d17b5fa5f61375decb1c99
BUG: 1435943
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16948
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

protocol/server: Fix crash bug in unlink flow

2017-03-02T19:23:20+00:00

Summary: Fixes a crash bug during unlink in server-rpc-fops.c

Signed-off-by: Shreyas Siravara 

Change-Id: I049a9863ffd4003742276e0aa9e8d1224488182d
BUG: 1428093
Tested-by: Shreyas Siravara 
Reviewed-on: https://review.gluster.org/16814
Tested-by: Vijay Bellur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy

rpc/clnt: remove locks while notifying CONNECT/DISCONNECT

2017-03-01T14:35:48+00:00

Locking during notify was introduced as part of commit
aa22f24f5db7659387704998ae01520708869873 [1]. The fix was introduced
to fix out-of-order CONNECT/DISCONNECT events from rpc-clnt to parent
xlators [2]. However as part of handling DISCONNECT protocol/client
does unwind saved frames (with failure) waiting for responses. This
saved_frames_unwind can be a costly operation and hence ideally
shouldn't be included in the critical section of notifylock, as it
unnecessarily delays the reconnection to same brick. Also, its not a
good practise to pass control to other xlators holding a lock as it
can lead to deadlocks. So, this patch removes locking in rpc-clnt
while notifying parent xlators.

To fix [2], two changes are present in this patch:

* notify DISCONNECT before cleaning up rpc connection (same as commit
  a6b63e11b7758cf1bfcb6798, patch [3]).
* protocol/client uses rpc_clnt_cleanup_and_start, which cleans up rpc
  connection and does a start while handling a DISCONNECT event from
  rpc. Note that patch [3] was reverted as rpc_clnt_start called in
  quick_reconnect path of protocol/client didn't invoke connect on
  transport as the connection was not cleaned up _yet_ (as cleanup was
  moved post notification in rpc-clnt). This resulted in clients never
  attempting connect to bricks.

Note that one of the neater ways to fix [2] (without using locks) is
to introduce generation numbers to map CONNECT and DISCONNECTS across
epochs and ignore DISCONNECT events if they don't belong to current
epoch. However, this approach is a bit complex to implement and
requires time. So, current patch is a hacky stop-gap fix till we come
up with a more cleaner solution.

[1] http://review.gluster.org/15916
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1386626
[3] http://review.gluster.org/15681

Change-Id: I62daeee8bb1430004e28558f6eb133efd4ccf418
Signed-off-by: Raghavendra G 
BUG: 1427012
Reviewed-on: https://review.gluster.org/16784
Smoke: Gluster Build System 
Reviewed-by: Milind Changire 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/client: Add gfid in request for better tcpdumps

2017-02-28T04:28:07+00:00

It is difficult to match opendir/releasedir or
open/release calls without gfid associlation to find
fd-leaks on the bricks. So assigning gfid also in
the request which was already there in the on-wire
format.

BUG: 1425676
Change-Id: Iec908eeaa2f97295d45140a529b7f8fb834a1553
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/16706
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs, gfdb, glusterfs: Add missing breaks

2017-02-26T16:29:26+00:00

A few switches did not have breaks causing fall throughs. Most of them
have been fixed with fall through comments for those that are
intentional.

Change-Id: I84c85726b542f38504b50fefab5eba5dbcd27a07
BUG: 1424894
Signed-off-by: Nigel Babu 
Reviewed-on: https://review.gluster.org/16677
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Shyamsundar Ranganathan