glusterfs.git/rpc, branch v3.7.11

Revert "glusterd: Bug fixes for IPv6 support"

2016-04-16T02:18:23+00:00

This reverts commit b33f3c95ec9c8112e6677e09cea05c4c462040d0.

This commit exposes some issues with management encryption that prevents
GlusterFS from operating properly. This will be added again once
problems with management encryption are fixed.

socket: Don't cleanup encrypted transport in socket_connect()

2016-04-10T04:54:47+00:00

..instead cleanup only in socket_poller()

  Backport of be99ddd from master

With commit d117466 socket_poller() wasn't launched from socket_connect
(for encrypted connections), if connect() failed. This was done to
prevent the socket private data from being double unreffed, from the
cleanups in both socket_poller() and socket_connect(). This allowed
future reconnects to happen successfully.

If a socket reconnects is sort of decided by the rpc notify function
registered. The above change worked with glusterd, as the glusterd rpc
notify function (glusterd_peer_rpc_notify()) continuously allowed
reconnects on failure.

mgmt_rpc_notify(), the rpc notify function in glusterfsd, behaves
differently.

For a DISCONNECT event, if more volfile servers are available or if more
addresses are available in the dns cache, it allows reconnects. If not
it terminates the program.

For a CONNECT event, it attempts to do a volfile fetch rpc request. If
sending this rpc fails, it immediately terminates the program.

One side effect of commit d117466, was that the encrypted socket was
registered with epoll, unintentionally, on a connect failure.  A weird
thing happens because of this. The epoll notifier notifies
mgmt_rpc_notify() of a CONNECT event, instead of a DISCONNECT as
expected. This causes mgmt_rpc_notify() to attempt an unsuccessful
volfile fetch rpc request, and terminate.
(I still don't know why the epoll raises the CONNECT event)

Commit 46bd29e fixed some issues with IPv6 in GlusterFS. This caused
address resolution in GlusterFS to also request of IPv6 addresses
(AF_UNSPEC) instead of just IPv4. On most systems, this causes the IPv6
addresses to be returned first.

GlusterD listens on 0.0.0.0:24007 by default. While this attaches to all
interfaces, it only listens on IPv4 addresses. GlusterFS daemons and
bricks are given 'localhost' as the volfile server. This resolves to
'::1' as the first address.

When using management encryption, the above reasons cause the daemon
processes to fail to fetch volfiles and terminate.

Solution
--------
The solution to this is simple. Instead of cleaning up the encrypted
socket in socket_connect(), launch socket_poller() and let it cleanup
the socket instead. This prevents the unintentional registration with
epoll, and socket_poller() sends the correct events to the rpc notify
functions, which allows proper reconnects to happen.

Change-Id: Idb0c0a828743cccca51cfdd1aa6458cfa0a9d100
BUG: 1325491
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/13931
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Tested-by: Gluster Build System 
CentOS-regression: Gluster Build System

afr: add mtime based split-brain resolution to CLI

2016-04-04T17:33:56+00:00

Backport of http://review.gluster.org/#/c/13828/

Extended the CLI to include support for split-brain resolution based on
mtime. The command syntax is:

$:gluster volume heal  split-brain latest-mtime 

where  can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file.

Change-Id: I7a16f72ff1a4495aa69f43f22758a9404e958b4f
BUG: 1321748
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13838
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Anuradha Talur 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

rpc: Connect back only if rpc is not disabled

2016-03-31T04:36:06+00:00

This is a backport of the below fix -
 http://review.gluster.org/13592

As discussed over http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/14284,
patch #13456 caused a regression where in if there are any pending
rpc invocations, we end up accessing freed object. This patch
fixes it by allowing reconnect during rpc submit only if rpc
is not disabled.

This is to fix regression caused by below patch -
http://review.gluster.org/#/c/13507/

Change-Id: I4ef4dd52bd42368bb89129f98bc973e46c6a39f4
BUG: 1311441
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13592
Reviewed-on: http://review.gluster.org/13824
Tested-by: soumya k 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

rpc: Fix for rpc_transport_t leak

2016-03-24T15:02:18+00:00

The transport object needs to get unref'ed when the rpc clnt
object is getting destroyed. But currently in rpc_clnt_disable()
we set conn->trans to NULL before it gets unref'ed leading to
transport object leak.

This change is to fix it by setting conn-tran to NULL only when
it is being unref'ed.

This is backport of the below patch
	- http://review.gluster.org/13456

Change-Id: I79ba34e28ae19eb616035f36bbed1c2f47875b94
BUG: 1311441
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/13456
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/13507
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/rpc : Discard duplicate Disconnect events

2016-03-23T14:50:23+00:00

Backport of http://review.gluster.org/#/c/13790/

If a peer rpc disconnect event has been already processed, skip the furthers as
processing them are overheads and sometimes may lead to a crash like due to a
double free

Change-Id: Iec589ce85daf28fd5b267cb6fc82a4238e0e8adc
BUG: 1320374
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/13790
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/13813

glusterd: Bug fixes for IPv6 support

2016-03-21T17:52:50+00:00

Backport of http://review.gluster.org/#/c/11988/

Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.

Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.

Tests Run:
Regression tests using ./run-tests.sh
IPv4: Regression tests using ./run-tests.sh for release-3.7 branch verified by comparing with clean repo.
IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
    Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here

Change-Id: Idd7513aa2347ce0de2b1f68daeecce1b7a39a7af
BUG: 1310445
Signed-off-by: Nithin D 
Reviewed-on: http://review.gluster.org/13787
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

socket: Launch socket_poller only if connect succeeded

2016-03-09T09:10:06+00:00

  Backport of 92abe07 from master

For an encrypted connection, sockect_connect() used to launch
socket_poller() in it's own thread (ON by default), even if the connect
failed. This would cause two unrefs to be done on the transport, once in
socket_poller() and once in socket_connect(), causing the transport to
be freed and cleaned up. This would cause further reconnect attempts
from failing as the transport wouldn't be available.

By starting socket_poller() only if connect succeeded, this is avoided.

BUG: 1314641
Change-Id: Ifd1bc4d48a8bdf741e32d02bdbac91530e0e8111
Signed-off-by: Kaushal M 
Originally-reviewed-on: http://review.gluster.org/13554
Reviewed-on: http://review.gluster.org/13604
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur 
CentOS-regression: Gluster Build System

socket: reduce rate of readv failure logs due to disconnect

2016-02-23T16:34:59+00:00

Backport of http://review.gluster.org/8210

... by using GF_LOG_OCCASIONALLY

Change-Id: I779ff32ead13c8bb446a57b5baccf068ae992df1
BUG: 1310969
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/8210
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/13487

rdma:restore device linked list structure in case of failure

2016-02-22T08:02:04+00:00

We maintain a linked list strcture to store device information,
and maintain head in a ctx variable. A new device will be added
at the beginning of list. But if the device creation failed, then
we need to adjust the linked list to proper state.

back port of :
>Change-Id: I07cefd3b808d8973a915728b3ba7f2955d29c92a
>BUG: 1250297
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/11829
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Raghavendra G 


(cherry picked from commit 7641eb8b469a6dd4db6db59d2a5ef4d5a65e1a61)

Change-Id: I5e7bcdef9402c11a139db8047ae2a9a18cdd8f4e
BUG: 1254430
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/11943
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G