glusterfs.git/rpc, branch v3.8.1

xdr/nfs: free complete groupnode structure

2016-06-13T10:10:48+00:00

The groupnode->gr_next pointer is not traversed upon free. This is
currently not a problem, because the pointer is never used. However the
correct way to free a groupnode should check the ->gr_next pointer and
free any of the groups that it encounters.

This problem was identified while correcting a problem with the MOUNT
protocol. The change "nfs: build exportlist with multiple groups" starts
to use ->gr_next.

This is backport of below mainline fix -
	http://review.gluster.org/#/c/14666/

Change-Id: I9d04eaf4c65bdb8db136321d60e70789da1739d7
BUG: 1343287
Signed-off-by: Niels de Vos 
Signed-off-by: Bipin Kunal 
Reviewed-on: http://review.gluster.org/14699
Smoke: Gluster Build System 
Reviewed-by: bipin kunal 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol: Add framework to send transaction id with recall

2016-06-10T11:03:54+00:00

Backport of: http://review.gluster.org/#/c/14647/

Issue:
The upcall(cache invalidation/recall) event is sent from the bricks
to clients. In AFR/EC setup, it can so happen that all the bricks
will send the upcall for the same event, and if AFR/EC doesn't filter
out these duplicate notifications, the logic above cluster xlators
can fail.

Solution:
Use transaction id to filter out duplicate notifications.

This patch adds framework for duplicate notifications.
AFR/EC can build up on this patch for deduping the notifications

Change-Id: I66b08e63b8799bc5932f2b2545376138a5701168
BUG: 1337638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14648
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs (timer): race conditions, illegal mem access, mem leak

2016-06-08T12:15:20+00:00

While investigating gfapi memory consumption with valgrind, valgrind
reported several memory access issues.

Also see the timer 'registry' being recreated (shortly) after being
freed during teardown due to the way it's currently written.

Passing ctx as data to gf_timer_proc() is prone to memory access
issues if ctx is freed before gf_timer_proc() terminates. (And in
fact this does happen, at least in valgrind.) gf_timer_proc() doesn't
need ctx for anything, it only needs ctx->timer, so just pass that.

Nothing ever calls gf_timer_registry_init(). Nothing outside of
timer.c that is. Making it and gf_timer_proc() static.

backport mainline:
> http://review.gluster.org/14247
> BUG: 1333925

Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a
BUG: 1342620
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14644
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: Honour mandatory lock flags during lock migration

2016-05-27T14:50:37+00:00

lk_flags from posix_lock_t structure is the primary key used to
differentiate locks as either advisory and mandatory type. During
lock migration this field is not read in getactivelk() call path.
So in order to copy the exact lock state from source to destination
it is necessary to include lk_flags within lock_migration_info_t
structure to maintain accurate state. This change also includes
minor modifications to setactivelk() call to consider lk_flags
during lock migration.

> Reviewed-on: http://review.gluster.org/14189
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Poornima G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

(cherry picked from commit deaf8439fc42435988aae6a7b9ab681cc0d36b09)

Change-Id: I20a7b6b6a0f3bdac5734cce8a2cd2349eceff195
BUG: 1337805
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14457
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

rpc: change client insecure port ceiling from 65535 to 49151

2016-05-19T08:56:53+00:00

current port allocation to various processes (clumsy):

 1023 - 1       -> client ports range if bind secure is turned on
49151 - 1024    -> fall back to this, if in above case ports exhaust
65535 - 1024    -> client port range if bind insecure is on
49152 - 65535   -> brick port range

now, we have segregated port ranges 0 - 65535 to below 3 ranges

 1023 - 1       -> client ports range if bind secure is turned on
49151 - 1024    -> client port range if bind insecure is on
                   (fall back to this, if in above case ports exhaust)
49152 - 65535   -> brick port range

so now we have a clean segregation of port mapping

Backport of:
> Change-Id: Ie3b4e7703e0bbeabbe0adbdd6c60d9ef78ef7c65
> BUG: 1335776
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14326
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Raghavendra Talur 
> Tested-by: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Raghavendra G 

Change-Id: Ie3b4e7703e0bbeabbe0adbdd6c60d9ef78ef7c65
BUG: 1337127
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14413
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

socket: Fix incorrect handling of partial reads

2016-05-12T05:17:25+00:00

The usage of function local variables in the protocol state
machine caused an incorrect behaviour when a partial read
from the socket forced the function to return and restart
later when more data was available. At this point the local
variables contained incorrect data.

> Change-Id: I4db1f4ef5c46a3d2d7f7c5328e906188c3af49e6
> BUG: 1334285
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: http://review.gluster.org/14270
> Reviewed-by: Raghavendra G 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Tested-by: Raghavendra G 
> CentOS-regression: Gluster Build System 

Change-Id: I92b7c91b4c0dfc15224aea39308c93b27028dd4f
BUG: 1334287
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/14293
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

glusterd: add defence mechanism to avoid brick port clashes

2016-05-10T07:11:33+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Id7d8351a0082b44310177e714edc0571ad0f7195
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14235
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

rpc: define client port range

2016-05-10T07:11:20+00:00

Problem:
when bind-insecure is 'off', all the clients bind to secure ports,
if incase all the secure ports exhaust the client will no more bind
to secure ports and tries gets a random port which is obviously insecure.

we have seen the client obtaining a port number in the range 49152-65535
which are actually reserved as part of glusterd's pmap_registry for bricks,
hence this will lead to port clashes between client and brick processes.

Solution:
If we can define different port ranges for clients incase where secure ports
exhaust, we can avoid the maximum port clashes with in gluster processes.

Still we are prone to have clashes with other non-gluster processes, but
the chances being very low, but that's a different story on its own,
which will be handled in upcoming patches.

Backportof:
> Change-Id: Ib5ce05991aa1290ccb17f6f04ffd65caf411feaf
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/13998
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Raghavendra G 

Change-Id: I2ab9608ddbefcdf5987d817c23dd066010148e19
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14234
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

protocol: add setactivelk () fop

2016-05-02T01:04:52+00:00

Change-Id: I60fe2d59c454095febce4c0fbef87a2dad9636e4
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14013
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: add setactivelk () fop

2016-05-02T01:04:42+00:00

Change-Id: Ic2ba77a1fdd27801a6e579e04e6c0dd93cd7127b
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14011
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System