glusterfs.git/rpc, branch v3.8.3

rpc/socket.c : Modify socket_poller code in case of ENODATA error code.

2016-07-26T17:07:37+00:00

Problem:  Polling failure errors are coming till volume is not come while
          SSL is enabled.

Solution: To avoid the message update one condition in socket_poller code
          It will not exit from thread in case of received ENODATA from
          ssl_do function.
Backport of commit 84e9fc2fb5fabf9d1e553a420854a306cdb8a168

> Change-Id: Ia514e99b279b07b372ee950f4368ac0d9c702d82
> BUG: 1349709
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/14786
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> (cherry picked from commit 84e9fc2fb5fabf9d1e553a420854a306cdb8a168)

BUG: 1359654
Signed-off-by: Mohit Agrawal 

Change-Id: If1820c0b3d0cd976875137bc1175d4b2008779af
Reviewed-on: http://review.gluster.org/14999
Tested-by: MOHIT AGRAWAL 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-25T06:40:30+00:00

Backport of: http://review.gluster.org/13658

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359364
Signed-off-by: Kotresh HR 
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14994
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

rpc/socket: pthread resources are not cleaned up

2016-07-22T12:47:11+00:00

A socket_connect failure creates a new pthread which
is not a detached thread. As no pthread_join is called,
the thread resources are not cleaned up causing a memory leak.

Now, socket_connect creates a detached thread to handle failure.

> Change-Id: Idbf25d312f91464ae20c97d501b628bfdec7cf0c
> BUG: 1343374
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/14875
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit 9886d568a7a8839bf3acc81cb1111fa372ac5270)

Change-Id: I69ef46013c8dbc70cbda2695f12be1f6d3720055
BUG: 1354250
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/14979
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

rpc/socket.c: Modify approach to cleanup threads of socket_poller in socket_spawn.

2016-07-21T14:24:48+00:00

Problem: Current approach to cleanup threads of socket_poller is not appropriate.

Solution: Enable detach flag at the time of thread creation in socket_spawn.

Fix: Write a new wrapper(gf_create_detach_thread) to create detachable thread
     instead of store thread ids in a queue.

Test: Fix is verfied on gluster process, To test the patch followed below procedure
      Enable the client.ssl and server.ssl option on the volume
      Start the volume and count anon segment in pmap output for glusterd process
      pmap -x  | grep "\[ anon \]" | wc -l
      Stop the volume and check again count of anon segment it should not increase.

Backport of commit 2ee48474be32f6ead2f3834677fee89d88348382

> Signed-off-by: Mohit Agrawal 
> Change-Id: Ib8f7ec7504ec8f6f74b45ce6719b6fb47f9fdc37
> BUG: 1336508
> Reviewed-on: http://review.gluster.org/14694
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1354395
Change-Id: Ibdbbae508d9dda2fd36220a9b1e47f7776336929
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14891
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

socket: log the client identifier in ssl connect

2016-07-16T18:18:47+00:00

Backport of commit d308fb5e152d8c908bf4f5da81f553fbe3d0400a

> Change-Id: I4b463ecafb66de16cbe7ed23fae800bb1204f829
> BUG: 1333912
> Signed-off-by: Raghavendra Bhat 
> Reviewed-on: http://review.gluster.org/14242
> Tested-by: Vijay Bellur 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> Smoke: Gluster Build System 
> (cherry picked from commit d308fb5e152d8c908bf4f5da81f553fbe3d0400a)

Change-Id: Id007d3e28292f504913b7df8b8eb693c0427b22b
BUG: 1351878
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14845
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

rpc: invalid argument when function setsockopt sets option TCP_USER_TIMEOUT

2016-07-12T12:07:29+00:00

If option "transport.tcp-user-timeout" hasn't been setted, glusterd's
priv->timeout will be -1, which will cause invalid argument when
set TCP_USER_TIMEOUT.

Cherry picked from commit b2c73cbf423de6201f956f522b7429615c88869d:
> Change-Id: Ibc16264ceac0e69ab4a217ffa27c549b9fa21df9
> BUG: 1349657
> Signed-off-by: Zhou Zhengping 
> Reviewed-on: http://review.gluster.org/14785
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: Ibc16264ceac0e69ab4a217ffa27c549b9fa21df9
BUG: 1354405
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14888
Reviewed-by: Zhou Zhengping 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Tested-by: Gluster Build System 
Smoke: Gluster Build System

xdr/nfs: free complete groupnode structure

2016-06-13T10:10:48+00:00

The groupnode->gr_next pointer is not traversed upon free. This is
currently not a problem, because the pointer is never used. However the
correct way to free a groupnode should check the ->gr_next pointer and
free any of the groups that it encounters.

This problem was identified while correcting a problem with the MOUNT
protocol. The change "nfs: build exportlist with multiple groups" starts
to use ->gr_next.

This is backport of below mainline fix -
	http://review.gluster.org/#/c/14666/

Change-Id: I9d04eaf4c65bdb8db136321d60e70789da1739d7
BUG: 1343287
Signed-off-by: Niels de Vos 
Signed-off-by: Bipin Kunal 
Reviewed-on: http://review.gluster.org/14699
Smoke: Gluster Build System 
Reviewed-by: bipin kunal 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol: Add framework to send transaction id with recall

2016-06-10T11:03:54+00:00

Backport of: http://review.gluster.org/#/c/14647/

Issue:
The upcall(cache invalidation/recall) event is sent from the bricks
to clients. In AFR/EC setup, it can so happen that all the bricks
will send the upcall for the same event, and if AFR/EC doesn't filter
out these duplicate notifications, the logic above cluster xlators
can fail.

Solution:
Use transaction id to filter out duplicate notifications.

This patch adds framework for duplicate notifications.
AFR/EC can build up on this patch for deduping the notifications

Change-Id: I66b08e63b8799bc5932f2b2545376138a5701168
BUG: 1337638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14648
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs (timer): race conditions, illegal mem access, mem leak

2016-06-08T12:15:20+00:00

While investigating gfapi memory consumption with valgrind, valgrind
reported several memory access issues.

Also see the timer 'registry' being recreated (shortly) after being
freed during teardown due to the way it's currently written.

Passing ctx as data to gf_timer_proc() is prone to memory access
issues if ctx is freed before gf_timer_proc() terminates. (And in
fact this does happen, at least in valgrind.) gf_timer_proc() doesn't
need ctx for anything, it only needs ctx->timer, so just pass that.

Nothing ever calls gf_timer_registry_init(). Nothing outside of
timer.c that is. Making it and gf_timer_proc() static.

backport mainline:
> http://review.gluster.org/14247
> BUG: 1333925

Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a
BUG: 1342620
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14644
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: Honour mandatory lock flags during lock migration

2016-05-27T14:50:37+00:00

lk_flags from posix_lock_t structure is the primary key used to
differentiate locks as either advisory and mandatory type. During
lock migration this field is not read in getactivelk() call path.
So in order to copy the exact lock state from source to destination
it is necessary to include lk_flags within lock_migration_info_t
structure to maintain accurate state. This change also includes
minor modifications to setactivelk() call to consider lk_flags
during lock migration.

> Reviewed-on: http://review.gluster.org/14189
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Poornima G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

(cherry picked from commit deaf8439fc42435988aae6a7b9ab681cc0d36b09)

Change-Id: I20a7b6b6a0f3bdac5734cce8a2cd2349eceff195
BUG: 1337805
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14457
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri