glusterfs.git/rpc, branch v3.9.0rc2

rpc/socket: Close pipe on disconnection

2016-10-25T11:09:08+00:00

Encrypted connections create a pipe, which isn't closed when the
connection disconnects. This leaks fds, and gluster eventually ends up
in a situation with fd starvation which leads to operation failures.

> Change-Id: I144e1f767cec8c6fc1aa46b00cd234129d2a4adc
> BUG: 1336371
> Signed-off-by: Kaushal M 
> Reviewed-on: http://review.gluster.org/14356
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 
> Signed-off-by: Kaushal M 

Change-Id: I144e1f767cec8c6fc1aa46b00cd234129d2a4adc
BUG: 1387960
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/15702
Tested-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

rpc: Fix the race between notification and reconnection

2016-10-25T11:00:46+00:00

Problem:
There was a hang because unlock on an entry failed with
ENOTCONN.
Client thinks the connection is down where as server thinks
the connection is up.

This is the race we are seeing:
1) Connection from client to the brick disconnects.
2) Saved frames unwind is called which unwinds all
   frames that were wound before disconnect.
3) connection from client to the brick happens and
   setvolume.
4) Disconnect notification for the connection in 1)
   comes now and calls client_rpc_notify() which
   marks the connection to be offline even when the
   connection is up.

This is happening because I/O can retrigger connection
before disconnect notification is sent to the higher
layers in rpc.

Fix:
Notify the higher layers that a disconnect happened and then
go ahead with reconnect logic.

For the logs which point to the information above check:
https://bugzilla.redhat.com/show_bug.cgi?id=1386626#c1

Thanks to Raghavendra G for suggesting the correct fix.

 >BUG: 1386626
 >Change-Id: I3c84ba1f17010bd69049fa88ec5f0ae431f8cda9
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15681
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Niels de Vos 
 >CentOS-regression: Gluster Build System 
 >Smoke: Gluster Build System 
 >Reviewed-by: Raghavendra G 
 >(cherry picked from commit a6b63e11b7758cf1bfcb67985e25ec02845f0995)

Change-Id: Ifa721193c26b70e26b47b7698c077da0ad5f2e1d
BUG: 1388323
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15717
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

compound fops: Fix file corruption issue

2016-10-25T04:14:54+00:00

        Backport of: http://review.gluster.org/#/c/15654/

1. Address of a local variable @args is copied into state->req
in server3_3_compound (). But even after the function has gone out of
scope, in server_compound_resume () this pointer is accessed and
dereferenced. This patch fixes that.

2. Compound fops, by virtue of NOT having a vector sizer (like the one
writev has), ends up having both the header and the data (in case one of
its member fops is WRITEV) in the same hdr_iobuf. This buffer was not
being preserved through the lifetime of the compound fop, causing it to
be overwritten by a parallel write fop, even when the writev associated
with the currently executing compound fop is yet to hit the desk, thereby
corrupting the file's data. This is fixed by associating the hdr_iobuf with
the iobref so its memory remains valid through the lifetime of the fop.

3. Also fixed a use-after-free bug in protocol/client in compound fops cbk,
missed by Linux but caught by NetBSD.

Finally, big thanks to Pranith Kumar K and Raghavendra Gowdappa for their
help in debugging this file corruption issue.

Change-Id: I58da39ae544ad81192849926399a971c4c01c986
BUG: 1387984
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15709
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

rpc/socket.c : Modify socket_poller code in case of ENODATA error code.

2016-10-24T13:46:51+00:00

Problem:  Continuous warning message(ENODATA) are coming in socket_rwv
          while SSL is enabled.

Solution: To avoid the warning message update one condition in
          socket_poller loop code before break from loop in case
          of error returned by poll functions.

> BUG: 1386450
> Change-Id: I19b3a92d4c3ba380738379f5679c1c354f0ab9b1
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15677
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit ec64ce2e1684003f4e7a20d4372e414bfbddb6fb)

BUG: 1387975
Change-Id: Ia61e6576d16f377850dc43991180571b2ed6997f
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15711
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Jeff Darcy

socket: pollerr event shouldn't trigger socket_connnect_finish

2016-09-21T15:21:18+00:00

If connect fails with any other error than EINPROGRESS we cannot get
the error status using getsockopt (... SO_ERROR ... ). Hence we need
to remember the state of connect and take appropriate action in the
event_handler for the same.

As an added note, a event can come where poll_err is HUP and we have
poll_in as well (i.e some status was written to the socket), so for
such cases we need to finish the connect, process the data and then
the poll_err as is the case in the current code.

Special thanks to Kaushal M & Raghavendra G for figuring out the issue.

>Signed-off-by: Shyam 
>Reviewed-on: http://review.gluster.org/15440
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra G 

Change-Id: Ic45ad59ff8ab1d0a9d2cab2c924ad940b9d38528
BUG: 1377386
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15533
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

rpc: increase RPC/XID with each callback

2016-09-21T03:11:47+00:00

The RPC/XID for callbacks has been hardcoded to GF_UNIVERSAL_ANSWER. In
Wireshark these RPC-calls are marked as "RPC retransmissions" because of
the repeating RPC/XID. This is most confusing when verifying the
callbacks that the upcall framework sends. There is no way to see the
difference between real retransmissions and new callbacks.

This change was verified by create and removal of files through
different Gluster clients. The RPC/XID is increased on a per connection
(or client) base. The expectations of the RPC protocol are met this way.

> Change-Id: I2116bec0e294df4046d168d8bcbba011284cd0b2
> BUG: 1377097
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15524
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit e9b39527d5dcfba95c4c52a522c8ce1f4512ac21)

Change-Id: I2116bec0e294df4046d168d8bcbba011284cd0b2
BUG: 1377288
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15527
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

io-stats: Add stats for upcall notifications

2016-09-01T03:18:42+00:00

With this patch, there will be additional entries seen in
the profile info:

UPCALL : Total number of upcall events that were sent from
         the brick(in brick profile), and number of upcall
         notifications recieved by client(in client profile)

Cache invalidation events:
-------------------------
CI_IATT : Number of upcalls that were cache invalidation and
         had one of the IATT_UPDATE_FLAGS set. This indicates
         that one of the iatt value was changed.

CI_XATTR : Number of upcalls that were cache invalidation, and
         had one of the UP_XATTR or UP_XATTR_RM set. This indicates
         that an xattr was updated or deleted.

CI_RENAME : Number of upcalls that were cache invalidation,
         resulted by the renaming of a file or directory

CI_UNLINK : Number of upcalls that were cache invalidation,
         resulted by the unlink of a file.

CI_FORGET : Number of upcalls that were cache invalidation,
         resulted by the forget of inode on the server side.

Lease events:
------------
LEASE_RECALL : Number of lease recalls sent by the brick (in
         brick profile), and number of lease recalls recieved
         by client(in client profile)

Note that the sum of CI_IATT, CI_XATTR, CI_RENAME, CI_UNLINK,
CI_FORGET, LEASE_RECALL may not be equal to UPCALL. This is
because, each cache invalidation can carry multiple flags.
Eg:
- Every CI_XATTR will have CI_IATT
- Every CI_UNLINK will also increment CI_IATT as link count is an
iatt attribute.

Also UP_PARENT_DENTRY_FLAGS is currently not accounted for,
as CI_RENAME and CI_UNLINK will always have the flag
UP_PARENT_DENTRY_FLAGS

Change-Id: Ieb8cd21dde2c4c7618f12d025a5e5156f9cc0fe9
BUG: 1371543
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15193
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterd : Introduce reset brick

2016-08-30T02:55:53+00:00

The command basically allows replace brick with src and
dst bricks as same.

Usage:
gluster v reset-brick   start
This command kills the brick to be reset. Once this command is run,
admin can do other manual operations that they need to do,
like configuring some options for the brick. Once this is done,
resetting the brick can be continued with the following options.

gluster v reset-brick    commit {force}

Does the job of resetting the brick. 'force' option should be used
when the brick already contains volinfo id.

Problem: On doing a disk-replacement of a brick in a replicate volume
the following 2 scenarios may occur :

a) there is a chance that reads are served from this replaced-disk brick,
which leads to empty reads. b) potential data loss if next writes succeed
only on replaced brick, and heal is done to other bricks from this one.

Solution: After disk-replacement, make sure that reset-brick command is
run for that brick so that pending markers are set for the brick and it
is not chosen as source for reads and heal. But, as of now replace-brick
for the same brick-path is not allowed. In order to fix the above
mentioned problem, same brick-path replace-brick is needed.
With this patch reset-brick commit {force} will be allowed even when
source and destination  are identical as long as
1) destination brick is not alive
2) source and destination brick have the same brick uuid and path.
Also, the destination brick after replace-brick will use the same port
as the source brick.

Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de
BUG: 1266876
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/12250
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Pranith Kumar Karampuri

rpc: dead code fix

2016-08-29T16:24:38+00:00

CID: 1357866

Change-Id: I117cfd455b05df8d2bd9c8df4a081ffe4d053d09
BUG: 789278
Signed-off-by: ankit 
Reviewed-on: http://review.gluster.org/15310
Smoke: Gluster Build System 
Tested-by: ankitraj
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

cli,rpc : Removing logically dead code

2016-08-29T12:11:46+00:00

CID = 1357865, 1357866

Change-Id: I05eb345e9357d889872d259a600759392b188bb3
BUG: 789278
Signed-off-by: Muthu-vigneshwaran 
Reviewed-on: http://review.gluster.org/15148
Tested-by: Muthu Vigneshwaran
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Manikandan Selvaganesh