glusterfs.git/xlators/cluster, branch v3.8.5

dht: udpate stbuf from servers those have layout

2016-09-29T09:37:15+00:00

Problem: For healing of uid/gid we check if local->stbuf.ia_ctime is
lesser than stbuf->ia_ctime (received from brick). If yes then uid/gid
is updated to local->prebuf(source of healing).

But we merge local->stbuf also form the newly added brick. So if we
receive response from the newly added brick first and update the
local->stbuf, then local->prebuf will remain empty since the newly added
brick will have the latest ctime among all servers. And this can result
in healing wrong uid/gids to the rest of servers.

Hence, we should update local->stbuf from servers with a layout which
will ignore merging stbufs from newly added bricks.

> Reviewed-on: http://review.gluster.org/15126
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 36af81ac7cb2d459f9bfc0c436f0038a68f85235)

Change-Id: If4b64f75a0ea669abdbe9f5a3d1d18ff19374c2f
BUG: 1375096
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15464
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

afr: Ignore gluster internal (virtual) xattrs in metadata heal check

2016-09-29T07:10:18+00:00

Backport of http://review.gluster.org/#/c/15548/

Problem:
In arbiter configuration, posix-xlator in the arbiter brick always sets the
GF_CONTENT_KEY in the response dict with a value 0. If the file size on the
data bricks is more than quick-read's max-file-size (64kb default), those
bricks don't set the key. Because of this difference in the no. of dict
elements, afr triggers metadata heal in lookup code path, in turn leading to
extra lookups+inodelks.

Fix:
Changed afr dict comparison logic to ignore all virtual xattrs and the on-disk
ones that we should not be healing.

Change-Id: I05730bdd39d8fb0b9a49a5fc9c0bb01f0d3bb308
BUG: 1377193
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15578
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

dht: "replica.split-brain-status" attribute value is not correct

2016-09-27T06:49:45+00:00

Problem: In a distributed-replicate volume attribute
         "replica.split-brain-status" value does not display split-brain
           condition though directory is in split-brain.
         If directory is in split brain on mutiple replica-pairs
         it does not show full list of replica pairs.

Solution: Update the dht_aggregate code to aggregate the xattr
          value in this specific condition.

Fix:      1) function getChoices returns the choices from split-brain
             status string.
          2) function add_opt adding the choices to local buffer to
             store in dictionary
          3) For the key "replica.split-brain-status" function dht_aggregate
             call dht_aggregate_split_brain_xattr to prepare the list.

Test:     To verify the patch followed below steps
          1) Create a distributed replica volume and create mount point
          2) Stop heal daemon
          3) Touch file and directories on mount point
             mkdir test{1..5};touch tmp{1..5}
          4) Down brick process on one of the replica set
             pkill -9 glusterfsd
          5) Change permission of dir on mount point
             chmod 755 test{1..5}
          6) Restart brick process on node with force option
          7) kill brick process on other node in same replica set
          8) Change permission of dir again on mount point
             chmod 766 test{1..5}
          9) Reexecute same step from 4-9 on other replica set also
          10) After check heal status on server it will show dir's are
              in split brain on all replica sets
          11) After check the replica.split-brain-status attr on mount
              point it will show wrong status of split brain.
          12) After apply the patch the attribute shows correct value.

> Change-Id: Icdfd72005a4aa82337c342762775a3d1761bbe4a
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15201
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit c4e9ec653c946002ab6d4c71ee8e6df056438a04)

Change-Id: I85a5ae60189066d9e80799f00f1352c2f33ef4f8
Backport of commit c4e9ec653c946002ab6d4c71ee8e6df056438a04
BUG: 1375098
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15467
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: heal root permission post add-brick

2016-09-14T06:33:06+00:00

Post add-brick event the new brick will have permission of 755
by default. If the root directory permission was other than 755,
that does not get healed to the new brick leading to permission
errors/inconsistencies.

For choosing source of attr heal we can trust the subvols which
have layouts with latest ctime(as part of missing directory heal,
we heal the proper attr). In case none of the subvols have layout,
return ESTALE to retrigger a fresh lookup.

Note: This patch heals the permission of the root directories only.
Since, permission healing of directory is not straight forward and
required intrusive fix, those are not addressed here.

> Reviewed-on: http://review.gluster.org/15195
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
>Reviewed-by: Raghavendra G 
(cherry picked from commit 801cd07a4c6ec65ff930b2ae6bb5e405ccd03334)

Change-Id: If894e3895d070d46b62d2452e52c1eaafcf56c29
BUG: 1374573
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15465
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: Skip layout overlap maximization on weighted rebalance

2016-09-14T05:20:46+00:00

During a fix-layout, dht_selfheal_layout_maximize_overlap () does not
consider chunk sizes while calculating layout overlaps, causing smaller
bricks to sometimes get larger ranges than larger bricks. Temporarily
enabling this operation if only if weighted rebalance is disabled
or all bricks are the same size.

> Change-Id: I5ed16cdff2551b826a1759ca8338921640bfc7b3
> BUG: 1366494
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15403
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 

(cherry picked from commit b93692cce603006d9cb6750e08183bca742792ac)

Change-Id: Icf0dd83f36912e721982bcf818a06c4b339dc974
BUG: 1374135
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15422
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/dht: move layout logs to DEBUG level

2016-09-07T14:57:17+00:00

> Reviewed-on: http://review.gluster.org/15343
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: N Balachandran 
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Signed-off-by: Susant Palai 
(cherry picked from commit 15c790b502ba92caa17f2d1870c3d75d547e6bad)

Change-Id: Iad96256218be643b272762b5638a3f6837aff28d
BUG: 1366496
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15413
Reviewed-by: N Balachandran 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quotad: fix potential buffer overflows

2016-08-27T11:10:42+00:00

This converts sprintf to gf_asprintf in following components:                                                                                                          * quotad.c
* dht
* afr
* protocol/client
* rpc/rpc-lib
* rpc/rpc-transport

        This is a backport of http://review.gluster.org/#/c/14102/

> Change-Id: If8a267bab3d91003bdef3a92664077a0136745ee
> BUG: 1332073
> Signed-off-by: Raghavendra G 
> Reviewed-on: http://review.gluster.org/14102
> Tested-by: Manikandan Selvaganesh 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 

Change-Id: If8a267bab3d91003bdef3a92664077a0136745ee
BUG: 1366746
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15325
Smoke: Gluster Build System 
Tested-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/afr: copy loc before passing to syncop

2016-08-23T15:08:42+00:00

Problem:
When io-threads is enabled on the client side, io-threads destroys the
call-stub in which the loc is stored as soon as the c-stack unwinds.
Because afr is creating a syncop with the address of loc passed in
setxattr by the time syncop tries to access it, io-threads would have
already freed the call-stub. This will lead to crash.

Fix:
Copy loc to frame->local and use it's address.

> Reviewed-on: http://review.gluster.org/15070
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Ravishankar N 

BUG: 1369042
Change-Id: I16987e491e24b0b4e3d868a6968e802e47c77f7a
Signed-off-by: Pranith Kumar K 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15233
Smoke: Gluster Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T10:22:36+00:00

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

> Change-Id: I9ca634b026ac830b172bac076437cc3bf1ae7d8a
> BUG: 1363721
> Signed-off-by: Krutika Dhananjay 
> Reviewed-on: http://review.gluster.org/15080
> Tested-by: Pranith Kumar Karampuri 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Oleksandr Natalenko 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit fcb5b70b1099d0379b40c81f35750df8bb9545a5)

Change-Id: I157f1025aebd6624fa3d412abc69a4ae6f2fe9e0
BUG: 1367272
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15221
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/afr: Bug fixes in txn codepath

2016-08-17T10:22:15+00:00

        Backport of: http://review.gluster.org/15145

AFR sets transaction.pre_op[] array even before actually doing the
pre-op on-disk. Therefore, AFR must not only consider the pre_op[] array
but also the failed_subvols[] information before setting the pre_op_done[]
flag. This patch fixes that.

Change-Id: I726b2acd4025e2e75a87dea547ca6e088bc82c00
BUG: 1367272
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15164
Reviewed-by: Ravishankar N 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
Reviewed-by: Anuradha Talur 
CentOS-regression: Gluster Build System