glusterfs.git/xlators, branch v3.8.5

dht: udpate stbuf from servers those have layout

2016-09-29T09:37:15+00:00

Problem: For healing of uid/gid we check if local->stbuf.ia_ctime is
lesser than stbuf->ia_ctime (received from brick). If yes then uid/gid
is updated to local->prebuf(source of healing).

But we merge local->stbuf also form the newly added brick. So if we
receive response from the newly added brick first and update the
local->stbuf, then local->prebuf will remain empty since the newly added
brick will have the latest ctime among all servers. And this can result
in healing wrong uid/gids to the rest of servers.

Hence, we should update local->stbuf from servers with a layout which
will ignore merging stbufs from newly added bricks.

> Reviewed-on: http://review.gluster.org/15126
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 36af81ac7cb2d459f9bfc0c436f0038a68f85235)

Change-Id: If4b64f75a0ea669abdbe9f5a3d1d18ff19374c2f
BUG: 1375096
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15464
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

afr: Ignore gluster internal (virtual) xattrs in metadata heal check

2016-09-29T07:10:18+00:00

Backport of http://review.gluster.org/#/c/15548/

Problem:
In arbiter configuration, posix-xlator in the arbiter brick always sets the
GF_CONTENT_KEY in the response dict with a value 0. If the file size on the
data bricks is more than quick-read's max-file-size (64kb default), those
bricks don't set the key. Because of this difference in the no. of dict
elements, afr triggers metadata heal in lookup code path, in turn leading to
extra lookups+inodelks.

Fix:
Changed afr dict comparison logic to ignore all virtual xattrs and the on-disk
ones that we should not be healing.

Change-Id: I05730bdd39d8fb0b9a49a5fc9c0bb01f0d3bb308
BUG: 1377193
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/15578
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterd: fix return val in glusterd_op_volume_dict_uuid_to_hostname ()

2016-09-29T05:28:45+00:00

glusterd_op_volume_dict_uuid_to_hostname () ignores few dict_get_str failures
but doesn't reset the ret value to 0.

Reproducer steps:
1. Configure a volume and start it (gNFS has to be disabled)
2. gluster volume status

An warning log is observed with:
[MSGID: 106217] [glusterd-op-sm.c:4706:glusterd_op_modify_op_ctx] 0-management:
Failed uuid to hostname conversion
[2016-09-22 09:21:38.537533] W [MSGID: 106387]
[glusterd-op-sm.c:4812:glusterd_op_modify_op_ctx] 0-management: op_ctx
modification failed

>Reviewed-on: http://review.gluster.org/15547
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Samikshan Bairagya 
>Reviewed-by: Kaushal M 

Change-Id: I1d3aa79304d83a9d5db1b7198773d8c2780e24a9
BUG: 1379284
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15571
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai

dht: "replica.split-brain-status" attribute value is not correct

2016-09-27T06:49:45+00:00

Problem: In a distributed-replicate volume attribute
         "replica.split-brain-status" value does not display split-brain
           condition though directory is in split-brain.
         If directory is in split brain on mutiple replica-pairs
         it does not show full list of replica pairs.

Solution: Update the dht_aggregate code to aggregate the xattr
          value in this specific condition.

Fix:      1) function getChoices returns the choices from split-brain
             status string.
          2) function add_opt adding the choices to local buffer to
             store in dictionary
          3) For the key "replica.split-brain-status" function dht_aggregate
             call dht_aggregate_split_brain_xattr to prepare the list.

Test:     To verify the patch followed below steps
          1) Create a distributed replica volume and create mount point
          2) Stop heal daemon
          3) Touch file and directories on mount point
             mkdir test{1..5};touch tmp{1..5}
          4) Down brick process on one of the replica set
             pkill -9 glusterfsd
          5) Change permission of dir on mount point
             chmod 755 test{1..5}
          6) Restart brick process on node with force option
          7) kill brick process on other node in same replica set
          8) Change permission of dir again on mount point
             chmod 766 test{1..5}
          9) Reexecute same step from 4-9 on other replica set also
          10) After check heal status on server it will show dir's are
              in split brain on all replica sets
          11) After check the replica.split-brain-status attr on mount
              point it will show wrong status of split brain.
          12) After apply the patch the attribute shows correct value.

> Change-Id: Icdfd72005a4aa82337c342762775a3d1761bbe4a
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15201
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit c4e9ec653c946002ab6d4c71ee8e6df056438a04)

Change-Id: I85a5ae60189066d9e80799f00f1352c2f33ef4f8
Backport of commit c4e9ec653c946002ab6d4c71ee8e6df056438a04
BUG: 1375098
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15467
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

performance/open-behind: Pass O_DIRECT flags for anon fd reads when required

2016-09-26T04:42:23+00:00

        Backport of: http://review.gluster.org/15537
        cherry-picked from a412a4f50d8ca2ae68dbfa93b80757889150ce99

Writes are already passing the correct flags at the time of open().

Also, make io-cache honor direct-io for anon-fds with
O_DIRECT flag during reads.

Change-Id: I9eb89c3bda34f9287861eb3b53c3d6a7b967c105
BUG: 1375959
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15552
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

Tier: failing detach commit on detach failure and in-progress

2016-09-16T12:56:53+00:00

        back-port of:http://review.gluster.org/#/c/15438/

PROBLEM: if detach status has failed or if it remains in progress
we allow detach commit to happen. only detach force should
be allowed.

FIX: check the detach status for failure or inprogress and disallow
with the apt error message.

> Change-Id: Ib97d540fec67717bb55c18d133187c665cf69ef1
> BUG: 1374584
> Signed-off-by: hari gowtham 
> Reviewed-on: http://review.gluster.org/15438
> Smoke: Gluster Build System 
> Tested-by: hari gowtham 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

Change-Id: I4098c45f5338f8bc4262e75fa6630dabc755e133
BUG: 1375565
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/15492
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright

system/posix-acl: Unwind with NULL xdata on error

2016-09-15T16:30:50+00:00

In posix-acl when there are errors xdata that comes as part of input is used to
unwind which can be used as response xdata which may lead to problems as the
keys in the input will match with keys in the output but the values the
response xdata may expect can be completely different.

For example, we see that dht sends DHT_IATT_IN_XDATA_KEY in setxattr
which will be unwound with the same key in the xdata-response which
dht thinks is valid response and fills stbuf with invalid values
leading to EIO

> BUG: 1374093
> Change-Id: I6b77a1fa1ee99cb62e181e1db2e6fea73f6eaaa3
> Signed-off-by: Pranith Kumar K 
> Reviewed-on: http://review.gluster.org/15421
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Niels de Vos 
(cherry picked from commit c9271ff14d3efa8279cf67907548b3f43970d4fb)

Change-Id: I6b77a1fa1ee99cb62e181e1db2e6fea73f6eaaa3
BUG: 1374640
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15473
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/dht: heal root permission post add-brick

2016-09-14T06:33:06+00:00

Post add-brick event the new brick will have permission of 755
by default. If the root directory permission was other than 755,
that does not get healed to the new brick leading to permission
errors/inconsistencies.

For choosing source of attr heal we can trust the subvols which
have layouts with latest ctime(as part of missing directory heal,
we heal the proper attr). In case none of the subvols have layout,
return ESTALE to retrigger a fresh lookup.

Note: This patch heals the permission of the root directories only.
Since, permission healing of directory is not straight forward and
required intrusive fix, those are not addressed here.

> Reviewed-on: http://review.gluster.org/15195
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
>Reviewed-by: Raghavendra G 
(cherry picked from commit 801cd07a4c6ec65ff930b2ae6bb5e405ccd03334)

Change-Id: If894e3895d070d46b62d2452e52c1eaafcf56c29
BUG: 1374573
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15465
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: Skip layout overlap maximization on weighted rebalance

2016-09-14T05:20:46+00:00

During a fix-layout, dht_selfheal_layout_maximize_overlap () does not
consider chunk sizes while calculating layout overlaps, causing smaller
bricks to sometimes get larger ranges than larger bricks. Temporarily
enabling this operation if only if weighted rebalance is disabled
or all bricks are the same size.

> Change-Id: I5ed16cdff2551b826a1759ca8338921640bfc7b3
> BUG: 1366494
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15403
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 

(cherry picked from commit b93692cce603006d9cb6750e08183bca742792ac)

Change-Id: Icf0dd83f36912e721982bcf818a06c4b339dc974
BUG: 1374135
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15422
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

feature/bitrot: Fix recovery of corrupted hardlink

2016-09-09T14:05:06+00:00

Problem:
When a file with hardlink is corrupted in ec volume,
the recovery steps mentioned was not working.
Only name and metadata was healing but not the data.

Cause:
The bad file marker in the inode context is not removed.
Hence when self heal tries to open the file for data
healing, it fails with EIO.

Background:
The bitrot deletes inode context during forget.

Briefly, the recovery steps involves following steps.
   1. Delete the entry marked with bad file xattr
      from backend. Delete all the hardlinks including
      .glusters hardlink as well.
   2. Access the each hardlink of the file including
      original from the mount.

The step 2 will send lookup to the brick where the files
are deleted from backend and returns with ENOENT. On
ENOENT, server xlator forgets the inode if there are
no dentries associated with it. But in case hardlinks,
the forget won't be called as dentries (other hardlink
files) are associated with the inode. Hence bitrot stube
won't delete it's context failing the data self heal.

Fix:
Bitrot-stub should delete the inode context on getting
ENOENT during lookup.

>Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1
>BUG: 1373520
>Signed-off-by: Kotresh HR 
>Reviewed-on: http://review.gluster.org/15408
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra Bhat 
(cherry picked from commit b86a7de9b5ea9dcd0a630dbe09fce6d9ad0d8944)


Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1
BUG: 1374565
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15433
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat