glusterfs.git/xlators/cluster/dht, branch v3.8.7

dht: Proper log message if data migration is skipped

2016-11-03T05:33:02+00:00

There was a misleading message from logs about 
available disk space while rebalancing of bricks
while calculating free space.

Bug: 1390870

Backprt of http://review.gluster.org/#/c/15345/

Change-Id: Ie9df0b2cbf00faaf13a0a3f0dbd657377a082755
Signed-off-by: ankitraj 
Reviewed-on: http://review.gluster.org/15765
Tested-by: ankitraj
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

dht: udpate stbuf from servers those have layout

2016-09-29T09:37:15+00:00

Problem: For healing of uid/gid we check if local->stbuf.ia_ctime is
lesser than stbuf->ia_ctime (received from brick). If yes then uid/gid
is updated to local->prebuf(source of healing).

But we merge local->stbuf also form the newly added brick. So if we
receive response from the newly added brick first and update the
local->stbuf, then local->prebuf will remain empty since the newly added
brick will have the latest ctime among all servers. And this can result
in healing wrong uid/gids to the rest of servers.

Hence, we should update local->stbuf from servers with a layout which
will ignore merging stbufs from newly added bricks.

> Reviewed-on: http://review.gluster.org/15126
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 36af81ac7cb2d459f9bfc0c436f0038a68f85235)

Change-Id: If4b64f75a0ea669abdbe9f5a3d1d18ff19374c2f
BUG: 1375096
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15464
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

dht: "replica.split-brain-status" attribute value is not correct

2016-09-27T06:49:45+00:00

Problem: In a distributed-replicate volume attribute
         "replica.split-brain-status" value does not display split-brain
           condition though directory is in split-brain.
         If directory is in split brain on mutiple replica-pairs
         it does not show full list of replica pairs.

Solution: Update the dht_aggregate code to aggregate the xattr
          value in this specific condition.

Fix:      1) function getChoices returns the choices from split-brain
             status string.
          2) function add_opt adding the choices to local buffer to
             store in dictionary
          3) For the key "replica.split-brain-status" function dht_aggregate
             call dht_aggregate_split_brain_xattr to prepare the list.

Test:     To verify the patch followed below steps
          1) Create a distributed replica volume and create mount point
          2) Stop heal daemon
          3) Touch file and directories on mount point
             mkdir test{1..5};touch tmp{1..5}
          4) Down brick process on one of the replica set
             pkill -9 glusterfsd
          5) Change permission of dir on mount point
             chmod 755 test{1..5}
          6) Restart brick process on node with force option
          7) kill brick process on other node in same replica set
          8) Change permission of dir again on mount point
             chmod 766 test{1..5}
          9) Reexecute same step from 4-9 on other replica set also
          10) After check heal status on server it will show dir's are
              in split brain on all replica sets
          11) After check the replica.split-brain-status attr on mount
              point it will show wrong status of split brain.
          12) After apply the patch the attribute shows correct value.

> Change-Id: Icdfd72005a4aa82337c342762775a3d1761bbe4a
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15201
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> (cherry picked from commit c4e9ec653c946002ab6d4c71ee8e6df056438a04)

Change-Id: I85a5ae60189066d9e80799f00f1352c2f33ef4f8
Backport of commit c4e9ec653c946002ab6d4c71ee8e6df056438a04
BUG: 1375098
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15467
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: heal root permission post add-brick

2016-09-14T06:33:06+00:00

Post add-brick event the new brick will have permission of 755
by default. If the root directory permission was other than 755,
that does not get healed to the new brick leading to permission
errors/inconsistencies.

For choosing source of attr heal we can trust the subvols which
have layouts with latest ctime(as part of missing directory heal,
we heal the proper attr). In case none of the subvols have layout,
return ESTALE to retrigger a fresh lookup.

Note: This patch heals the permission of the root directories only.
Since, permission healing of directory is not straight forward and
required intrusive fix, those are not addressed here.

> Reviewed-on: http://review.gluster.org/15195
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
>Reviewed-by: Raghavendra G 
(cherry picked from commit 801cd07a4c6ec65ff930b2ae6bb5e405ccd03334)

Change-Id: If894e3895d070d46b62d2452e52c1eaafcf56c29
BUG: 1374573
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15465
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

cluster/dht: Skip layout overlap maximization on weighted rebalance

2016-09-14T05:20:46+00:00

During a fix-layout, dht_selfheal_layout_maximize_overlap () does not
consider chunk sizes while calculating layout overlaps, causing smaller
bricks to sometimes get larger ranges than larger bricks. Temporarily
enabling this operation if only if weighted rebalance is disabled
or all bricks are the same size.

> Change-Id: I5ed16cdff2551b826a1759ca8338921640bfc7b3
> BUG: 1366494
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15403
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 

(cherry picked from commit b93692cce603006d9cb6750e08183bca742792ac)

Change-Id: Icf0dd83f36912e721982bcf818a06c4b339dc974
BUG: 1374135
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15422
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/dht: move layout logs to DEBUG level

2016-09-07T14:57:17+00:00

> Reviewed-on: http://review.gluster.org/15343
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: N Balachandran 
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Signed-off-by: Susant Palai 
(cherry picked from commit 15c790b502ba92caa17f2d1870c3d75d547e6bad)

Change-Id: Iad96256218be643b272762b5638a3f6837aff28d
BUG: 1366496
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15413
Reviewed-by: N Balachandran 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quotad: fix potential buffer overflows

2016-08-27T11:10:42+00:00

This converts sprintf to gf_asprintf in following components:                                                                                                          * quotad.c
* dht
* afr
* protocol/client
* rpc/rpc-lib
* rpc/rpc-transport

        This is a backport of http://review.gluster.org/#/c/14102/

> Change-Id: If8a267bab3d91003bdef3a92664077a0136745ee
> BUG: 1332073
> Signed-off-by: Raghavendra G 
> Reviewed-on: http://review.gluster.org/14102
> Tested-by: Manikandan Selvaganesh 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 

Change-Id: If8a267bab3d91003bdef3a92664077a0136745ee
BUG: 1366746
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15325
Smoke: Gluster Build System 
Tested-by: Manikandan Selvaganesh 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/dht: initialize cbk before attempting inode-link

2016-08-16T06:20:40+00:00

Otherwise inode-link failures in selfheal codepath will result in a
crash.

> Change-Id: I9061629ae9d1eb1ac945af5f448d0d8b397a5022
> BUG: 1345748
> Signed-off-by: Raghavendra G 
> Reviewed-on: http://review.gluster.org/14707
> Reviewed-by: N Balachandran 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Poornima G 
> Reviewed-by: Susant Palai 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
(cherry picked from commit a4d35ccb8afeefae4d9cdd36ac19b0e97d0d04d0)

Change-Id: I9061629ae9d1eb1ac945af5f448d00dba97a5022
BUG: 1366482
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15157
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

dht/rebalance: allocate migrator thread pool dynamically

2016-08-10T10:21:54+00:00

Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a
crash or memory corruption can be seen.

Fix: Make the migratior thread pool dynamic.

> Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
> BUG: 1362069
> Reviewed-on: http://review.gluster.org/15000
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b)

Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1362069
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15061
Smoke: Gluster Build System 
Tested-by: N Balachandran 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

cluster/tier: dont promote if estimated block consumption > hi watermark

2016-08-05T08:12:11+00:00

Add test to fail promotion if estimated block consumption grows
beyond hi watermark.

Skip file migrations until next cycle if tier_get_fs_stat() fails
in tier_migrate_using_query_file()

> Reviewed-on: http://review.gluster.org/14780
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: mohammed rafi  kc 
> Reviewed-by: Dan Lambright 

(cherry picked from commit 1f4e41e8c2f5f4af4564caba0a08996853f089f4)

Change-Id: Ice04572fa739c09109c4433e65965197482a7beb
BUG: 1362198
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/15065
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos