glusterfs.git/xlators/cluster/dht, branch v3.12.5

cluster/dht: Use percentages for space check

2018-01-12T05:37:34+00:00

With heterogenous bricks now being supported in DHT
we could run into issues where files are not migrated
even though there is sufficient space in newly added bricks
which just happen to be considerably smaller than older
bricks. Using percentages instead of absolute available
space for space checks can mitigate that to some extent.

Marking bug-1247563.t as that used to depend on the easier
code to prevent a file from migrating. This will be removed
once we find a way to force a file migration failure.

> Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
> BUG: 1529440
> Signed-off-by: N Balachandran 

Change-Id: I3452520511f304dbf5af86f0632f654a92fcb647
BUG: 1530455
Signed-off-by: N Balachandran

cluster/dht: don't overfill the buffer in readdir(p)

2017-12-08T05:36:52+00:00

Superflous dentries that cannot be fit in the buffer size provided by
kernel are thrown away by fuse-bridge. This means,

* the next readdir(p) seen by readdir-ahead would have an offset of a
dentry returned in a previous readdir(p) response. When readdir-ahead
detects non-monotonic offset it turns itself off which can result in
poor readdir performance.

* readdirp can be cpu-intensive on brick and there is no point to read
 all those dentries just to be thrown away by fuse-bridge.

So, the best strategy would be to fill the buffer optimally - neither
overfill nor underfill.

> Change-Id: Idb3d85dd4c08fdc4526b2df801d49e69e439ba84
> BUG: 1492625
> Signed-off-by: Raghavendra G 
(cherry picked from commit e785faead91f74dce7c832848f2e8f3f43bd0be5)
Change-Id: Idb3d85dd4c08fdc4526b2df801d49e69e439ba84
BUG: 1478411
Signed-off-by: Raghavendra G

cluster/dht: populate inode in dentry for single subvolume dht

2017-12-06T10:08:54+00:00

... in readdirp response if dentry points to a directory inode. This
is a special case where the entire layout is stored in one single
subvolume and hence no need for lookup to construct the layout

>Change-Id: I44fd951e2393ec9dac2af120469be47081a32185
>BUG: 1492625
>Signed-off-by: Raghavendra G 

(cherry picked from commit 59d1cc720f52357f7a6f20bb630febc6a622c99c)
Change-Id: I44fd951e2393ec9dac2af120469be47081a32185
BUG: 1478411
Signed-off-by: Raghavendra G

cluster/dht: make rebalance use truncate incase

2017-11-23T09:46:16+00:00

..
the brick file system does not support fallocate.

> Change-Id: Id76cda2d8bb3b223b779e5e7a34f17c8bfa6283c
> BUG: 1488103
> Signed-off-by: Susant Palai 

Change-Id: Id76cda2d8bb3b223b779e5e7a34f17c8bfa6283c
BUG: 1516691
Signed-off-by: Susant Palai

cluster/dht: Don't set ACLs on linkto file

2017-11-20T03:39:52+00:00

The trusted.SGI_ACL_FILE appears to set posix
ACLs on the linkto file that is a target of
file migration. This can mess up file permissions
and cause linkto identification to fail.
Now we remove all ACL xattrs from the results of
the listxattr call on the source before setting them
on the target.

> BUG: 1514329
> Signed-off-by: N Balachandran 

Change-Id: I56802dbaed783a16e3fb90f59f4ce849f8a4a9b4
BUG: 1515042
Signed-off-by: N Balachandran

cluster/dht: fix crash when deleting directories

2017-10-25T11:36:23+00:00

In DHT, after locks on all subvolumes are acquired, it would perform the
following steps sequentially,
1. send remove dir on all other subvolumes except the hashed one in a loop;
2. wait for all pending rmdir to be done
3. remove dir on the hashed subvolume

The problem is that in step 1 there is a check to skip hashed subvolume
in the loop. If the last subvolume to check is actually the
hashed one, and step 3 is quickly done before the last and hashed
subvolume is checked, by accessing shared context data be destroyed in
step 3, would cause a crash.

Fix by saving shared data in a local variable to access later in the
loop.

> BUG: 1490642
> Signed-off-by: Zhang Huan 
(cherry picked from commit 206120126d455417a81a48ae473d49be337e9463)
Change-Id: I8db7cf7cb262d74efcb58eb00f02ea37df4be4e2
BUG: 1505221
Signed-off-by: N Balachandran

cluster/dht: Don't store the entire uuid for subvols

2017-10-12T18:25:29+00:00

Comparing the uuid string of the local node against that stored in the
local_subvol information is inefficient, especially as it is
done for every file to be migrated. The code has now been changed
to set the value of info to 1 if the nodeuuid is that of the node
making the comparison so this becomes an integer comparison.

> BUG: 1451434
> Signed-off-by: N Balachandran 
> https://review.gluster.org/#/c/17851

(cherry picked from commit c4a608799a577a4f38139f6bb8a47da8efb0fec3)
Change-Id: I7491d59caad3b71dbf5facc94dcde0cd53962775
BUG: 1500472
Signed-off-by: N Balachandran

dht: add FOP check to dht_file_setattr_cbk

2017-09-10T18:32:02+00:00

Problem:
bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a
non fd-based fop, namely setattr. This caused
dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL.

Fix:
Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk
only for dht_fsetattr and not dht_setattr or dht_setattr2

> Reviewed-on: https://review.gluster.org/18208
> Smoke: Gluster Build System 
> Reviewed-by: Susant Palai 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Raghavendra G 
> Reviewed-by: N Balachandran 
> CentOS-regression: Gluster Build System 
(cherry picked from commit 47188e9eac59de416a5c86c7ec7540ed6aaa1c98)

Signed-off-by: Ravishankar N 
Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99
BUG: 1489260
Reviewed-on: https://review.gluster.org/18222
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

cluster/dht: Log files skipped by rebalance

2017-09-07T07:09:20+00:00

There was no easy way to find out which files were
skipped during a rebalance.
Rebalance now logs a message for every skipped file
using msgid 109126, making it easier to find
all files that were skipped.

> BUG: 1480445
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18021
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: hari gowtham 
> Reviewed-by: Raghavendra G 

Change-Id: I2cac7db7285e2f82354251f3ea4094827b0daf3e
BUG: 1486557
(cherry picked from commit a4c43ba9374b8f75a48d38a032353a0c7d311a73)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18149
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

cluster/dht: Aggregate xattrs only for dirs in dht_discover_cbk

2017-09-07T07:07:55+00:00

If dht_discover finds data files on more than one subvol,
racing calls to dht_discover_cbk could end up calling
dht_aggregate_xattr which could delete dictionary data
that is being accessed by higher layer translators.
Fixed to call dht_aggregate_xattr only for directories and
consider only the first file to be found.

> BUG: 1484709
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/18137
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I4f3d2a405ec735d4f1bb33a04b7255eb2d179f8a
BUG: 1486538
(cherry picked from commit 9420022df0962b1fa4f3ea3774249be81bc945cc)
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18146
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System