glusterfs.git/xlators/cluster/dht/src/dht-helper.c, branch v3.12.2

cluster/dht: Don't store the entire uuid for subvols

2017-10-12T18:25:29+00:00

Comparing the uuid string of the local node against that stored in the
local_subvol information is inefficient, especially as it is
done for every file to be migrated. The code has now been changed
to set the value of info to 1 if the nodeuuid is that of the node
making the comparison so this becomes an integer comparison.

> BUG: 1451434
> Signed-off-by: N Balachandran 
> https://review.gluster.org/#/c/17851

(cherry picked from commit c4a608799a577a4f38139f6bb8a47da8efb0fec3)
Change-Id: I7491d59caad3b71dbf5facc94dcde0cd53962775
BUG: 1500472
Signed-off-by: N Balachandran

cluster/dht: EBADF handling for fremovexattr and fsetxattr

2017-08-12T13:37:31+00:00

Add EBADF handling for dht_fremovexattr and dht_fsetxattr.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17999
> Smoke: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 747a08d34e2a1e94d7fce68a3577370288bb1955)
Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309
BUG: 1479303
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/18012
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Check for open fd only on EBADF

2017-08-09T11:29:50+00:00

DHT fd based fops used to check if the fd was open
on the cached subvol before winding the call. However,
this introduced a performance regression of about
30% for reads.

This check was introduced to handle cases where files
were migrated while IOs were happening. As this is not
the common case, dht will now check if the fd is
open on the cached subvol only if the call fails
with EBADF.

This will prevent a performance hit where a rebalance
is not running.

> BUG: 1476665
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17976
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Amar Tumballi 
> Reviewed-by: Susant Palai 
> Reviewed-by: Raghavendra G 

(cherry picked from commit cdca1cb26a0aba390c6d8485c0d6d95e22ffc8bd)
Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808
BUG: 1479303
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17995
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/dht: Fix fd check race

2017-07-11T05:02:52+00:00

There is a another race between the cached subvol
being updated in the inode_ctx and the fd being opened on
the target.

1. fop1 -> fd1 -> subvol0
2. file migrated from subvol0 to subvol1 and cached_subvol
   changed to subvol1 in inode_ctx
3. fop2 -> fd1 -> subvol1 [takes new cached subvol]
4. fop2 -> checks fd ctx (fd not open on subvol1) -> opens fd1 on subvol1
5. fop1 -> checks fd ctx (fd not open on subvol0)
   -> tries to open fd1 on subvol0 -> fails with "No such file on directory".

Fix:
If dht_fd_open_on_dst fails with ENOENT or ESTALE, wind to old subvol
and let the phase1/phase2 checks handle it.

Change-Id: I34f8011574a8b72e3bcfe03b0cc4f024b352f225
BUG: 1465075
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17731
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Amar Tumballi

cluster/dht: Check if fd is opened on dst subvol

2017-06-28T11:42:21+00:00

If an fd is opened on a file, the file is migrated
and the cached subvol is updated in the inode_ctx
before an fd based fop is sent, the fop is sent to
the dst subvol on which the fd is not opened.
This causes the FOP to fail with EBADF.

Now, every fd based fop will check to see that the fd
has been opened on the dst subvol before winding it down.

Change-Id: Id92ef5eb7a5b5226688e2d2868b15e383f5f240e
BUG: 1465075
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17630
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Susant Palai 
CentOS-regression: Gluster Build System

cluster/dht: Make optimal usage of buffer provided with readdir(p)

2017-05-31T14:13:01+00:00

dht_readdirp must unwind with list of entries only after
the entire buffer requested by kernel is filled to avoid
extra syscalls occuring when returning partially filled
buffer. Also wind readdir call to next subvol on reaching
EOD for directory on that subvol to avoid extra network call.

Change-Id: If2e1a2722f813d95457c7542bff25fef56c7a041
BUG: 1356453
Signed-off-by: Sakshi 
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/12271
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Susant Palai

cluster/dht: Rebalance on all nodes should migrate files

2017-05-16T16:01:39+00:00

Problem:
Rebalance compares the node-uuid of a file against its own
to and migrates a file only if they match. However, the
current behaviour in both AFR and EC is to return
the node-uuid of the first brick in a replica set for all
files. This means a single node ends up migrating all
the files if the first brick of every replica set is on the
same node.

Fix:
AFR and EC will return all node-uuids for the replica set.
The rebalance process will divide the files to be migrated
among all the nodes by hashing the gfid of the file and
using that value to select a node to perform the migration.
This patch makes the required DHT and tiering changes.

Some tests in rebal-all-nodes-migrate.t will need to be
uncommented once the AFR and EC changes are merged.

Change-Id: I5ce41600f5ba0e244ddfd986e2ba8fa23329ff0c
BUG: 1366817
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17239
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy 
Reviewed-by: Shyamsundar Ranganathan

feature/dht: Directory synchronization

2017-04-26T09:00:34+00:00

Design doc: https://review.gluster.org/16876

Directory creation is now synchronized with blocking inodelk of the
parent on the hashed subvolume followed by the entrylk on the hashed
subvolume between dht_mkdir, dht_rmdir, dht_rename_dir and lookup
selfheal mkdir.

To maintain internal consistency of directories across all subvols of
dht, we need locks. Specifically we are interested in:

 1. Consistency of layout of a directory. Only one writer should modify
    the layout at a time. A writer (layout setting during directory heal
    as part of lookup) shouldn't modify the layout while there are
    readers (all other fops like create, mkdir etc., which consume
    layout) and readers shouldn't read the layout while a writer is in
    progress. Readers can read the layout simultaneously. Writer takes
    a WRITE inodelk on the directory (whose layout is being modified)
    across ALL subvols. Reader takes a READ inodelk on the directory
    (whose layout is being read) on ANY subvol.

 2. Consistency of directory namespace across subvols. The path and
    associated gfid should be same on all subvols. A gfid should not be
    associated with more than one path on any subvol. All fops that can
    change directory names (mkdir, rmdir, renamedir, directory creation
    phase in lookup-heal) takes an entrylk on hashed subvol of the
    directory.

 NOTE1: In point 2 above, since dht takes entrylk on hashed subvol of a
        directory, the transaction itself is a consumer of layout on
        parent directory. So, the transaction is a reader of parent
        layout and does an inodelk on parent directory just like any
        other layout reader. So a mkdir (dir/subdir) would:

     > Acquire a READ inodelk on "dir" on any subvol.
     > Acquire an entrylk (dir, "subdir") on hashed subvol of "subdir".
     > creates directory on hashed subvol and possibly on non-hashed subvols.
     > UNLOCK (entrylk)
     > UNLOCK (inodelk)

 NOTE2: mkdir fop while setting the layout of the directory being created
        is considered as a reader, but NOT a writer. The reason is for
        a fop which can consume the layout of a directory to come either
        of the following conditions has to be true:

     > mkdir syscall from application has to complete. In this case no
       need of synchronization.
     > A lookup issued on the directory racing with mkdir has to complete.
       Since layout setting by a lookup is considered as a writer, only
       one of either mkdir or lookup will set the layout.

Code re-organization:
   All the lock related routines are moved to "dht-lock.c" file.
   New wrapper function is introduced to take blocking inodelk
   followed by entrylk 'dht_protect_namespace'

Updates #191
Change-Id: I01569094dfbe1852de6f586475be79c1ba965a31
Signed-off-by: Kotresh HR 
BUG: 1443373
Reviewed-on: https://review.gluster.org/15472
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System

refcount: typecast function for calling on free

2017-01-31T05:04:44+00:00

All of the functions called to free the refcounted structure are doing a
typecast from (void*) to their own type taht is being free'd. This
really is not needed and the refcount interface is made a little simpler
without the requirement of typecasting.

With this small improvement in the API, all callers are updated too.

Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
BUG: 1416889
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16471
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Xavier Hernandez 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

dht/cluster: add logs to fix-layout code path

2017-01-20T10:49:57+00:00

Currently there is no helpful log in fix-layout code path. Adding
the logs to be helpful for debugging fix-layout failures.

BUG: 1414782
Change-Id: I61c29ceedcaa2e235fa7be99866709d6ca6de3ae
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/16040
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G