glusterfs.git/xlators/cluster/dht/src, branch v6.4 [RFC] change get_real_filename implementation to use ENOATTR instead of ENOENT 2019-07-04T05:53:23+00:00 Michael Adam obnox@samba.org 2019-06-20T11:09:37+00:00 fb93da37914458996987665d8e10f7ef87772133 get_real_filename is implemented as a virtual extended attribute to help Samba implement the case-insensitive but case preserving SMB protocol more efficiently. It is implemented as a getxattr call on the parent directory with the virtual key of "get_real_filename:<entryname>" by looking for a spelling with different case for the provided file/dir name (<entryname>) and returning this correct spelling as a result if the entry is found. Originally (05aaec645a6262d431486eb5ac7cd702646cfcfb), the implementation used the ENOENT errno to return the authoritative answer that <entryname> does not exist in any case folding. Now this implementation is actually a violation or misuse of the defined API for the getxattr call which returns ENOENT for the case that the dir that the call is made against does not exist and ENOATTR (or the synonym ENODATA) for the case that the xattr does not exist. This was not a problem until the gluster fuse-bridge was changed to do map ENOENT to ESTALE in 59629f1da9dca670d5dcc6425f7f89b3e96b46bf, after which we the getxattr call for get_real_filename returned an ESTALE instead of ENOENT breaking the expectation in Samba. It is an independent problem that ESTALE should not leak out to user space but is intended to trigger retries between fuse and gluster. But nevertheless, the semantics seem to be incorrect here and should be changed. This patch changes the implementation of the get_real_filename virtual xattr to correctly return ENOATTR instead of ENOENT if the file/directory being looked up is not found. The Samba glusterfs_fuse vfs module which takes advantage of the get_real_filename over a fuse mount will receive a corresponding change to map ENOATTR to ENOENT. Without this change, it will still work correctly, but the performance optimization for nonexisting files is lost. On the other hand side, this change removes the distinction between the old not-implemented case and the implemented case. So Samba changed to treat ENOATTR like ENOENT will not work correctly any more against old servers that don't implement get_real_filename. I.e. existing files will be reported as non-existing Backport of: > Change-Id: I971b427ab8410636d5d201157d9af70e0d075b67 > fixes: bz#1722977 > Signed-off-by: Michael Adam <obnox@samba.org> Change-Id: I971b427ab8410636d5d201157d9af70e0d075b67 fixes: bz#1723659 Signed-off-by: Michael Adam <obnox@samba.org> (cherry picked from commit dc1b87fcfef08c9497b0c02b2410c9d18bbc2dba) 
get_real_filename is implemented as a virtual extended attribute to help
Samba implement the case-insensitive but case preserving SMB protocol
more efficiently. It is implemented as a getxattr call on the parent directory
with the virtual key of "get_real_filename:<entryname>" by looking for a
spelling with different case for the provided file/dir name (<entryname>)
and returning this correct spelling as a result if the entry is found.
Originally (05aaec645a6262d431486eb5ac7cd702646cfcfb), the
implementation used the ENOENT errno to return the authoritative answer
that <entryname> does not exist in any case folding.

Now this implementation is actually a violation or misuse of the defined
API for the getxattr call which returns ENOENT for the case that the dir
that the call is made against does not exist and ENOATTR (or the synonym
ENODATA) for the case that the xattr does not exist.

This was not a problem until the gluster fuse-bridge was changed
to do map ENOENT to ESTALE in 59629f1da9dca670d5dcc6425f7f89b3e96b46bf,
after which we the getxattr call for get_real_filename returned an
ESTALE instead of ENOENT breaking the expectation in Samba.

It is an independent problem that ESTALE should not leak out to user
space but is intended to trigger retries between fuse and gluster.
But nevertheless, the semantics seem to be incorrect here and should
be changed.

This patch changes the implementation of the get_real_filename virtual
xattr to correctly return ENOATTR instead of ENOENT if the file/directory
being looked up is not found.

The Samba glusterfs_fuse vfs module which takes advantage of the
get_real_filename over a fuse mount will receive a corresponding change
to map ENOATTR to ENOENT. Without this change, it will still work
correctly, but the performance optimization for nonexisting files is
lost. On the other hand side, this change removes the distinction
between the old not-implemented case and the implemented case.
So Samba changed to treat ENOATTR like ENOENT will not work correctly
any more against old servers that don't implement get_real_filename.
I.e. existing files will be reported as non-existing

Backport of:
> Change-Id: I971b427ab8410636d5d201157d9af70e0d075b67
> fixes: bz#1722977
> Signed-off-by: Michael Adam <obnox@samba.org>

Change-Id: I971b427ab8410636d5d201157d9af70e0d075b67
fixes: bz#1723659
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit dc1b87fcfef08c9497b0c02b2410c9d18bbc2dba)
cluster/dht: Fixed a memleak in dht_rename_cbk 2019-07-03T06:36:46+00:00 N Balachandran nbalacha@redhat.com 2019-06-21T03:34:19+00:00 93aec95ee88d3b5313fae8f7d58b1322dabf444d Fixed a memleak in dht_rename_cbk when creating a linkto file. >Change-Id: I705adef3cb79e33806520fc2b15558e90e2c211c >fixes: bz#1722698 Change-Id: I705adef3cb79e33806520fc2b15558e90e2c211c fixes: bz#1726294 Signed-off-by: N Balachandran <nbalacha@redhat.com> (cherry picked from commit 532b0fc8b1ace9ad48fdaf643e0b1a34020b6cd8) 
Fixed a memleak in dht_rename_cbk when creating
a linkto file.

>Change-Id: I705adef3cb79e33806520fc2b15558e90e2c211c
>fixes: bz#1722698

Change-Id: I705adef3cb79e33806520fc2b15558e90e2c211c
fixes: bz#1726294
Signed-off-by: N Balachandran <nbalacha@redhat.com>
(cherry picked from commit 532b0fc8b1ace9ad48fdaf643e0b1a34020b6cd8)
cluster/dht: Refactor dht lookup functions 2019-05-09T12:10:25+00:00 N Balachandran nbalacha@redhat.com 2019-04-10T08:58:55+00:00 ab296b5be79ac2859072fd89cda32715f2ac57a5 Part 2: Modify dht_revalidate_cbk to call dht_selfheal_directory instead of separate calls to heal attrs and xattrs. Change-Id: Id41ac6c4220c2c35484812bbfc6157fc3c86b142 fixes: bz#1707393 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
Part 2: Modify dht_revalidate_cbk to call
dht_selfheal_directory instead of separate calls
to heal attrs and xattrs.

Change-Id: Id41ac6c4220c2c35484812bbfc6157fc3c86b142
fixes: bz#1707393
Signed-off-by: N Balachandran <nbalacha@redhat.com>
cluster/dht: refactor dht lookup functions 2019-05-08T14:00:05+00:00 N Balachandran nbalacha@redhat.com 2019-03-25T10:26:56+00:00 7f780f30e5b1c54702a2bf4f5624bac7b8912eee Part 1: refactor the dht_lookup_dir_cbk and dht_selfheal_directory functions. Added a simple dht selfheal directory test Change-Id: I1410c26359e3c14b396adbe751937a52bd2fcff9 updates: bz#1707393 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
Part 1:  refactor the dht_lookup_dir_cbk
and dht_selfheal_directory functions.
Added a simple dht selfheal directory test

Change-Id: I1410c26359e3c14b396adbe751937a52bd2fcff9
updates: bz#1707393
Signed-off-by: N Balachandran <nbalacha@redhat.com>
cluster/dht: Request linkto xattrs in dht_rmdir opendir 2019-02-22T03:34:36+00:00 N Balachandran nbalacha@redhat.com 2019-02-06T04:28:55+00:00 951abf5c53d4658b86b00ffb1147989c07527c3f If parallel-readdir is enabled, the rda xlator is loaded below dht in the graph and proactively lists and caches entries when an opendir is performed. dht_rmdir checks if the directory being deleted contains stale linkto files by performing a readdirp on its child subvols. However, as the entries are actually read in during the opendir operation which does not request the linkto xattr,no linkto xattrs are present for the entries causing dht to incorrectly identify them as data files and fail the rmdir operation with ENOTEMPTY. DHT now always adds the linkto xattr in the list of xattrs requested in the opendir. Change-Id: I0711198e66c59146282eb8b88084170bedfb4018 fixes: bz#1679004 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
If parallel-readdir is enabled, the rda xlator is loaded
below dht in the graph and proactively lists and caches
entries when an opendir is performed. dht_rmdir checks if
the directory being deleted contains stale linkto files by
performing a readdirp on its child subvols. However, as
the entries are actually read in during the opendir operation
which does not request the linkto xattr,no linkto xattrs are
present for the entries causing dht to incorrectly identify
them as data files and fail the rmdir operation with ENOTEMPTY.
DHT now always adds the linkto xattr in the list of xattrs
requested in the opendir.

Change-Id: I0711198e66c59146282eb8b88084170bedfb4018
fixes: bz#1679004
Signed-off-by: N Balachandran <nbalacha@redhat.com>
dht: fix double extra unref of inode at heal path 2019-02-22T03:34:06+00:00 Kinglong Mee kinglongmee@gmail.com 2019-01-08T08:38:45+00:00 5c9b2dfb4d251cc0201759c495bbd4d1ad5e3c87 The loc_wipe is done in the _out_ section, inode_unref(loc.parent) here casues a double extra unref of loc.parent. > Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6 > Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> (cherry-pick of https://review.gluster.org/#/c/glusterfs/+/21998/) Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6 updates: bz#1679275 Signed-off-by: Susant Palai <spalai@redhat.com> 
The loc_wipe is done in the _out_ section, inode_unref(loc.parent) here
casues a double extra unref of loc.parent.

> Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6
> Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
(cherry-pick of https://review.gluster.org/#/c/glusterfs/+/21998/)

Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6
updates: bz#1679275
Signed-off-by: Susant Palai <spalai@redhat.com>
cluster/dht: Fix lookup selfheal and rmdir race 2019-02-18T14:45:30+00:00 N Balachandran nbalacha@redhat.com 2019-02-12T08:28:54+00:00 8d4b8e2644917ed25a9d9bef37803c4a2f7da84f A race between the lookup selfheal and rmdir can cause directories to be healed only on non-hashed subvols. This can prevent the directory from being listed from the mount point and in turn causes rm -rf to fail with ENOTEMPTY. Fix: Update the layout information correctly and reduce the call count only after processing the response. Change-Id: I812779aaf3d7bcf24aab1cb158cb6ed50d212451 fixes: bz#1677260 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
A race between the lookup selfheal and rmdir can cause
directories to be healed only on non-hashed subvols.
This can prevent the directory from being listed from
the mount point and in turn causes rm -rf to fail with
ENOTEMPTY.
Fix: Update the layout information correctly and reduce
the call count only after processing the response.

Change-Id: I812779aaf3d7bcf24aab1cb158cb6ed50d212451
fixes: bz#1677260
Signed-off-by: N Balachandran <nbalacha@redhat.com>
cluster/dht: Do not use gfid-req in fresh lookup 2019-02-02T03:09:36+00:00 N Balachandran nbalacha@redhat.com 2019-01-29T04:17:32+00:00 da5ae3afcd1b16bb0f2f9bffc1878b17b75429ed Fuse sets a random gfid-req value for a fresh lookup. Posix lookup will set this gfid on entries with missing gfids causing a GFID mismatch for directories. DHT will now ignore the Fuse provided gfid-req and use the GFID returned from other subvols to heal the missing gfid. Change-Id: I5f541978808f246ba4542564251e341ec490db14 fixes: bz#1670259 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
Fuse sets a random gfid-req value for a fresh lookup. Posix
lookup will set this gfid on entries with missing gfids causing
a GFID mismatch for directories.
DHT will now ignore the Fuse provided gfid-req and use the GFID
returned from other subvols to heal the missing gfid.

Change-Id: I5f541978808f246ba4542564251e341ec490db14
fixes: bz#1670259
Signed-off-by: N Balachandran <nbalacha@redhat.com>
core: make gf_thread_create() easier to use 2019-02-01T09:37:20+00:00 Xavi Hernandez xhernandez@redhat.com 2019-01-24T17:31:10+00:00 4674678951a1315975d66016fb55c49100b7819f This patch creates a specific function to set the thread name using a string format and a variable argument list, like printf(). This function is used to set the thread name from gf_thread_create(), which now accepts a variable argument list to create the full name. It's not necessary anymore to use a local array to build the name of the thread. This is done automatically. Change-Id: Idd8d01fd462c227359b96e98699f8c6d962dc17c Updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> 
This patch creates a specific function to set the thread name using a
string format and a variable argument list, like printf().

This function is used to set the thread name from gf_thread_create(),
which now accepts a variable argument list to create the full name. It's
not necessary anymore to use a local array to build the name of the
thread. This is done automatically.

Change-Id: Idd8d01fd462c227359b96e98699f8c6d962dc17c
Updates: bz#1193929
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
cluster/dht: Remove internal permission bits 2019-02-01T05:29:59+00:00 N Balachandran nbalacha@redhat.com 2019-01-28T06:25:17+00:00 7eda6fe62d4e47937afc2ba71b4fd858399bca05 Rebalance sets the sgid and t bits on a file that is being migrated. These permissions are not removed in dht_readdirp_cbk when listing files causing them to show up on the mountpoint. We now remove these permissions if a non-linkto file has the linkto xattr set. Change-Id: I5c69b2ecfe2df804fe50faea903b242d01729596 fixes: bz#1669937 Signed-off-by: N Balachandran <nbalacha@redhat.com> 
Rebalance sets the sgid and t bits on a file
that is being migrated. These permissions are
not removed in dht_readdirp_cbk when listing files
causing them to show up on the mountpoint.
We now remove these permissions if a non-linkto
file has the linkto xattr set.

Change-Id: I5c69b2ecfe2df804fe50faea903b242d01729596
fixes: bz#1669937
Signed-off-by: N Balachandran <nbalacha@redhat.com>