glusterfs.git/xlators/storage/posix/src/posix-handle.h, branch v3.7.14

cluster/afr: Fix data loss due to race between sh and ongoing write.

2016-01-29T01:43:33+00:00

        Backport of: http://review.gluster.org/#/c/13001/

Problem:

When IO is happening on a file and a brick goes down comes back up
during this time, protocol/client translator attempts reopening of the
fd on the gfid handle of the file. But if another client renames this
file while a brick was down && writes were in progress on it, once this
brick is back up, there can be a race between reopening of the fd and
entry self-heal replaying the effect of the rename() on the sink brick.
If the reopening of the fd happens first, the application's writes
continue to go into the data blocks associated with the gfid.
Now entry-self-heal deletes 'src' and creates 'dst' file on the sink,
marking dst as a 'newentry'.  Data self-heal is also completed on 'dst'
as a result and self-heal terminates. If at this point the application
is still writing to this fd, all writes on the file after self-heal
would go into the data blocks associated with this fd, which would be
lost once the fd is closed. The result - the 'dst' file on the source
and sink are not the same and there is no pending heal on the file,
leading to silent corruption on the sink.

Fix:

Leverage http://review.gluster.org/#/c/12816/ to ensure the gfid handle
path gets saved in .glusterfs/unlink until the fd is closed on the file.
During this time, when self-heal sends mknod() with gfid of the file,
do the following:
link() the gfid handle under .glusterfs/unlink to the new path to be
created in mknod() and
rename() the gfid handle to go back under .glusterfs/ab/cd/.


Change-Id: I5dc49c127ef0a1bf3cf4ce1b24610b1527f84d6f
BUG: 1293265
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/13036
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

posix: posix_make_ancestryfromgfid shouldn't log ENOENT

2015-08-17T04:25:45+00:00

This is a backport of http://review.gluster.org/11861

posix_make_ancestryfromgfid shouldn't log ENOENT
and it should set proper op_errno

> Change-Id: I8a87f30bc04d33cab06c91c74baa9563a1c7b45d
> BUG: 1251449
> Signed-off-by: vmallika 
> Reviewed-on: http://review.gluster.org/11861
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Manikandan Selvaganesh 
> Reviewed-by: Raghavendra G 

Change-Id: Iefe010117e64586ac6b499459286721b0065cea8
BUG: 1253260
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/11910
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G

Porting new log messages for posix

2015-06-25T06:59:47+00:00

        Backport of http://review.gluster.org/#/c/9893/

Cherry picked from 59e1c79ac6953e552fbdc8c627f5dc1d02eeacc8
>Change-Id: I29bdeefb755805858e3cb1817b679cb6f9a476a9
>BUG: 1194640
>Signed-off-by: Hari Gowtham 
>Reviewed-on: http://review.gluster.org/9893
>Reviewed-by: Pranith Kumar Karampuri 
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Signed-off-by: Hari Gowtham 

Change-Id: I29bdeefb755805858e3cb1817b679cb6f9a476a9
BUG: 1217722
Signed-off-by: Hari Gowtham 
Signed-off-by: Hari Gowtham 
Reviewed-on: http://review.gluster.org/11279
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Pranith Kumar Karampuri

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

Storage/posix : Adding error checks in path formation

2015-02-19T06:55:08+00:00

Renaming directories can cause the size of the buffer
required for posix_handle_path to increase between the
first call, which calculates the size, and the second call
which forms the path in the buffer allocated based on
the size calculated in the first call.

The path created in the second call overflows the
allocated buffer and overwrites the stack causing the
brick process to crash.

The fix adds a buffer size check to prevent the buffer
overflow. It also checks and returns an error if the
posix_handle_path call is unable to form the path instead
of working on the incomplete path, which is likely to cause
subsequent calls using the path to fail with ELOOP.

Preventing buffer overflow and handling errors
BUG: 1113960

Change-Id: If3d3c1952e297ad14f121f05f90a35baf42923aa
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/9289
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

POSIX filesystem compliance: PATH_MAX

2014-10-03T09:05:48+00:00

POSIX mandates the filesystem to support paths of lengths up to
_XOPEN_PATH_MAX (1024).  This is the PATH_MAX limit here:
http://pubs.opengroup.org/onlinepubs/009604499/basedefs/limits.h.html

When using a path of 1023 bytes, the posix xlator attempts to create
an absolute path by  prefixing the 1023 bytes path by the brick
base path. The result is an absolute path of more than _XOPEN_PATH_MAX
bytes which may be rejected by the backend filesystem.

Linux's ext3fs PATH_MAX seems to defaut to 4096, which means it
will work (except if brick base path is longer than 2072 bytes but
it is unlikely to happen. NetBSD's FFS PATH_MAX defaults to 1024,
which means the bug can happen regardless of brick base path length.

If this condition is detected for a brick, the proposed fix is to
chdir() the brick glusterfsd daemon to its brick base directory.
Then when encountering a path that will exceed _XOPEN_PATH_MAX once
prefixed by the brick base path, a relative path is used instead
of an absolute one. We do not always use relative path because some
operations require an absolute path on the brick base path itself
(e.g.: statvfs).

At least on NetBSD, this chdir() uncovers a race condition which
causes file lookup to fail with ENODATA for a few seconds. The
volume quickly reaches a sane state, but regression tests are fast
enough to choke on it. The reason is obscure (as often with race
conditions), but sleeping one second after the chdir() seems to
change scheduling enough that the problem disapear.

Note that since the chdir() is done if brick backend filesystem
does not support path long enough, it will not occur with Linux
ext3fs (except if brick base path is over 2072 bytes long).

BUG: 1129939
Change-Id: I7db3567948bc8fa8d99ca5f5ba6647fe425186a9
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8596
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Tested-by: Harshavardhana 
Reviewed-by: Vijay Bellur

glusterd/quota: Heal pgfid xattr on existing data when the quota is

2014-09-30T16:42:02+00:00

enable

The pgfid extended attributes are used to construct the ancestry path
(from the file to the volume root) for nameless lookups on files.
As NFS relies on nameless lookups heavily, quota enforcement through NFS
would be inconsistent if quota were to be enabled on a volume with
existing data.

Solution is to heal the pgfid extended attributes as a part of lookup
perfomed by quota-crawl process. In a posix lookup check for pgfid xattr
and if it is missing set the xattr.

Change-Id: I5912ea96787625c496bde56d43ac9162596032e9
BUG: 1147378
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/8878
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Always check for ENODATA with ENOATTR

2014-09-08T09:06:52+00:00

Linux defines ENODATA and ENOATTR with the same value, which means that
code can miss on on the two without breaking.

FreeBSD does not have ENODATA and GlusterFS defines it as ENOATTR just
like Linux does.

On NetBSD, ENODATA != ENOATTR, hence we need to check for both values
to get portable behavior.

BUG: 764655
Change-Id: I003a3af055fdad285d235f2a0c192c9cce56fab8
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8447
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

storage/posix: Prefer gfid links for inode-handle

2014-09-02T12:10:20+00:00

Problem:
File path could change by other entry operations in-flight so if renames are in
progress at the time of other operations like open, it may lead to failures.
We observed that this issue can also happen while renames and readdirps/lookups
are in progress because dentry-table is going stale sometimes.

Fix:
Prefer gfid-handles over paths for files. For directory handles prefering
gfid-handles hits performance issues because it needs to resolve paths
traversing up the symlinks.
Tests which test if files are opened should check on gfid path after this change.
So changed couple of tests to reflect the same.

Note:
This patch doesn't fix the issue for directories. I think a complete fix is to
come up with an entry operation serialization xlator. Until then lets live with
this.

Change-Id: I10bda1083036d013f3a12588db7a71039d9da6c3
BUG: 1136159
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8575
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

storage/posix: do not dereference gfid symlinks before posix_handle_mkdir_hashes()

2014-05-02T16:34:45+00:00

Whenever a new directory is created, its corresponding gfid file must
also be created. This was done first calling MAKE_HANDLE_PATH() to get
the path of the gfid file, then calling posix_handle_mkdir_hashes() to
create the parent directories of the gfid, and finally creating the
soft-link.

In normal circumstances, the gfid we want to create won't exist and
MAKE_HANDLE_PATH() will return a simple path to the new gfid. However if
the volume is damaged and a self-heal is running, it is possible that we
try to create an already existing gfid. In this case, MAKE_HANDLE_PATH()
will return a path to the directory instead of the path to the gfid.

To solve this problem, every time a path to a gfid is needed, a call to
MAKE_HANDLE_ABSPATH() is made instead of the call to MAKE_HANDLE_PATH().

Change-Id: Ic319cc38c170434db8e86e2f89f0b8c28c0d611a
BUG: 859581
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/5075
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Vijay Bellur