glusterfs.git/xlators/performance/open-behind, branch v6dev

Land part 2 of clang-format changes

2018-09-12T12:22:45+00:00

Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu

Land clang-format changes

2018-09-12T11:52:48+00:00

Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b

performance/ob: stringent synchronization between rename/unlink and open

2018-08-03T14:47:03+00:00

Issue 1:
========
open all pending fds before resuming rename and unlink

currently ob uses fd_lookup to find out the opened-behind. But,
fd_lookup gives the recent fd opened on the inode, but the oldest
fd(s) (there can be multiple fds opened-behind when the very first
opens on an inode are issued in parallel) are the candidates for fds
with pending opens on backend. So, this patch explictily tracks the
opened-behind fds on an inode and opens them before resuming rename or
unlink.

similar code changes are also done for setattr and setxattr to make
sure pending opens are complete before permission change.

This patch also adds a check for an open-in-progress to
ob_get_wind_fd. If there is already an open-in-progress,
ob_get_wind_fd won't return an anonymous fd as a result. This is done
to make sure that rename/unlink/setattr/setxattr don't race with an
operation like readv/fstat on an anonymous fd already in progress.

Issue 2:
========
once renamed/unlinked, don't open-behind any future opens on the same
inode.

Issue 3:
========
Don't use anonymous fds by default. Note that rename/unlink can race
with a read/fd on anonymous fds and these operations can fail with
ESTALE. So, for better consistency in default mode, don't use
anonymous fds. If performance is needed with tradeoff of consistency,
one can switch on the option "use-anonymous-fd"

Change-Id: Iaf130db71ce61ac37269f422e348a45f6ae6e82c
Signed-off-by: Raghavendra G 
Updates: bz#1512691

performance/open-behind: don't use anonymous fds for reads by default

2018-08-02T12:13:11+00:00

anonymous fds interfere with working of read-ahead as read-ahead won't
be able to store its cache in fd. Also, as seen in bz 1455872,
anonymous fds also affect performance of large file sequential reads
as the cost of opening fd for each read on brick stack is
significant. So, have a proper fd which enables read-ahead to store
its cache and brick stack to reuse the fd during reads.

With this change test
tests/bugs/snapshot/bug-1167580-set-proper-uid-and-gid-during-nfs-access.t
fails consistently. The failure can also be seen with open-behind
off. bz 1611532 has been filed to track the issue with test. Thanks to
Rafi  for assistance provided in debugging test
failure.

Change-Id: Ifa52d8ff017f115e83247f3396b9d27f0295ce3f
Signed-off-by: Raghavendra G 
Fixes: bz#1455872

performance/open-behind: open pending fds before permission change

2018-05-29T08:44:21+00:00

setattr, posix-acl and selinux changes on a file can revoke permission
to open the file after permission changes. To prevent that, make sure
the pending fd is opened before winding down setattr or setxattr (for
posix-acl and selinux) calls.

Change-Id: Ib0b91795d286072e445190f9a1b3b1e9cd363282
Signed-off-by: Raghavendra G 
fixes: bz#1405147

xlators/performance: Add pass-through option

2018-04-11T10:53:48+00:00

Add pass-through option in performance traslators. Set the option in
GF_OPTION_INIT() and GF_OPTION_RECONF()

Updates: #304

Change-Id: If1537450147d154905831e36f7162a32866d7ad6
Signed-off-by: Varsha Rao

all: Simplify component message id's definition

2017-12-14T02:33:51+00:00

This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.

Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez

perf/open-behind: update options for gd2

2017-12-10T23:23:53+00:00

Updates: #302

Change-Id: I12e5bc6d9e1fcd285299818a7f7e8f1622deefcb
Signed-off-by: Milind Changire

performance/open-behind: Avoid deadlock in statedump

2016-11-10T06:42:57+00:00

Problem:
open-behind is taking fd->lock then inode->lock where as statedump is taking
inode->lock then fd->lock, so it is leading to deadlock

In open-behind, following code exists:
void
ob_fd_free (ob_fd_t *ob_fd)
{
        loc_wipe (&ob_fd->loc); <<--- this takes (inode->lock)
.......
}

int
ob_wake_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
             int op_ret, int op_errno, fd_t *fd_ret, dict_t *xdata)
{
	.......
        LOCK (&fd->lock); <<---- fd->lock
        {
	.......
                __fd_ctx_del (fd, this, NULL);
                ob_fd_free (ob_fd); <<<---------------
        }
        UNLOCK (&fd->lock);
.......
}
=================================================================
In statedump this code exists:
inode_dump (inode_t *inode, char *prefix)
{
.......
	ret = TRY_LOCK(&inode->lock); <<---- inode->lock
.......
	fd_ctx_dump (fd, prefix); <<<-----
.......
}
fd_ctx_dump (fd_t *fd, char *prefix)
{
.......
        LOCK (&fd->lock); <<<------------------ this takes fd-lock
        {
.......
}

Fix:
Make sure open-behind doesn't call ob_fd_free() inside fd->lock

BUG: 1393259
Change-Id: I4abdcfc5216270fa1e2b43f7b73445f49e6d6e6e
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15808
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Poornima G 
Reviewed-by: Raghavendra G

performance/open-behind: Pass O_DIRECT flags for anon fd reads when required

2016-09-23T05:32:14+00:00

Writes are already passing the correct flags at the time of open().

Also, make io-cache honor direct-io for anon-fds with
O_DIRECT flag during reads.

Change-Id: I215cb09ef1b607b9f95cabf0ef3065c00edd9e78
BUG: 1377556
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/15537
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System