glusterfs.git/xlators/performance, branch v6.8

To fix readdir-ahead memory leak

2020-02-11T08:27:55+00:00

Glusterfs client process has memory leak if create serveral files under one folder, and delete the folder.
According to statedump, the ref counts of readdir-ahead is bigger than zero in the inode table. Readdir-ahead get parent inode by inode_parent in rda_mark_inode_dirty when each rda_writev_cbk,the inode ref count of parent folder will be increased in inode_parent, but readdir-ahead do not unref it later.
The correction is unref the parent inode at the end of rda_mark_inode_dirty

Backport of:
> Change-Id: Iee68ab1089cbc2fbc4185b93720fb1f66ee89524
> Fixes: bz#1779055
> Signed-off-by: HuangShujun <549702281@qq.com>

Change-Id: Iee68ab1089cbc2fbc4185b93720fb1f66ee89524
(cherry picked from commit 99044a5cedcff9a9eec40a07ecb32bd66271cd02)
Signed-off-by: Krutika Dhananjay 
Fixes: bz#1789337

Detach iot_worker to release its resources

2019-11-06T11:02:35+00:00

When iot_worker terminates, its resources have not been reaped, which
will consumes lots of memory.

Detach iot_worker to automically release its resources back to the
system.

Change-Id: I71fabb2940e76ad54dc56b4c41aeeead2644b8bb
fixes: bz#1768726
Signed-off-by: Liguang Li 
Signed-off-by: N Balachandran

perf/write-behind: Clear frame->local on conflict error

2019-10-04T05:22:36+00:00

WB saves the wb_inode in frame->local for the truncate and
ftruncate fops. This value is not cleared in case of error
on a conflicting write request. FRAME_DESTROY finds a non-null
frame->local and tries to free it using mem_put. However,
wb_inode is allocated using GF_CALLOC, causing the
process to crash.

credit: vpolakis@gmail.com

Change-Id: I217f61470445775e05145aebe44c814731c1b8c5
fixes: bz#1755679
Signed-off-by: N Balachandran

performance/md-cache: Do not skip caching of null character xattr values

2019-08-28T08:29:47+00:00

Null character string is a valid xattr value in file system. But for
those xattrs processed by md-cache, it does not update its entries if
value is null('\0'). This results in ENODATA when those xattrs are
queried afterwards via getxattr() causing failures in basic operations
like create, copy etc in a specially configured Samba setup for Mac OS
clients.

On the other side snapview-server is internally setting empty string("")
as value for xattrs received as part of listxattr() and are not intended
to be cached. Therefore we try to maintain that behaviour using an
additional dictionary key to prevent updation of entries in getxattr()
and fgetxattr() callbacks in md-cache.

Credits: Poornima G 

Change-Id: I7859cbad0a06ca6d788420c2a495e658699c6ff7
Fixes: bz#1743782
Signed-off-by: Anoop C S 
(cherry picked from commit b4b683736367d93daad08a5ee6ca95778c07c5a4)

performance/readdir-ahead: fix deadlock

2019-03-08T14:08:29+00:00

This deadlock happens while processing dentry corresponding to current
directory (.) in rda_fill_readdirp. In this case following order is
followed:

LOCK(directory_fd_ctx->lock);
  rda_inode_ctx_get_iatt -> LOCK(directory_inode->lock);

However, in rda_mark_inode_dirty following lock order is followed:
LOCK(directory_inode->lock);
  LOCK(directory_fd_ctx->lock);

these two codepaths when executed concurrently resulted in a deadlock.

Current patch fixes this by removing locking directory inode and
fd-ctx in rda_fill_readdirp. This is fine as directory inode's stat
won't change due to writes to files within directory.

Change-Id: Ic93a67a0dac8229bb0d79582e526a512e6f2569c
Signed-off-by: Raghavendra Gowdappa 
Fixes: bz#1686399

io-threads: Prioritize fops with NO_ROOT_SQUASH pid

2019-03-06T03:16:53+00:00

There was 30% regression observed in mkdir path with commit
b139bc58eb504adf5ef81658896c9283ae21f390. On analysis it is found
that io-threads xlator deprioritzes fops with all -ve pid.

Some context in to the no-root-squash pid requirement:
DHT xlator does some of the internal fops with root privileges. This is
needed so that operations like layout healing should not be abandoned
because a non root user is operating.  If root-squash option is enabled
the layout set operation looses its root privilege as server xlator
converts the uid and pid to random numbers. Hence, the above mentioned
commit converted pid to GF_CLIENT_PID_NO_ROOT_SQUASH to continue fops
as root.

Combining the above I am proposing not to deprioritize fops with
no-root-squash pid.

> Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
> fixes: bz#1676430
> Signed-off-by: Susant Palai 
(cherry picked from commit f5c3b1727f55ffaa3dcdb3c3a09b968ebb45dbb2)

Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
fixes: bz#1676429
Signed-off-by: Susant Palai

md-cache: Adapt integer data types to avoid integer overflow

2019-02-25T15:25:57+00:00

The "struct iatt" in iatt.h is using int64_t types for storing
the atime, mtime and ctime. Therefore the struct 'struct md_cache' in
md-cache.c should also use this types to avoid an integer overflow.

This can happen e.g. if someone uses a very high default-retention-period
in the WORM-Xlator.

Change-Id: I605268d300ab622b9c8ab30e459dc00d9340aad1
fixes: bz#1680020
Signed-off-by: David Spisla 
(cherry picked from commit 15423e14f16dd1a15ee5e5cbbdbdd370e57ed59f)

performance/write-behind: fix use-after-free in readdirp

2019-02-22T03:33:07+00:00

Two issues were found:
1. in wb_readdirp_cbk, inode should unrefed after wb_inode is
unlocked. Otherwise, inode and hence the context wb_inode can be freed
by the type we try to unlock wb_inode
2. wb_readdirp_mark_end iterates over a list of wb_inodes of children
of a directory. But inodes could've been freed and hence the list
might be corrupted. To fix take a reference on inode before adding it
to invalidate_list of parent.

Change-Id: I911b0e0b2060f7f41ded0b05db11af6f9b7c09c5
Signed-off-by: Raghavendra Gowdappa 
Updates: bz#1678570
(cherry picked from commit 64cc4458918e8c8bfdeb114da0a6501b2b98491a)

performance/write-behind: handle call-stub leaks

2019-02-19T06:23:34+00:00

Change-Id: I7be9a5f48dcad1b136c479c58b1dca1e0488166d
Signed-off-by: Raghavendra Gowdappa 
Fixes: bz#1678570
(cherry picked from commit 6175cb10cd5f59f3c7ae4100bc78f359b68ca3e9)

performance/md-cache: introduce an option to control invalidation of inodes

2019-02-18T14:44:27+00:00

Explicit invalidation by calling inode_invalidate is necessary when
same (meta)data is shared/access across multiple mounts. Without an
explicit inode_invalidate call, caches in the mount which didn't
witness writes wouldn't be aware of changes as writes wouldn't have
passed through them. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, explicit inode
invalidation can be disabled for this case which gives a huge
performance boost for workloads that write data and then immediately
read the data they just wrote. Note that otherwise, local writes
(which pass through the cache) will change ctime and cause unnecessary
invalidations.

The name of the option that controls this behavior is
"performance.global-cache-invalidation". This option is global and it
purges caches both in glusterfs and kernel stack for native FUSE
mounts. For non-native FUSE mounts, it purges cache only from
glusterfs stack. This option is effective only when
performance.stat-prefetch is on.

Note that there is a similar option "performance.cache-invalidation",
but the scope of that option is limited to quick-read and md-cache.

Change-Id: I462bb4b65ff9aae1f6ba76f50b1f2f94fb10323b
Signed-off-by: Raghavendra Gowdappa 
updates: bz#1674364
(cherry picked from commit 2b5aa4489de2017a03bcb6ec8986286f0c76a670)