glusterfs.git/xlators/mount, branch v5.8

fuse: add --lru-limit option

2019-01-16T15:59:03+00:00

The inode LRU mechanism is moot in fuse xlator (ie. there is no
limit for the LRU list), as fuse inodes are referenced from
kernel context, and thus they can only be dropped on request of
the kernel. This might results in a high number of passive
inodes which are useless for the glusterfs client, causing a
significant memory overhead.

This change tries to remedy this by extending the LRU semantics
and allowing to set a finite limit on the fuse inode LRU.

A brief history of problem:

When gluster's inode table was designed, fuse didn't have any
'invalidate' method, which means, userspace application could
never ask kernel to send a 'forget()' fop, instead had to wait
for kernel to send it based on kernel's parameters. Inode table
remembers the number of times kernel has cached the inode based
on the 'nlookup' parameter. And 'nlookup' field is not used by
no other entry points (like server-protocol, gfapi etc).

Hence the inode_table of fuse module always has to have lru-limit
as '0', which means no limit. GlusterFS always had to keep all
inodes in memory as kernel would have had a reference to it.
Again, the reason for this is, kernel's glusterfs inode reference
was pointer of 'inode_t' structure in glusterfs. As it is a
pointer, we could never free it (to prevent segfault, or memory
corruption).

Solution:

In the inode table, handle the prune case of inodes with 'nlookup'
differently, and call a 'invalidator' method, which in this case is
fuse_invalidate(), and it sends the request to kernel for getting
the forget request.

When the kernel sends the forget, it means, it has dropped all
the reference to the inode, and it will send the forget with the
'nlookup' parameter too. We just need to make sure to reduce the
'nlookup' value we have when we get forget. That automatically
cause the relevant prune to happen.

Credits: Csaba Henk, Xavier Hernandez, Raghavendra Gowdappa, Nithya B

fixes: bz#1623107
Change-Id: Ifee0737b23b12b1426c224ec5b8f591f487d83a2
Signed-off-by: Amar Tumballi

all: fix the format string exceptions

2018-11-09T14:03:02+00:00

Currently, there are possibilities in few places, where a user-controlled
(like filename, program parameter etc) string can be passed as 'fmt' for
printf(), which can lead to segfault, if the user's string contains '%s',
'%d' in it.

While fixing it, makes sense to make the explicit check for such issues
across the codebase, by making the format call properly.

Fixes: CVE-2018-14661

Fixes: bz#1647666
Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4
Signed-off-by: Amar Tumballi

gfapi: revert several patchs that introduced pre/post attrs

2018-09-17T14:26:06+00:00

Reverted the following:
  - 248152767b0599986bbb6bb35fc27197f6be6964
  - 09943beb499617212f2985ca8ea9ecd1ed1b470e
  - d01f7244e9d9f7e3ef84e0ba7b48ef1b1b09d809

The reverts are redone by hand, due to clang format changes
that made using git to revert the changes more tedious.

Change-Id: I96489638a2b641fb2206a110298543225783f7be
Updates: bz#1628620
Signed-off-by: ShyamsundarR

Land part 2 of clang-format changes

2018-09-12T12:22:45+00:00

Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu

Land clang-format changes

2018-09-12T11:52:48+00:00

Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b

mount/fuse: convert ENOENT to ESTALE in open(dir)_resume

2018-09-11T11:13:29+00:00

This patch is continuation of commit
fb4b914ce84bc83a5f418719c5ba7c25689a9251.


mount/fuse: never fail open(dir) with ENOENT

    open(dir) being an operation on inode should never fail with
    ENOENT. If gfid is not present, the appropriate error is
    ESTALE. This will enable kernel to retry open after a revalidate
    lookup.


Earlier commit failed to fix codepath where error response is sent
back on gfid resolution failures in fuse_open(dir)_resume. Current
patch completes that work

Change-Id: Ia07e3cece404811703c8cfbac9b402ca5fe98c1e
Signed-off-by: Raghavendra G 
updates: bz#1627620

Multiple files: calloc -> malloc

2018-09-04T04:58:14+00:00

xlators/storage/posix/src/posix-inode-fd-ops.c:
xlators/storage/posix/src/posix-helpers.c:
xlators/storage/bd/src/bd.c:
xlators/protocol/client/src/client-lk.c:
xlators/performance/quick-read/src/quick-read.c:
xlators/performance/io-cache/src/page.c
xlators/nfs/server/src/nfs3-helpers.c
xlators/nfs/server/src/nfs-fops.c
xlators/nfs/server/src/mount3udp_svc.c
xlators/nfs/server/src/mount3.c
xlators/mount/fuse/src/fuse-helpers.c
xlators/mount/fuse/src/fuse-bridge.c
xlators/mgmt/glusterd/src/glusterd-utils.c
xlators/mgmt/glusterd/src/glusterd-syncop.h
xlators/mgmt/glusterd/src/glusterd-snapshot.c
xlators/mgmt/glusterd/src/glusterd-rpc-ops.c
xlators/mgmt/glusterd/src/glusterd-replace-brick.c
xlators/mgmt/glusterd/src/glusterd-op-sm.c
xlators/mgmt/glusterd/src/glusterd-mgmt.c
xlators/meta/src/subvolumes-dir.c
xlators/meta/src/graph-dir.c
xlators/features/trash/src/trash.c
xlators/features/shard/src/shard.h
xlators/features/shard/src/shard.c
xlators/features/marker/src/marker-quota.c
xlators/features/locks/src/common.c
xlators/features/leases/src/leases-internal.c
xlators/features/gfid-access/src/gfid-access.c
xlators/features/cloudsync/src/cloudsync-plugins/src/cloudsyncs3/src/libcloudsyncs3.c
xlators/features/bit-rot/src/bitd/bit-rot.c
xlators/features/bit-rot/src/bitd/bit-rot-scrub.c
bxlators/encryption/crypt/src/metadata.c
xlators/encryption/crypt/src/crypt.c
xlators/performance/md-cache/src/md-cache.c:

Move to GF_MALLOC() instead of GF_CALLOC() when possible

It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.

In some cases, also changed allocation size to be sizeof some
struct or type instead of a pointer - easier to read.
In some cases, removed redundant strlen() calls by saving the result
into a variable.

1. Only done for the straightforward cases. There's room for improvement.
2. Please review carefully, especially for string allocation, with the
terminating NULL string.

Only compile-tested!

.. and allocate memory as much as needed.

xlators/nfs/server/src/mount3.c :

Don't blindly allocate PATH_MAX, but strlen() the string and allocate
appropriately.
Also, align error messges.

updates: bz#1193929
Original-Author: Yaniv Kaul 
Signed-off-by: Yaniv Kaul 
Signed-off-by: Yaniv Kaul 
Change-Id: Ibda6f33dd180b7f7694f20a12af1e9576fe197f5

performance/readdir-ahead: keep stats of cached dentries in sync with modifications

2018-08-18T07:28:53+00:00

PROBLEM:

Stats of dentries that are readdirp'd ahead can become stale due to
fops like writes, truncate etc that modify the file pointed by
dentries. When a readdir is finally wound at offset corresponding to
these entries, the iatts that are returned to the application come
from readdir-ahead's cache, which are stale by now. This problem gets
further aggravated when caching translators/modules cache and continue
to serve this stale information.

FIX:

* Store the iatt in context of the inode pointed by dentry.
* Whenever the inode pointed by dentry undergoes modification, in cbk
  of modification fop, update the iatt stored in inode-ctx to reflect
  the modification.
* When serving a readdirp response from application, update iatts of
  dentries with the iatts stored in the context of inodes pointed by
  these dentries.
* Some fops don't have valid iatts in their responses. For eg., write
  response whose data is still cached in write-behind will have zeroed
  out stat. In this case keep only ia_type and ia_gfid and reset rest
  of the iatt members to zero.
  - fuse-bridge in this case just sends "entry" information back to
    kernel and attr is not sent.
  - gfapi sets entry->inode to NULL and zeroes out the entire stat
* There is one tiny race between the entry creation and a readdirp on
  its parent dir, which could cause the inode-ctx setting and inode
  ctx reading to happen on two different inode objects. To prevent
  this, when entry->inode doesn't eqaul to linked_inode,
  - fuse-bridge is made to send only "entry" information without
    attributes
  - gfapi sets entry->inode to NULL and zeroes out the entire stat.

Change-Id: Ia27ff49a61922e88c73a1547ad8aacc9968a69df
BUG: 1390050
Updates: bz#1390050
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G

All: run codespell on the code and fix issues.

2018-07-22T14:40:16+00:00

Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...

Only compile-tested!

Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul

add check if no matching password record was found with getpwuid_r(uid)

2018-07-13T13:03:30+00:00

Change-Id: Iae712828ee656008faf5fe2bc4e6f96fa12ea4cb
fixes: bz#1600687
Signed-off-by: Vitaly Lipatov