glusterfs.git/xlators/nfs/server/src, branch v3.5.1

gNFS: Make NFS DRC off by default

2014-06-10T09:37:13+00:00

DRC in NFS causes memory bloat and there are known memory corruptions.
It would be good to disable drc by default till the feature is stable.

Cherry picked from 4215d071cec4fc8a62ca4fd6212d83f931838829:
> Change-Id: I93db6ef5298672c56fb117370bb582a5e5550b17
> BUG: 1105524
> Original-patch-by: Santosh Kumar Pradhan 
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/8004
> Tested-by: Gluster Build System 
> Reviewed-by: Krishnan Parthasarathi 
> Reviewed-by: Santosh Pradhan 
> Reviewed-by: Vijay Bellur 

Change-Id: I93db6ef5298672c56fb117370bb582a5e5550b17
BUG: 1105524
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/8013
Reviewed-by: Santosh Pradhan 
Tested-by: Gluster Build System

rpc: warn and truncate grouplist if RPC/AUTH can not hold everything

2014-05-22T13:02:21+00:00

The GlusterFS protocol currently uses AUTH_GLUSTERFS_V2 in the RPC/AUTH
header. This header contains the uid, gid and auxiliary groups of the
user/process that accesses the Gluster Volume.

The AUTH_GLUSTERFS_V2 structure allows up to 65535 auxiliary groups to
be passed on. Unfortunately, the RPC/AUTH header is limited to 400 bytes
by the RPC specification: http://tools.ietf.org/html/rfc5531#section-8.2

In order to not cause complete failures on the client-side when trying
to encode a AUTH_GLUSTERFS_V2 that would result in more than 400 bytes,
we can calculate the expected size of the other elements:

    1 | pid
    1 | uid
    1 | gid
    1 | groups_len
   XX | groups_val (GF_MAX_AUX_GROUPS=65535)
    1 | lk_owner_len
   YY | lk_owner_val (GF_MAX_LOCK_OWNER_LEN=1024)
  ----+-------------------------------------------
    5 | total xdr-units

  one XDR-unit is defined as BYTES_PER_XDR_UNIT = 4 bytes
  MAX_AUTH_BYTES = 400 is the maximum, this is 100 xdr-units.
  XX + YY can be 95 to fill the 100 xdr-units.

  Note that the on-wire protocol has tighter requirements than the
  internal structures. It is possible for xlators to use more groups and
  a bigger lk_owner than that can be sent by a GlusterFS-client.

This change prevents overflows when allocating the RPC/AUTH header. Two
new macros are introduced to calculate the number of groups that fit in
the RPC/AUTH header, when taking the size of the lk_owner in account. In
case the list of groups exceeds the maximum possible, only the first
groups are passed over the RPC/GlusterFS protocol to the bricks.
A warning is added to the logs, so that most system administrators will
get informed.

The reducing of the number of groups is not a new inventions. The
RPC/AUTH header (AUTH_SYS or AUTH_UNIX) that NFS uses has a limit of 16
groups. Most, if not all, NFS-clients will reduce any bigger number of
groups to 16. (nfs.server-aux-gids can be used to workaround the limit
of 16 groups, but the Gluster NFS-server will be limited to a maximum of
93 groups, or fewer in case the lk_owner structure contains more items.)

Cherry picked from commit 8235de189845986a535d676b1fd2c894b9c02e52:
> BUG: 1053579
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/7202
> Tested-by: Gluster Build System 
> Reviewed-by: Harshavardhana 
> Reviewed-by: Santosh Pradhan 
> Reviewed-by: Vijay Bellur 

Change-Id: I8410e59d0fd246d601b54b961d3ae9cb5a858c10
BUG: 1096425
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7829
Reviewed-by: Lalatendu Mohanty 
Tested-by: Gluster Build System

gNFS: Set default outstanding RPC limit to 16

2014-03-07T20:11:28+00:00

Backport of http://review.gluster.org/#/c/6696/

With 64, NFS server hangs with large I/O load (~ 64 threads writing
to NFS server). The test results from Ben England (Performance expert)
suggest to set it as 16 instead of 64.

Change-Id: Iaa9dda512904d2e359d8122a05e5bf65f99a7e78
BUG: 1073441
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/7200
Tested-by: Gluster Build System 
Reviewed-by: Santosh Pradhan

rpc,glusterd: Use rpc_clnt notifyfn to cleanup mydata

2013-12-23T14:58:18+00:00

        Backport of http://review.gluster.org/5512

rpc:
- On a RPC_TRANSPORT_CLEANUP event, rpc_clnt_notify calls the registered
  notifyfn with a RPC_CLNT_DESTROY event. The notifyfn should properly
  cleanup the saved mydata on this event.
- Break the reconnect chain when an rpc client is disabled. This will
  prevent new disconnect events which can lead to crashes.

glusterd:
- Added support for RPC_CLNT_DESTROY in glusterd_brick_rpc_notify
- Use a common glusterd_rpc_clnt_unref() function throught glusterd in
  place of rpc_clnt_unref(). This function correctly gives up the
  big-lock before performing the unref.

Change-Id: I93230441c5089039643fc9f5632477ef1b695348
BUG: 962619
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6566
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: Client cache invalidation with bad fsid

2013-12-17T18:26:10+00:00

1. Problem:
Couple of issues are seen when NFS-ACL is turned ON. i.e.
i) NFS directory access is too slow, impacting customer workflows
   with ACL
ii)dbench fails with 100 directories.

2. Root cause: Frequent cache invalidation in the client side when ACL
is turned ON with NFS because NFS server getacl() code returns the
wrong fsid to the client.

3. This attr-cache invlaidation triggers the frequent LOOKUP ops for
each file instead of relying on the readdir or readdirp data. As
a result performance gets impacted.

4. In case of dbench workload, the problem is more severe. e.g.

Client side rpcdebug output:
===========================

Dec 16 10:16:53 santosh-3 kernel: NFS:
         nfs_update_inode(0:1b/12061953567282551806 ct=2 info=0x7e7f)
Dec 16 10:16:53 santosh-3 kernel: NFS:
         nfs_fhget(0:1b/12061953567282551806 ct=2)
Dec 16 10:16:53 santosh-3 kernel: <-- nfs_xdev_get_sb() = -116 [splat]
Dec 16 10:16:53 santosh-3 kernel: nfs_do_submount: done
Dec 16 10:16:53 santosh-3 kernel: <-- nfs_do_submount() =
ffffffffffffff8c
Dec 16 10:16:53 santosh-3 kernel: <-- nfs_follow_mountpoint() =
ffffffffffffff8c
Dec 16 10:16:53 santosh-3 kernel: NFS: dentry_delete(clients/client77,
20008)

As per Jeff Layton, This occurs when the client detects that the fsid on
a filehandle is different from its parent. At that point, it tries to
do a new submount of the new filesystem onto the correct point. It means
client got a superblock reference for the new fs and is now looking to
set up the root of the mount. It calls nfs_get_root to do that, which
basically takes the superblock and a filehandle and returns a dentry.
The problem here is that the dentry->d_inode you're getting back looks
wrong. It's not a directory as expected -- it's something else. So the
client gives up and tosses back an ESTALE.

Which clearly says that, In getacl() code while it does the stat() call
to get the attrs, it forgets to populate the deviceid or fsid before
going ahead and does getxattr().

FIX:
1. Fill the deviceid in iatt.
2. Do bit more clean up for the confusing part of the code.

NB: Many many thanks to Niels de Vos and Jeff Layton for their
help to debug the issue.

Change-Id: I44d8d2fa3ec7fb33a67dfdd4bbe2c45cdf67db8c
BUG: 1043737
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6526
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: Inconsistent behaviour of setfacl/getfacl

2013-12-05T18:36:11+00:00

The permissions returned by NFS ACL are wrong, which are rejected
by NFS client as "Invalid argument". Refactor the NFS ACL code
to return the proper permissions which would match with the
requested permissions.

Upstream master review: http://review.gluster.org/6368

Change-Id: Ieb079b5da98b061291b44655e18a1dee92a8e463
BUG: 1035218
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6418
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

gNFS: More clean up for Gluster NFS

2013-11-25T13:40:57+00:00

1) Fix the typo in NFS default ACL
   The typo was introduced as part of the Fix to BZ 1009210 i.e.
   http://review.gluster.org/5980. The user ACL xattr structure
   was passed to default ACL xattr.

2) Clean up NFS code to avoid unnecessary SEGV in
   rpcsvc_drc_reconfigure() which was not validating the
   svc->drc. Add a routine rpcsvc_drc_deinit() to handle
   the clean up of DRC specific data structures. For init(),
   use rpcsvc_drc_init().

3) nfs_init_state() was returning wrong value even if the
   registration with portmapper failed, causing the NFS
   server process to hang around. As a result it used to
   get SEGV during rpcsvc_drc_reconfigure().

4) Clean up memfactor usage across nfs.c nfs3.c.

Change-Id: I5cea26cb68dd8a822ec0ae104952f67fe63fa703
BUG: 1009210
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6329
Reviewed-by: Rajesh Joseph 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

fuse: revalidate group id cache on uid/gid change detection

2013-11-21T21:11:18+00:00

- Remember the uid and gid of the pid at the time of caching
  the group id list.

- Next time when referring to the cache confirm that uid and gid
  of that pid has not changed since. If it has, treat it like
  a timeout/cache miss.

- Solves group id caching issue caused when Samba runs on gluster
  FUSE mount and changes the uid/gid on a per syscall basis.

Change-Id: I3382b037ff0b6d5eaaa36d9c898232543475aeda
BUG: 1032438
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6320
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

gNFS: RFE for NFS connection behavior

2013-11-15T00:07:02+00:00

Implement reconfigure() for NFS xlator so that volume set/reset wont
restart the NFS server process. But few options can not be reconfigured
dynamically e.g. nfs.mem-factor, nfs.port etc which needs NFS to be
restarted.

Change-Id: Ic586fd55b7933c0a3175708d8c41ed0475d74a1c
BUG: 1027409
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6236
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Anand Avati

gNFS: NFS segfaults with nfstest_posix tool

2013-11-05T01:33:37+00:00

Problem:
nfs3_stat_to_fattr3() missed a NULL check.

FIX:
(1) Added a NULL check.
(2) In all fop cbk path, if the op_ret is -1 and op_errno is 0,
    then handle it as a special case. Set the NFS3 status as
    NFS3ERR_SERVERFAULT instead of NFS3_OK.
(3) The other component of FIX would be in DHT module and
    is on the way.

Change-Id: I6f03c9a02d794f8b807574f2755094dab1b90c92
BUG: 1010241
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6026
Reviewed-by: Rajesh Joseph 
Reviewed-by: Niels de Vos 
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System