glusterfs.git/xlators/nfs/server/src, branch v3.10.0

gNFS: Keep the mountdict as long as the service is active

2017-02-15T23:19:57+00:00

We initialize and take ref once on mountdict during NFS/MNT3 server
initialization but seem to be unref'in it for every UMNTALL request.
This can lead to crash when there are multiple UMNTALL requests
with >=1 active mount entry(/ies) in the mountlist.

Since we take the ref only once, we should keep the mountdict through
out the life of the process and dereference it only during unitialization
of mnt3 service.

Cherry picked from commit a88ae92de190af0956013780939ba6bdfd509ff8:
> Change-Id: I3238a8df09b8972e56dd93fee426d866d40d9959
> BUG: 1421759
> Signed-off-by: Soumya Koduri 
> Reviewed-on: https://review.gluster.org/16611
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Niels de Vos 

Change-Id: I3238a8df09b8972e56dd93fee426d866d40d9959
BUG: 1422391
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16625
Reviewed-by: soumya k 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: jiffin tony Thottan 
CentOS-regression: Gluster Build System

core: run many bricks within one glusterfsd process

2017-02-02T00:54:58+00:00

This patch adds support for multiple brick translator stacks running in
a single brick server process.  This reduces our per-brick memory usage
by approximately 3x, and our appetite for TCP ports even more.  It also
creates potential to avoid process/thread thrashing, and to improve QoS
by scheduling more carefully across the bricks, but realizing that
potential will require further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global
option.  By default it's off, and bricks are started in separate
processes as before.  If multiplexing is enabled, then *compatible*
bricks (mostly those with the same transport options) will be started in
the same process.

Backport of:
> Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
> BUG: 1385758
> Reviewed-on: https://review.gluster.org/14763

Change-Id: I4bce9080f6c93d50171823298fdf920258317ee8
BUG: 1418091
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/16496
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

refcount: return pointer to the structure instead of a counter

2016-12-11T08:47:05+00:00

There are no real users of the counter. It was thought of a handy tool
to track and debug refcounting, but it is not used at all. Some parts of
the code would benefit from a pointer getting returned instead.

BUG: 1399780
Change-Id: I97e52c48420fed61be942ea27ff4849b803eed12
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15971
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Reviewed-by: Kaleb KEITHLEY

afr,dht,ec: Replace GF_EVENT_CHILD_MODIFIED with event SOME_DESCENDENT_DOWN/UP

2016-11-21T09:32:05+00:00

Currently these are few events related to child_up/down:
GF_EVENT_CHILD_UP :  Issued when any of the protocol client
connects.
GF_EVENT_CHILD_MODIFIED : Issued by afr/dht/ec
GF_EVENT_CHILD_DOWN : Issued when any of the protocol client
disconnects.
These events get modified at the dht/afr/ec layers. Here is a
brief on the same.

DHT:
- All the subvolumes reported once, and atleast one child came
  up, then GF_EVENT_CHILD_UP is issued
- connect GF_EVENT_CHILD_UP is issued
- disconnect GF_EVENT_CHILD_MODIFIED is issued
- All the subvolumes disconnected, GF_EVENT_CHILD_DOWN is issued

AFR:
- First subvolume came up, then GF_EVENT_CHILD_UP is issued
- Subsequent subvolumes coming up, results in GF_EVENT_CHILD_MODIFIED
- Any of the subvolumes go down, then GF_EVENT_SOME_CHILD_DOWN is issued
- Last up subvolume goes down, then GF_EVENT_CHILD_DOWN is issued

Until the patch [1] introduced GF_EVENT_SOME_CHILD_UP,
GF_EVENT_CHILD_MODIFIED was issued by afr/dht when any of the subvolumes
go up or down.

Now with md-cache changes, there is a necessity to differentiate between
child up and down. Hence, introducing GF_EVENT_SOME_DESCENDENT_DOWN/UP and
getting rid of GF_EVENT_CHILD_MODIFIED.

[1] http://review.gluster.org/12573

Change-Id: I704140b6598f7ec705493251d2dbc4191c965a58
BUG: 1396038
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15764
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Rajesh Joseph

nfs: revalidate lookup converted to fresh lookup

2016-11-10T22:19:08+00:00

when an inode ctx is missing for a linked inode the revalidate
lookups are converted to fresh.

This could result in sending ESTALE when the gfid are recreated

We are not able to reproduce the issue with normal setup, most part of
RCA was done with code reading.

Possible scenario in which  this bug can reproduce,
Delete a file and recreate a new file with same name, at the same time
from another client process try to list/or access the file.

In this case the second client may throw an ESTALE error for such files

Thanks to Soumya and Pranith for doing the complete RCA

Change-Id: I73992a65844b09a169cefaaedc0dcfb129d66ea1
BUG: 1379720
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/15580
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Kaleb KEITHLEY

trivial: correct some spelling mistakes in comments and logs

2016-10-18T10:27:12+00:00

BUG: 1385593
Change-Id: Icfae9e557a284182c6c22e9606fdd641528906f0
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15656
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kotresh HR 
Smoke: Gluster Build System

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

nfs: fix unused variable warnings/errors

2016-08-23T07:45:52+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I0e872a8025c3b1b5e2aa15d8fe66248e2fd96bf1
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15253
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

nfs: Reset cs->resolvedhard while resolving an entry

2016-07-17T07:58:50+00:00

If an entry is not found in the inode table, nfs xlator should be
resolving it by sending an explicit lookup to the brick process.
But currently its broken in case of NFS3_LOOKUP fop where in the server
bails out early resulting in sending pargfid attributes to the client.
To fix the same reset 'cs->resolvedhard' so that an explicit lookup
is done for the entry in the resume_fn "nfs3_lookup_resume()".

Change-Id: I999f8bca7ad008526c174d13f69886dc809d9552
Signed-off-by: Soumya Koduri 
BUG: 1356068
Reviewed-on: http://review.gluster.org/14911
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos

nfs: allow hostnames with dashes in exports/netgroups files

2016-06-28T20:29:13+00:00

Hostnames with dashes (like "vagrant-testVM") are not correctly parsed
when reading the exports/netgroups files. This bacomes obvious when
running ./run-tests-in-vagrant.sh because it causes
tests/basic/mount-nfs-auth.t and tests/basic/netgroup_parsing.t to fail.

The regex for hostname (in exports) and the entry and hostname
(netgroups) parsing does not include the "-" sign, and hence the
hostnames are splitted at it.

BUG: 1350237
Change-Id: I38146a283561e1fa386cc841c43fd3b1e30a87ad
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14809
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy