glusterfs.git/xlators/mount/fuse/utils, branch v4.0dev

fuse: implement "-oauto_unmount"

2017-05-23T13:21:25+00:00

libfuse has an auto_unmount option which,
if enabled, ensures that the file system
is unmounted at FUSE server termination
by running a separate monitor process
that performs the unmount when that
occurs. (This feature would probably
better be called "robust auto-unmount",
as FUSE servers usually do try to unmount
their file systems upon termination,
it's just this mechanism is not crash
resilient.)

This change implements that option and
behavior for glusterfs.

Note that "auto unmount" (robust or not) is
a leaky abstraction, as the kernel cannot
guarantee that at the path where the FUSE
fs is mounted is actually the toplevel mount
at the time of the umount(2) call, for
multiple reasons, among others, see:

  fuse-devel: "fuse: feasible to distinguish between umount and abort?"
  http://fuse.996288.n3.nabble.com/fuse-feasible-to-distinguish-between-umount-and-abort-tt14358.html
  https://github.com/libfuse/libfuse/issues/122

Updates #153

Change-Id: Ia4432580c9fd2c156d9c73c3a44f4bfd42437599
Signed-off-by: Csaba Henk 
Reviewed-on: https://review.gluster.org/17230
Tested-by: Amar Tumballi 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Amar Tumballi 
NetBSD-regression: NetBSD Build System

Halo Replication feature for AFR translator

2017-05-02T10:23:53+00:00

Summary:
Halo Geo-replication is a feature which allows Gluster or NFS clients to write
locally to their region (as defined by a latency "halo" or threshold if you
like), and have their writes asynchronously propagate from their origin to the
rest of the cluster.  Clients can also write synchronously to the cluster
simply by specifying a halo-latency which is very large (e.g. 10seconds) which
will include all bricks.

In other words, it allows clients to decide at mount time if they desire
synchronous or asynchronous IO into a cluster and the cluster can support both
of these modes to any number of clients simultaneously.

There are a few new volume options due to this feature:
  halo-shd-latency:  The threshold below which self-heal daemons will
  consider children (bricks) connected.

  halo-nfsd-latency: The threshold below which NFS daemons will consider
  children (bricks) connected.

  halo-latency: The threshold below which all other clients will
  consider children (bricks) connected.

  halo-min-replicas: The minimum number of replicas which are to
  be enforced regardless of latency specified in the above 3 options.
  If the number of children falls below this threshold the next
  best (chosen by latency) shall be swapped in.

New FUSE mount options:
  halo-latency & halo-min-replicas: As descripted above.

This feature combined with multi-threaded SHD support (D1271745) results in
some pretty cool geo-replication possibilities.

Operational Notes:
- Global consistency is gaurenteed for synchronous clients, this is provided by
  the existing entry-locking mechanism.
- Asynchronous clients on the other hand and merely consistent to their region.
  Writes & deletes will be protected via entry-locks as usual preventing
  concurrent writes into files which are undergoing replication.  Read operations
  on the other hand should never block.
- Writes are allowed from _any_ region and propagated from the origin to all
  other regions.  The take away from this is care should be taken to ensure
  multiple writers do not write the same files resulting in a gfid split-brain
  which will require resolution via split-brain policies (majority, mtime &
  size).  Recommended method for preventing this is using the nfs-auth feature to
  define which region for each share has RW permissions, tiers not in the origin
  region should have RO perms.

TODO:
- Synchronous clients (including the SHD) should choose clients from their own
  region as preferred sources for reads.  Most of the plumbing is in place for
  this via the child_latency array.
- Better GFID split brain handling & better dent type split brain handling
  (i.e. create a trash can and move the offending files into it).
- Tagging in addition to latency as a means of defining which children you wish
  to synchronously write to

Test Plan:
- The usual suspects, clang, gcc w/ address sanitizer & valgrind
- Prove tests

Reviewers: jackl, dph, cjh, meyering

Reviewed By: meyering

Subscribers: ethanr

Differential Revision: https://phabricator.fb.com/D1272053

Tasks: 4117827

Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
BUG: 1428061
Signed-off-by: Kevin Vigor 
Reviewed-on: http://review.gluster.org/16099
Reviewed-on: https://review.gluster.org/16177
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterfsd/main: Add ability to set oom_score_adj

2016-06-01T13:27:10+00:00

Give the administrator a possibility to set oom_score_adj for glusterfs
process. Applies to Linux only.

Change-Id: Iff13c2f4cb28457871c6ebeff6130bce4a8bf543
BUG: 1336818
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/14399
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

fuse: accept the -s option to allow automounting

2016-05-31T09:01:41+00:00

autofs passes the -s option when mounting. All /sbin/mount. helpers
accept this, except mount.glusterfs. Because the helper fails when -s is
passed accessing the mountpoint through autofs gives the following
error:

  $ ls /lan/storage.lan.example.net/repos
  ls: cannot open directory /lan/storage.lan.example.net/repos: Too many levels of symbolic links

BUG: 1340936
Change-Id: I84755cdac59e630618cb745c0eb3228cc1e93a1a
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14559
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Oleksandr Natalenko

fuse: Add a new mount option capability

2016-03-08T05:57:28+00:00

Originally all security.* xattrs were forbidden if selinux is disabled,
which was causing Samba's acl_xattr module to not work, as it would
store the NTACL in security.NTACL. To fix this http://review.gluster.org/#/c/12826/
was sent, which forbid only security.selinux. This opened up a getxattr
call on security.capability before every write fop and others.

Capabilities can be used without selinux, hence if selinux is disabled,
security.capability cannot be forbidden. Hence adding a new mount
option called capability.

Only when "--capability" or "--selinux" mount option is used,
security.capability is sent to the brick, else it is forbidden.

Change-Id: I77f60e0fb541deaa416159e45c78dd2ae653105e
BUG: 1309462
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13540
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd: Bug fixes for IPv6 support

2016-02-20T17:16:42+00:00

Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
   needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.

Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.

Tests Run:
Regression tests using ./run-tests.sh
    IPv4: Ran manually till tests/basic/rpm.t .
    IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
        Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here

Unit Tests using Ipv6
    peer probe
    add-bricks
    remove-bricks
    create volume
    replace-bricks
    start volume
    stop volume
    delete volume

Change-Id: Iebc96e6cce748b5924ce5da17b0114600ec70a6e
BUG: 1117886
Signed-off-by: Nithin D 
Reviewed-on: http://review.gluster.org/11988
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Jeff Darcy

fuse: pass standard mount options to the kernel

2016-01-24T11:43:14+00:00

Some of the default mount options were made invalid with glusterfs-3.6.
The /sbin/mount.glusterfs script changed heavily and now requires all
valid mount options to be listed. Earlier versions (glusterfs-3.5 and
before) passed all unknown mount options on to fuse.

With this change, all mount options from 'man 8 mount' are explicitly
included in the /sbin/mount.glusterfs script. Some of the options are
marked with TODO, these are not commonly used and may require some
additional support in Gluster/FUSE too.

BUG: 1294809
Change-Id: Ic312140d7318b54523996bb08772ff065af7eb27
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/13166
Reviewed-by: Kaleb KEITHLEY 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

no-mtab (-n) mount option ignore next mount option

2016-01-10T19:28:43+00:00

The -n option does not take any arguments. It seems like this shift is
removing the next option. On my CentOS 7 system, automount calls
mount.glusterfs with the parameters:
 host:/volume /mountpoint -n -o rw,acl,_netdev
This causes the -o option to be siliently ignored.

Change-Id: Ice3c877f6ab346b04292e3dfed968d04d15077a5
BUG: 1297195
Signed-off-by: James Augustine 
Reviewed-on: http://review.gluster.org/12988
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos

fuse: pass default SElinux mount options on to the kernel

2015-12-03T10:31:01+00:00

In order to set default SElinux contexts on a Gluster mount, the
standard SElinux mount options need to be passed to the kernel. The
mount(8) manual page lists "context", "fscontext", "defcontext" and
"rootcontext" as valid options.

BUG: 1287763
Change-Id: I015fe27e4c6ff36a030e3480b23141aca2d91fc2
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/12858
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Humble Devassy Chirammal 
Reviewed-by: Manikandan Selvaganesh

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-08-05T11:53:12+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1246275
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11732
Tested-by: NetBSD Build System 
Reviewed-by: Ravishankar N 
Tested-by: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kaleb KEITHLEY