glusterfs.git/glusterfsd/src, branch release-3.6

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-21T06:45:07+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1252072
Tested-by: Patrick Glomski 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11876
Tested-by: Gluster Build System 
Reviewed-by: Anoop C S 
Reviewed-by: Raghavendra Bhat

rdma: client connection establishment takes more time

2015-01-06T09:43:45+00:00

        Backport of http://review.gluster.org/8934

For rdma type only volume client connection establishment
with server takes more than three seconds. Because for
tcp,rdma type volume, will have 2 ports one for tcp and
one for rdma, tcp port is stored with brickname and rdma
port is stored as "brickname.rdma" during pamap_sighin.
During the handshake when trying to get the brick port
for rdma clients, since we are not aware of server
transport type, we will append '.rdma' with brick name.
So for tcp,rdma volume there will be an entry with
'.rdma', but it will fail for rdma type only volume.
So we will try again, this time without appending '.rdma'
using a flag variable need_different_port, and it will succeed,
but the reconnection happens only after 3 seconds.
In this patch for rdma only type volume
we will append '.rdma' during the pmap_signin. So during the
handshake we will get the correct port for first try itself.
Since we don't need to retry , we can remove the
need_different_port flag variable.

Change-Id: I82a8a27f0e65a2e287f321e5e8292d86c6baf5b4
BUG: 1166515
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/8934
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/9177
Reviewed-by: Raghavendra Bhat

rdma:rdma fuse mount hangs for tcp,rdma volumes if brick is down.

2015-01-06T09:39:15+00:00

        Backport of http://review.gluster.org/8762

When we try to mount a tcp,rdma volume as rdma
transport using FUSE protocol, then mount will
hang if the brick is down. When we kill a process,
signal will be received in glusterfsd process and
it will call pmap_signout with port listening on tcp only.
In case of the tcp,rdma there will be two ports,
and port which is listening for rdma will not
called for sign out.
So the mount process will try to connect to a port
which is not open and it will keep trying to connect.
This patch will call pmap_signout for rdma port also,
So when mount tries to get the brick port,it will fail.

Change-Id: I73f90d7340afa3b0b1278924206f1488e4094a62
BUG: 1166515
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/8762
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/9176
Reviewed-by: Raghavendra Bhat

Enable aux-gfid-mount option on non Linux systems

2014-09-08T06:19:55+00:00

The aux-gfid-mount works on non Linux systems, and it is required
to pass tests/basic/gfid-access.t

This is a backport of Ic6c8ef425e091440a139bbd25fadbf4f82e378cb

BUG: 1138897
Change-Id: Iab5487437fde65a66cff9cfeb7d209db7eb12108
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8632
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-09-03T18:01:45+00:00

Backport from master branch - http://review.gluster.org/#/c/8246/

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System

socket/glusterd/client: enable SSL for management

2014-07-10T14:37:12+00:00

The feature is controlled by presence of the following file:

	/var/lib/glusterd/secure-access

See the comment near the definition of SECURE_ACCESS_FILE in glusterfs.h
for the rationale.  With this enabled, the following rules apply to
connections:

	UNIX-domain sockets never have SSL.

	Management-port sockets (both connecting and accepting, in
	daemons and CLI) have SSL based on presence of the file.

	Other IP sockets have SSL based on the existing client.ssl and
	server.ssl volume options.

Transport multi-threading is explicitly turned off in glusterd (it would
otherwise be turned on when SSL is) due to multi-threading issues.
Tests have been elided to avoid risk of leaving a file which will cause
all subsequent tests to run with management SSL still enabled.

IMPLEMENTATION NOTE
The implementation is a bit messy, and consists of two stages.  First we
decide whether to set the relevant fields in our context structure, based
on presence of the sentinel file OR a command-line override.  Later we
decide whether a particular connection should actually use SSL, based on the
context flags plus what kind of connection we're making[1] and what kind of
daemon we're in[2].

[1] inbound, outbound to glusterd port, other outbound
[2] glusterd, glusterfsd, other

TESTING NOTE
Instead of just running one special test for this feature, the ideal
would be to run all tests with management SSL enabled.  However, it
would be inappropriate or premature to set up an optional feature in the
patch itself.  Therefore, the method of choice is to submit a separate
patch on top, which modifies "cleanup" in include.rc to recreate the
secure-access file and associated SSL certificate/key files before each
test.

Change-Id: I0e04d6d08163893e24ec8c031748c5c447d7f780
BUG: 1114604
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8094
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

rpcsvc: Validate RPC procedure number before fetch

2014-05-17T18:56:01+00:00

While accessing the procedures of given RPC program in,
rpcsvc_get_program_vector_sizer(), It was not checking boundary
conditions which would cause buffer overflow and subsequently SEGV.

Make sure rpcsvc_actor_t arrays have numactors number of actors.

FIX:
Validate the RPC procedure number before fetching the actor.

Special Thanks to: Murray Ketchion, Grant Byers

Change-Id: I8b5abd406d47fab8fca65b3beb73cdfe8cd85b72
BUG: 1096020
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/7726
Tested-by: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Anand Avati

glusterd/snashot: Perform missed snap creates

2014-05-06T07:25:11+00:00

When a brick is started, and the glusterfsd process requests
for volfile, the brick_name is sent in the req dict. In
glusterd, after fetching the spec the brick_name is looked
up in the missed_snap_list, and any missing snap creates on
the same brick are performed. After this, the glusterd
responds back with the specfile.

Also collate brick data from the node's hosting the bricks
during restore. In case the data is absent, the local node's
data is used. This is needed to ensure that, during a restore
we collect the information created when a missed snap create
is performed.

Change-Id: I47cefdeba96f2702be810965734cf0fac61d3d2d
BUG: 1061685
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7551
Reviewed-by: Santosh Pradhan 
Reviewed-by: Rajesh Joseph 
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

meta: (re-)Implement Meta translator

2014-05-06T00:28:55+00:00

The meta translator exposes details about glusterfs itself
in the form of a virtual namespace.

Loading the translator on the client side creates the
meta virtual view under $mntpoint/.meta by default. The
directory is not listed (even with ls -a) and can be
accessed by doing a "cd /mnt/.meta"

Change-Id: I5ffdf39203841a9562a8280a1f79dc76d4dded5d
BUG: 1089216
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/7509
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana

mem-accounting: enable memory accounting by default

2014-05-06T00:28:29+00:00

memory accounting are constant time operations which
involve a few pointer dereferences and integer increments
(no loops or searches etc.)

benefits of having memory usage info outweigh the minor
accounting overheads

Change-Id: If9bc6db5ffd0e00f0fd64b2f6eed094bf3543996
BUG: 1089216
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/7543
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana