glusterfs.git/glusterfsd/src, branch v3.7.6

fuse: add "resolve-gids" mount option to overcome 32-groups limit

2015-09-28T09:51:08+00:00

Add a --resolve-gids commandline option to the glusterfs binary. This
option gets set when executing "mount -t glusterfs -o resolve-gids ...".

This option is most useful in combination with the "acl" mount option.
POSIX ACL permission checking is done on the FUSE-client side to improve
performance (in addition to the checking on the bricks).

The fuse-bridge reads /proc/$PID/status by default, and this file
contains maximum 32 groups. Any local (client-side) permission checking
that requires more than the first 32 groups will fail.

By enabling the "resolve-gids" option, the fuse-bridge will call
getgrouplist() to retrieve all the groups from the user accessing the
mountpoint. This is comparable to how "nfs.server-aux-gids" works.

Note that when a user belongs to more than ~93 groups, the volume option
server.manage-gids needs to be enabled too. Without this option, the
RPC-layer will need to reduce the number of groups to make them fit in
the RPC-header.

Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417:
> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
> BUG: 1246275
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/11732
> Tested-by: NetBSD Build System 
> Reviewed-by: Ravishankar N 
> Tested-by: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2
BUG: 1246397
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11875
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

glusterfsd : newly added brick receives fops only after it is started

2015-09-24T23:11:40+00:00

When new bricks are added in the middle of an on-going
fop like 'rm', the volfile changes without waiting for
the newly added bricks to get port. Fops are sent to all
bricks and may fail on some with ENOTCONN as these bricks
may not have a port yet.

This patch ensures that the volfile change happens only
after all the bricks have a port.

> Backport of http://review.gluster.org/#/c/11342/

> Change-Id: I7ed2413475f80d0cc8849fed33036ade8d75a191
> BUG: 1233151
> Signed-off-by: Sakshi 
> Reviewed-on: http://review.gluster.org/11342
> Reviewed-by: Atin Mukherjee 
> Tested-by: Atin Mukherjee 

Change-Id: I7ed2413475f80d0cc8849fed33036ade8d75a191
BUG: 1265890
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/12223
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

rpc: add owner xlator argument to rpc_clnt_new

2015-08-14T10:10:47+00:00

The @owner argument tells RPC layer the xlator that owns
the connection and to which xlator THIS needs be set during
network notifications like CONNECT and DISCONNECT.

Code paths that originate from the head of a (volume) graph and use
STACK_WIND ensure that the RPC local endpoint has the right xlator saved
in the frame of the call (callback pair). This guarantees that the
callback is executed in the right xlator context.

The client handshake process which includes fetching of brick ports from
glusterd, setting lk-version on the brick for the session, don't have
the correct xlator set in their frames. The problem lies with RPC
notifications. It doesn't have the provision to set THIS with the xlator
that is registered with the corresponding RPC programs. e.g,
RPC_CLNT_CONNECT event received by protocol/client doesn't have THIS set
to its xlator. This implies, call(-callbacks) originating from this
thread don't have the right xlator set too.

The fix would be to save the xlator registered with the RPC connection
during rpc_clnt_new. e.g, protocol/client's xlator would be saved with
the RPC connection that it 'owns'. RPC notifications such as CONNECT,
DISCONNECT, etc inherit THIS from the RPC connection's xlator.

Change-Id: I9dea2c35378c511d800ef58f7fa2ea5552f2c409
BUG: 1253212
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/11436
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
(cherry picked from commit f7668938cd7745d024f3d2884e04cd744d0a69ab)
Reviewed-on: http://review.gluster.org/11908
Reviewed-by: Pranith Kumar Karampuri

Porting new log messages for posix

2015-06-25T06:59:47+00:00

        Backport of http://review.gluster.org/#/c/9893/

Cherry picked from 59e1c79ac6953e552fbdc8c627f5dc1d02eeacc8
>Change-Id: I29bdeefb755805858e3cb1817b679cb6f9a476a9
>BUG: 1194640
>Signed-off-by: Hari Gowtham 
>Reviewed-on: http://review.gluster.org/9893
>Reviewed-by: Pranith Kumar Karampuri 
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Signed-off-by: Hari Gowtham 

Change-Id: I29bdeefb755805858e3cb1817b679cb6f9a476a9
BUG: 1217722
Signed-off-by: Hari Gowtham 
Signed-off-by: Hari Gowtham 
Reviewed-on: http://review.gluster.org/11279
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Enabling the fini() in cleanup_and_exit()

2015-06-17T04:15:11+00:00

Problem 1 : glusterd was crashing due to race between clean up thread and rpc event thread.

Scenario:
As we can observed, X thread is in the process of exiting the process. It has already
run the exit handlers, which cleanup things that require cleaning up. This includes
liburcu resources. By the time Y thread  called rcu_bp_register(), the liburcu resources
have been cleaned up. rcu_bp_register() tries to access these non-existent resources,
which leads to the segmentation fault.

Note1:
Crash happen when the process is almost at the point of stopping(exiting), it doesn't have any
serious impact to functionality apart from creating the core dump file and the log message.

Fix .Do proper clean up before calling exit().

Note2: Other xlator have clean up issues,so only glusterd clean up function invoked.

Note3: This patch also solve the selinux issue.

Problem 2 : glusterd runs as rpm_script_t when it's executed from the rpm scriptlet,files created
in this context are set as rpm_script_t, so glusterd unable to access these files when it runs
in glusterd_t context.

Fix: Fini clean up the files while glusterd exiting, so files are recreated by glusterd while
starting with proper SElinux context label.

Backport of :
>Change-Id: Idcfd087f51c18a729bdf44a146f9d294e2fca5e2
>BUG: 1209461
>Signed-off-by: anand 
>Reviewed-on: http://review.gluster.org/10894
>Tested-by: Gluster Build System 
>Reviewed-by: Atin Mukherjee 
>Tested-by: NetBSD Build System 
>Reviewed-by: Raghavendra Bhat 
>Reviewed-by: Vijay Bellur 

Change-Id: I59579e675bd73d7a19f7b965bd2c3c0fcd95d241
BUG: 1230026
Signed-off-by: anand 
Reviewed-on: http://review.gluster.org/11155
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Atin Mukherjee

core: Global timer-wheel

2015-05-10T12:27:40+00:00

Instantiate a process wide global instance of the timer wheel
data structure. Spawning glusterfs* process with option arg
"--global-timer-wheel" instantiates a global instance of
timer-wheel under global context (->ctx).

Translators can make use of this process wide instance [via a
call to glusterfs_global_timer_wheel()] instead of maintaining
an instance of their own and possibly consuming more memory.
Linux kernel too has a single instance of timer wheel where
subsystems such as IO, networking, etc.. make use of.

Bitrot daemon would be early consumers of this: bitrot translator
instances for multiple volumes would track objects belonging to
their respective bricks in this global expiry tracking data
structure. This is also a first step to move GlusterFS timer
mechanism to use timer-wheel.

> Change-Id: Ie882df607e07acaced846ea269ebf1ece306d6ae
> BUG: 1170075
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10380
> Tested-by: NetBSD Build System
> Reviewed-by: Vijay Bellur 
> Tested-by: Gluster Build System 

Change-Id: I35c840daa9996a059699f8ea5af54c76ede7e09c
Signed-off-by: Venky Shankar 
Signed-off-by: Gaurav Kumar Garg 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10716
Tested-by: Gluster Build System

glusterd: gluster volume status should show status of bitrot and scrubber daemon

2015-05-07T03:59:46+00:00

Command gluster volume status  should show the status of bitrot
and scrubber daemon and its pid information.

Along with displaying bitrot and scrubber daemon information in gluster
volume status command there should be command to show its individual status
separately.
Command to show individual status of bitrot and scrubber daemon will
following.

command to show only bitd daemon information will be
gluster volume status  bitd

command to show only scrubber daemon information
gluster volume status  scrub

Change-Id: Id86aae1156c8c599347c98e2a538f294d37376e4
BUG: 1218123
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/10175
Reviewed-by: Kaushal M 
Tested-by: Kaushal M 
(cherry picked from commit da1416051d19d612d131acfde8589bc8658979b5)
Reviewed-on: http://review.gluster.org/10519
Tested-by: Gluster Build System

protocol: increase default group-cache-timeout to 300 seconds

2015-05-06T17:34:22+00:00

sssd uses 300 seconds by default too. There is no need to overload sssd
with requests that it would have cached.

Cherry picked from commit 34833364e9839f0036bccd58ec0a8a963e69263e:
> BUG: 1215187
> Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10371
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Vijay Bellur 

Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
BUG: 1215189
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10523
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

glusterfsd: add "print-netgroups" and "print-exports" command

2015-03-19T05:16:34+00:00

NFS now has the ability to use a separate file for "netgroups" and
"exports". An administrator should have the ability to check the
validity of the files before applying the configuration.

The "glusterfsd" command now has the following additional arguments that
can be used to check the configuration:

   --print-netgroups: Validate the netgroups file and print it out
   --print-exports: Validate the exports file and print it out

BUG: 1143880
Change-Id: I24c40d50110d49d8290f9fd916742f7e4d0df85f
URL: http://www.gluster.org/community/documentation/index.php/Features/Exports_Netgroups_Authentication
Original-author: Shreyas Siravara 
CC: Richard Wareing 
CC: Jiffin Tony Thottan 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9365
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

protocol/client: sequence CHILD_UP, CHILD_DOWN etc notifications

2015-02-07T21:25:26+00:00

... from all bricks in the volume

This patch is important in the context of MT epoll. With MT epoll,
notification events from client xlators could reach cluster xlators like
afr, dht, ec, stripe etc. in different orders.

For e.g, In a distributed replicate volume of 2 bricks, namely Brick1
and Brick2, the following network events are observed by a mount
process.

- connection to Brick1 is broken.
- connection to Brick1 has been restored.

- connection to Brick2 is broken.
- connection to Brick2 has been restored.

Without establishing a total ordering of events, we can't guarantee that
cluster xlators like afr, dht perceive them in the same order.  While we
would expect afr (say) to perceive it as only one of Brick1 and Brick2
going down at any given time, it is possible for the notification of
Brick2 going offline to race with the notification of Brick1 coming back
online.

Change-Id: I78f5a52bfb05593335d0e9ad53ebfff98995593d
BUG: 1104462
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9591
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur