glusterfs.git/glusterfsd, branch v3.7.1

core: Global timer-wheel

2015-05-10T12:27:40+00:00

Instantiate a process wide global instance of the timer wheel
data structure. Spawning glusterfs* process with option arg
"--global-timer-wheel" instantiates a global instance of
timer-wheel under global context (->ctx).

Translators can make use of this process wide instance [via a
call to glusterfs_global_timer_wheel()] instead of maintaining
an instance of their own and possibly consuming more memory.
Linux kernel too has a single instance of timer wheel where
subsystems such as IO, networking, etc.. make use of.

Bitrot daemon would be early consumers of this: bitrot translator
instances for multiple volumes would track objects belonging to
their respective bricks in this global expiry tracking data
structure. This is also a first step to move GlusterFS timer
mechanism to use timer-wheel.

> Change-Id: Ie882df607e07acaced846ea269ebf1ece306d6ae
> BUG: 1170075
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10380
> Tested-by: NetBSD Build System
> Reviewed-by: Vijay Bellur 
> Tested-by: Gluster Build System 

Change-Id: I35c840daa9996a059699f8ea5af54c76ede7e09c
Signed-off-by: Venky Shankar 
Signed-off-by: Gaurav Kumar Garg 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10716
Tested-by: Gluster Build System

glusterd: gluster volume status should show status of bitrot and scrubber daemon

2015-05-07T03:59:46+00:00

Command gluster volume status  should show the status of bitrot
and scrubber daemon and its pid information.

Along with displaying bitrot and scrubber daemon information in gluster
volume status command there should be command to show its individual status
separately.
Command to show individual status of bitrot and scrubber daemon will
following.

command to show only bitd daemon information will be
gluster volume status  bitd

command to show only scrubber daemon information
gluster volume status  scrub

Change-Id: Id86aae1156c8c599347c98e2a538f294d37376e4
BUG: 1218123
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/10175
Reviewed-by: Kaushal M 
Tested-by: Kaushal M 
(cherry picked from commit da1416051d19d612d131acfde8589bc8658979b5)
Reviewed-on: http://review.gluster.org/10519
Tested-by: Gluster Build System

protocol: increase default group-cache-timeout to 300 seconds

2015-05-06T17:34:22+00:00

sssd uses 300 seconds by default too. There is no need to overload sssd
with requests that it would have cached.

Cherry picked from commit 34833364e9839f0036bccd58ec0a8a963e69263e:
> BUG: 1215187
> Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10371
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Vijay Bellur 

Change-Id: I3f04ea8cc90180d863253a9f46d62b71810a7b34
BUG: 1215189
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10523
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

glusterfsd: add "print-netgroups" and "print-exports" command

2015-03-19T05:16:34+00:00

NFS now has the ability to use a separate file for "netgroups" and
"exports". An administrator should have the ability to check the
validity of the files before applying the configuration.

The "glusterfsd" command now has the following additional arguments that
can be used to check the configuration:

   --print-netgroups: Validate the netgroups file and print it out
   --print-exports: Validate the exports file and print it out

BUG: 1143880
Change-Id: I24c40d50110d49d8290f9fd916742f7e4d0df85f
URL: http://www.gluster.org/community/documentation/index.php/Features/Exports_Netgroups_Authentication
Original-author: Shreyas Siravara 
CC: Richard Wareing 
CC: Jiffin Tony Thottan 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9365
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

protocol/client: sequence CHILD_UP, CHILD_DOWN etc notifications

2015-02-07T21:25:26+00:00

... from all bricks in the volume

This patch is important in the context of MT epoll. With MT epoll,
notification events from client xlators could reach cluster xlators like
afr, dht, ec, stripe etc. in different orders.

For e.g, In a distributed replicate volume of 2 bricks, namely Brick1
and Brick2, the following network events are observed by a mount
process.

- connection to Brick1 is broken.
- connection to Brick1 has been restored.

- connection to Brick2 is broken.
- connection to Brick2 has been restored.

Without establishing a total ordering of events, we can't guarantee that
cluster xlators like afr, dht perceive them in the same order.  While we
would expect afr (say) to perceive it as only one of Brick1 and Brick2
going down at any given time, it is possible for the notification of
Brick2 going offline to race with the notification of Brick1 coming back
online.

Change-Id: I78f5a52bfb05593335d0e9ad53ebfff98995593d
BUG: 1104462
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9591
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

epoll: Adding the ability to configure epoll threads

2015-02-07T21:23:03+00:00

Add the ability to configure the number of event threads
for various gluster services.

Currently with the multi thread epoll patch, it is possible
to have more than one thread waiting on socket activity and
processing the same. This thread count is currently static,
which this commit makes dynamic.

The current services which use IO path, i.e brick processes,
any client process (nfs, FUSE, gfapi, heal,
rebalance, etc.a), gain 2 set parameters to control the number
of threads that are processing events. These settings are,
  - client.event-threads 
  - server.event-threads 

The client setting affects the client graph consumers, and the
server setting affects the brick processes. These are processed
and inited/reconfigured using the client/server protocol xlators.

Other services (say glusterd) would need to extend similar
configuration settings to take advantage of multi threaded event
processing.

At present glusterd is not enabled with this commit, as it does not
stand to gain from this multi-threading (as I understand it).

Change-Id: Id8422fc57a9f95a135158eb6477ccf9d3c9ea4d9
BUG: 1104462
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/9488
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

uss: disable memory accounting for the snapshot daemon

2015-01-28T14:05:24+00:00

* Bring in option to disable memory accounting for a glusterfs process
  This reverses the changes done by the commit
  7fba3a88f1ced610eca0c23516a1e720d75160cd.

* Change the key from "memory-accounting" to "no-memory-accounting", as by
  default all the glusterfs process enable memory accounting now. So to
  disable memory accounting for some process, "no-mem-accounting" argument has
  to be passed.

Change-Id: I39c7cefb0fe764ea3e48f4e73e1305b084c5f497
BUG: 1184366
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/9469
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

api, glusterfsd: Cleanup upon failure in glusterfs_ctx_defaults_init()

2014-12-10T04:11:17+00:00

This got introduced due to 656711d935000c16. Coverity
also picked this up as CIDs 1256176, 1256178, 1256180.

Signed-off-by: Vijay Bellur 
Change-Id: If12fa0075634383975846181917a2f9650f790e3
BUG: 789278
Reviewed-on: http://review.gluster.org/9213
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Shyamsundar Ranganathan

rdma: client connection establishment takes more time

2014-11-18T08:50:13+00:00

For rdma type only volume client connection establishment
with server takes more than three seconds. Because for
tcp,rdma type volume, will have 2 ports one for tcp and
one for rdma, tcp port is stored with brickname and rdma
port is stored as "brickname.rdma" during pamap_sighin.
During the handshake when trying to get the brick port
for rdma clients, since we are not aware of server
transport type, we will append '.rdma' with brick name.
So for tcp,rdma volume there will be an entry with
'.rdma', but it will fail for rdma type only volume.
So we will try again, this time without appending '.rdma'
using a flag variable need_different_port, and it will succeed,
but the reconnection happens only after 3 seconds.
In this patch for rdma only type volume 
we will append '.rdma' during the pmap_signin. So during the
handshake we will get the correct port for first try itself.
Since we don't need to retry , we can remove the
need_different_port flag variable.

Change-Id: Ie8e3a7f532d4104829dbe995e99b35e95571466c
BUG: 1153569
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/8934
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

rdma:rdma fuse mount hangs for tcp,rdma volumes if brick is down.

2014-11-18T07:30:55+00:00

When we try to mount a tcp,rdma volume as rdma
transport using FUSE protocol, then mount will
hang if the brick is down. When we kill a process,
signal will be received in glusterfsd process and
it will call pmap_signout with port listening on tcp only.
In case of the tcp,rdma there will be two ports,
and port which is listening for rdma will not
called for sign out. 
So the mount process will try to connect to a port
which is not open and it will keep trying to connect.
This patch will call pmap_signout for rdma port also,
So when mount tries to get the brick port,it will fail.

Change-Id: I23676f65f96eb90b69b76478f7a21412a6aba70f
BUG: 1143886
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/8762
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G