glusterfs.git/xlators/mgmt/glusterd, branch round-robin2

io-threads: remove least-rate-limit option and code

2016-07-25T12:58:31+00:00

This will be unnecessary, and mostly in the way, as real fairness
guarantees are implemented.

Change-Id: Ic61ec1c9e9add58385f1a4eafcfe2cc554ceefc8
Signed-off-by: Jeff Darcy

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-22T15:12:52+00:00

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1316178
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13658
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd/geo-rep: fix memory leak and minor fixes

2016-07-20T22:17:10+00:00

Fix for memory leak and other issues revealed by coverity.

Change-Id: Ib66eaa3a1a086d4b487534cebc56af5e1f9bf576
BUG: 1356528
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/14925
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Jeff Darcy

md-cache: Add cache invalidation support to invalidate the meta data cache

2016-07-20T12:12:05+00:00

Problem:
md-cache currently updates its stat in cbks of selected fops.
The default cache time is 1 second, if this is increasd to reap the
benefits of caching, we may end up with stale cache for long time,
as there is no logic yet to notify md-cache of backend changes by
another client.

Solution:
Use the existing upcall mechanism to invalidate the cache.
For this feature to work, "features.cache-invalidation" volume
option should be enabled.

This patch as is doesn't improve any performance, the benifit of the
patch is that it provides coherency for stat cache, hence the cache
timeout can be quite longer which in turn can improve the performance.

Change-Id: I2dbb0afa7b5e4a5a248f910188e0918e02f18692
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/12951
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

glusterd: search for free port from base_port

2016-07-18T12:55:58+00:00

When a volume is deleted, the freed up ports are never considered for further
allocation since pmap_registry_alloc () always starts scanning from last_alloc.
So in use cases where gluster volumes are frequently created and deleted
managing ports become nightmare as for every new volume creation ports need to
be opened up by the admin based on the volume topology.

Solution: Instead of scanning from last_alloc, pmap_registry_alloc () always
starts from base_port now. What that means is glusterd will always try to find
out the ports which have been freed from earlier volumes and reallocate them for
the newer ones. There could be possibilities that when a volume is stopped and
started back their brick ports are changed which is completely acceptible IMHO.

Change-Id: I99ccc11732b6a75527fcb6abafaf249ed02b3b78
BUG: 1221623
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/14939
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System

core: use readdir(3) with glibc, and associated cleanup

2016-07-18T11:59:42+00:00

Starting with glibc-2.23 (i.e. what's in Fedora 25), readdir_r(3)
is marked as deprecated. Specifically the function decl in 
has the deprecated attribute, and now warnings are thrown during the
compile on Fedora 25 builds.

The readdir(_r)(3) man page (on Fedora 25 at least) and World+Dog say
that glibc's readdir(3) is, and always has been, MT-SAFE as long as
only one thread is accessing the directory object returned by opendir().
World+Dog also says there is a potential buffer overflow in readdir_r().
World+Dog suggests that it is preferable to simply use readdir(). There's
an implication that eventually readdir_r(3) will be removed from glibc.
POSIX has, apparently deprecated it in the standard, or even removed it
entirely.

Over and above that, our source near the various uses of readdir(_r)(3)
has a few unsafe uses of strcpy()+strcat().

(AFAIK nobody has looked at the readdir(3) implemenation in *BSD to see
if the same is true on those platforms, and we can't be sure of MacOS
even though we know it's based on *BSD.)

Change-Id: I5481f18ba1eebe7ee177895eecc9a80a71b60568
BUG: 1356998
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/14838
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Jeff Darcy

glusterd: fix compilation warning

2016-07-18T10:58:30+00:00

glusterd-utils.c: In function 'glusterd_handle_replicate_brick_ops':
glusterd-utils.c:11402:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (dict_get_str (THIS->options, "transport.socket.bind-address",
         ^~
glusterd-utils.c:11406:17: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the 'if'
                 snprintf (logfile, sizeof (logfile),

Solution : indentation does the magic :)

Change-Id: I887fcba69ba1e952cc635d939e636d69e227f8b8
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/14937
Smoke: Gluster Build System 
Reviewed-by: Anuradha Talur 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

features/locks: Add lock revocation functionality to posix locks translator

2016-07-18T09:30:38+00:00

Summary:
- Motivation: Prevents cluster instability by mis-behaving clients
causing bricks to OOM due to inode/entry lock pile-ups.
- Adds option to strip clients of entry/inode locks after N seconds
- Adds option to clear ALL locks should the revocation threshold get hit
- Adds option to clear all or granted locks should the max-blocked
threshold get hit (can be used in combination w/ revocation-clear-all).
- Options are:
features.locks-revocation-secs
features.locks-revocation-clear-all [on/off]
features.locks-revocation-max-blocked
- Adds monkey-locking option to ignore 1% of unlock requests (dev only)
features.locks-monkey-unlocking [on/off]
- Adds logging to indicate revocation event & reason

Test Plan:
First you will need TWO fuse mounts for this repro. Call them /mnt/patchy1 & /mnt/patchy2.

1. Enable monkey unlocking on the volume:
gluster vol set patchy features.locks-monkey-unlocking on

2. From the "patchy1", use DD or some other utility to begin writing to a file,
eventually the dd will hang due to the dropped unlocked requests. This now
simulates the broken client. Run:

for i in {1..1000};do dd if=/dev/zero of=/mnt/patchy1/testfile bs=1k count=10;done'

...this will eventually hang as the unlock request has been lost.

3. Goto another window and setup the mount "patchy2" @ /mnt/patchy2, and
observe that 'echo "hello" >> /mnt/patchy2/testfile" will hang due to the
inability of the client to take out the required lock.

4. Next, re-start the test this time enabling lock revocation; use a timeout of
2-5 seconds for testing:
'gluster vol set patchy features.locks-revocation-secs <2-5>'

5. Wait 2-5 seconds before executing step 3 above this time. Observe that this
time the access to the file will succeed, and the writes on patchy1 will
unblock until they hit another failed unlock request due to
"monkey-unlocking".

BUG: 1350867
Change-Id: I814b9f635fec53834a26db634d1300d9a61057d8
Signed-off-by: Pranith Kumar K
Reviewed-on: http://review.gluster.org/14816
NetBSD-regression: NetBSD Build System
Reviewed-by: Krutika Dhananjay
CentOS-regression: Gluster Build System
Smoke: Gluster Build System

glusterd: Fix gsyncd upgrade issue

2016-07-13T15:02:54+00:00

Problem:
    gluster upgrade is not generating new volfiles

Cause:
During upgrade, "glusterd --xlator-option *.upgrade=on -N"
is run to generate new volfiles. It is run post 'glusterfs'
rpm installation. The above command fails during upgrade
if geo-replication is installed. This is because on
glusterd start 'gsyncd' binary is called to configure
geo-replication related stuff. Since 'glusterfs' rpm is
installed prior to 'geo-rep' rpm, the 'gsyncd' binary
used to glusterd upgrade command is of old version and
hence it fails before generating new volfiles.

Solution:
Don't call geo-replication configure during upgrade/downgrade.
Geo-replication configuration happens during start of glusterd
after upgrade.

Change-Id: Id58ea44ead9f69982f86fb68dc5b9ee3f6cd11a1
BUG: 1355628
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14898
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

gluster v set help does not show ssl options

2016-07-12T09:30:12+00:00

Problem: "gluster v set help" does not show ssl options.

Solution: Remove NO_DOC option for client.ssl/server.ssl from glusterd_volopt_map.

Change-Id: Iabe982ea56398209bbf30d41260798e5ad7fce7b
BUG: 1351134
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14829
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System