glusterfs.git/xlators, branch round-robin2

io-threads: distribute work fairly among clients

2016-07-25T12:58:49+00:00

This is the full "queue of queues" approach where each client gets its
own queue (per priority) and we round-robin among them.

Change-Id: I73955d1b9bb93f2ff781b48dfe2509009c519ec6
Signed-off-by: Jeff Darcy

io-threads: remove least-rate-limit option and code

2016-07-25T12:58:31+00:00

This will be unnecessary, and mostly in the way, as real fairness
guarantees are implemented.

Change-Id: Ic61ec1c9e9add58385f1a4eafcfe2cc554ceefc8
Signed-off-by: Jeff Darcy

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-22T15:12:52+00:00

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1316178
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13658
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

jbr: coverity, resource leak

2016-07-22T15:12:15+00:00

Change-Id: I62428808ce52569ff0298b0fcdd07cbaec1621dc
BUG: 1358944
CID: 1357871
Signed-off-by: Zhou Zhengping 
Reviewed-on: http://review.gluster.org/14977
Reviewed-by: Vijay Bellur 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

md-cache: fix indention to silence Coverity

2016-07-21T18:05:59+00:00

Coverity complains about the incorrect indention:

    *** CID 1357867:  Control flow issues  (NESTING_INDENT_MISMATCH)
    ...
    2566                     if (conf->mdc_invalidation)
    2567                             ret = mdc_invalidate (this, data);
    >>>     CID 1357867:  Control flow issues  (NESTING_INDENT_MISMATCH)
    >>>     This 'if'  statement is indented to column 25, as if it were nested within the preceding parent statement, but it is not.
    2568                             if (default_notify (this, event, data) != 0)
    2569      	                         ret = -1;
    2570                     break;
    ...

Even when md-cache does not have cache-invalidation on, we need to pass
the upcall to the next xlator.

Change-Id: I6d2a174eb54e3df270920aae9673b5010c235f25
CID: 1357867
BUG: 1211863
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14971
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Poornima G 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Vijay Bellur

feature/bitrot: Ignore files with sticky bit set

2016-07-21T13:59:49+00:00

Scrubber scrubs entries in backend. It is scrubbing files with sticky
bit as well. This might include linkfiles which should be skipped.  This
patch adds the check to ignore linkfiles during scrub.

Change-Id: Ic21367b37770d391326c55c659491a1e5a82335b
BUG: 1355706
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14903
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
CentOS-regression: NetBSD Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

glusterd/geo-rep: fix memory leak and minor fixes

2016-07-20T22:17:10+00:00

Fix for memory leak and other issues revealed by coverity.

Change-Id: Ib66eaa3a1a086d4b487534cebc56af5e1f9bf576
BUG: 1356528
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/14925
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Jeff Darcy

md-cache: Add cache invalidation support to invalidate the meta data cache

2016-07-20T12:12:05+00:00

Problem:
md-cache currently updates its stat in cbks of selected fops.
The default cache time is 1 second, if this is increasd to reap the
benefits of caching, we may end up with stale cache for long time,
as there is no logic yet to notify md-cache of backend changes by
another client.

Solution:
Use the existing upcall mechanism to invalidate the cache.
For this feature to work, "features.cache-invalidation" volume
option should be enabled.

This patch as is doesn't improve any performance, the benifit of the
patch is that it provides coherency for stat cache, hence the cache
timeout can be quite longer which in turn can improve the performance.

Change-Id: I2dbb0afa7b5e4a5a248f910188e0918e02f18692
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/12951
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

feature/bitrot: Fix scrub status with sharded volume

2016-07-19T14:41:09+00:00

Bitrot scrubs each shard entries separately. Scrub
statistics was counting each shard entry which is
incorrect. This patch skips the statistics count
for sharded entries.

Change-Id: I184c315a4bc7f2cccabc506eef083ee926ec26d3
BUG: 1356851
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14927
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Vijay Bellur

xlator/trash : append '/' at the end in trash_notify_lookup_cbk

2016-07-19T13:16:11+00:00

In the notify function in trash xlator, a lookup is performed
to obtain path of old trash directory. The result usually contains
path without '/' at the end. The trash xlator maintains expects
'/' at the end for the values such as 'old trash dir' and
'new trash dir'. Otherwise certian checks in the code will fail.

Change-Id: I89e02e4b249314fb6536297f959865feee182c83
BUG: 1357397
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/14938
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Anoop C S 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy