glusterfs.git/xlators/mgmt/glusterd/src, branch v3.8.2

glusterd: Fix memory leak in glusterd (un)lock RPCs

2016-08-10T10:32:28+00:00

Problem:  At the time of execute "gluster volume profile  info" command
          It does have memory leak in glusterd.

Solution: Modify the code to prevent memory leak in glusterd.

Fix    : 1) Unref dict and free dict_val buffer in glusterd_mgmt_v3_lock_peer and
            glusterd_mgmt_v3_unlock_peers.

Test   : To verify the patch run below loop to generate io traffic
         for (( i=0 ; i<=1000000 ; i++ ));
           do echo "hi Start Line " > file$i;
           cat file$i >> /dev/null;
         done

         To verify the improvement in memory leak specific to glusterd run below command

         cnt=0;while [ $cnt -le 1000 ]; do
         pmap -x  | grep total;
         gluster volume profile distributed info > /dev/null; cnt=`expr $cnt + 1`; done

         After apply this patch it will reduce leak significantly.

> Reviewed-on: http://review.gluster.org/14862
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Prashanth Pai 

BUG: 1365743
Change-Id: I52a0ca47adb20bfe4b1848a11df23e5e37c5cea9
Signed-off-by: Mohit Agrawal 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15125
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd : skip non directories inside /var/lib/glusterd/vols

2016-08-09T15:03:14+00:00

Right now glusterd won't come up if vols directory contains an invalid entry.
Instead of doing that with this change a message will be logged and then skip
that entry

Backport details: 
>Change-Id: I665b5c35291b059cf054622da0eec4db44ec5f68
>BUG: 1318591
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/13764
>Reviewed-by: Prashanth Pai 
>Reviewed-by: Atin Mukherjee 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
(cherry picked from commit 720b63c24b07ee64e1338db28de602b9abbef0a1)

Change-Id: I665b5c35291b059cf054622da0eec4db44ec5f68
BUG: 1365265
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/15113
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

georep: add reset-sync-time option for session delete

2016-08-03T09:32:27+00:00

Set the stime xattr at all the brick roots to (0,0) if the argument
reset-sync-time has been provided on the command-line.
To avoid testing against directory specific stime, the remote
stime is assumed to be minus_infinity, if the root directory
stime is set to (0,0), before the directory scan begins.
This triggers a full volume resync to slave in the case of a
geo-rep session recreation with the same master-slave volume
pair.

Command synopsis:
gluster volume geo-replication  :: delete \
    [reset-sync-time]

Update gluster cli man page to include new sub-command reset-sync-time.

> Reviewed-on: http://review.gluster.org/14051
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Aravinda VK 
> Reviewed-by: Kotresh HR 

(cherry picked from commit 70fd68d94f768c098b3178c151fa92c5079a8cfd)

Change-Id: Ie4ce03b9425ed9bb81eda8681058c0fc6f990948
BUG: 1357773
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/14953
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Niels de Vos

snapshot/snapd: Don't display pid when snapd is offline

2016-08-02T06:19:02+00:00

    Backport of http://review.gluster.org/#/c/14981/

We were previously reading the pidfile, and displaying
the pid even if snapd daemon is not running. Now to fix
it, we re-assign pid value to -1, if snapd is offline.

> Reviewed-on: http://review.gluster.org/14981
> Tested-by: Vijay Bellur 
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit ec6925a379c7bee071df1638bc2751b266cee346)

Change-Id: I4baff8d489fe9380061c52aea006db90fa421cd7
BUG: 1360985
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15033
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-25T06:40:30+00:00

Backport of: http://review.gluster.org/13658

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359364
Signed-off-by: Kotresh HR 
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14994
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

feature/bitrot: Show whether scrub is in progress/idle

2016-07-18T10:53:50+00:00

Backport of http://review.gluster.org/14864/

Bitrot scrub status shows whether the scrub is paused
or active. It doesn't show whether the scrubber is
actually scrubbing or waiting in the timer wheel
for the next schedule. This patch shows this status
with "In Progress" and "Idle" respectively.

Change-Id: I995d8553d1ff166503ae1e7b46282fc3ba961f0b
BUG: 1355639
Signed-off-by: Kotresh HR 
(cherry picked from commit f4757d256e3e00132ef204c01ed61f78f705ad6b)
Reviewed-on: http://review.gluster.org/14901
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

glusterd: Fix gsyncd upgrade issue

2016-07-15T16:47:44+00:00

Backport of http://review.gluster.org/14898/

Problem:
    gluster upgrade is not generating new volfiles

Cause:
During upgrade, "glusterd --xlator-option *.upgrade=on -N"
is run to generate new volfiles. It is run post 'glusterfs'
rpm installation. The above command fails during upgrade
if geo-replication is installed. This is because on
glusterd start 'gsyncd' binary is called to configure
geo-replication related stuff. Since 'glusterfs' rpm is
installed prior to 'geo-rep' rpm, the 'gsyncd' binary
used to glusterd upgrade command is of old version and
hence it fails before generating new volfiles.

Solution:
Don't call geo-replication configure during upgrade/downgrade.
Geo-replication configuration happens during start of glusterd
after upgrade.

Change-Id: Id58ea44ead9f69982f86fb68dc5b9ee3f6cd11a1
BUG: 1356439
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14898
(cherry picked from commit 1b998788ece8c8b52657e8b9aae65d3279690c5b)
Reviewed-on: http://review.gluster.org/14916
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

Glusterd: printing the node details on error message of rebalance

2016-07-15T13:44:46+00:00

        back-port of : http://review.gluster.org/#/c/14495
        back-port of : http://review.gluster.org/#/c/14540

Problem: on the rebalance start with one of the glusterd being
down among the volume, the error message says only about the
brick path.

Fix: adding the node details

>Change-Id: I5827d3a9a15b0461c9ce3a51c0b16246ca58f335
>BUG: 1337899
>Signed-off-by: hari 

>Change-Id: I3075f3a73e289dfe577742a3d5086531026f567d
>BUG: 1339923
>Signed-off-by: hari gowtham 

Change-Id: Ic3988ff3042808d623642dc5a2ef82ef55ac8424
BUG: 1339928
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/14541
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Tested-by: hari gowtham 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System

glusterd: glusterd must store all rebalance related information

2016-07-15T09:02:03+00:00

Backport of http://review.gluster.org/14827

> Change-Id: I8404b864a405411e3af2fbee46ca20330e656045
> BUG: 1351021
> Signed-off-by: Sakshi Bansal 
> Reviewed-on: http://review.gluster.org/14827
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 

Change-Id: I8404b864a405411e3af2fbee46ca20330e656045
BUG: 1352771
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/14856
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

glusterd: Don't start bricks if server quorum is not met

2016-07-12T09:06:00+00:00

Upon glusterd restart if it is observered that the server quorum
isn't met anymore due to changes to the "server-quorum-ratio"
global option, the bricks should be stopped if they are running.
Also if glusterd has been restarted, and if server quorum is not
applicable for a volume, do not restart the bricks corresponding
to the volume to make sure that bricks that have been brought
down purposely, say for maintenance, are not brought up. This
commit moves this check that was previously inside
"glusterd_spawn_daemons" to "glusterd_restart_bricks" instead.

> Change-Id: I0a44a2e7cad0739ed7d56d2d67ab58058716de6b
> BUG: 1345727
> Signed-off-by: Samikshan Bairagya 
> Reviewed-on: http://review.gluster.org/14758
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit 807b9a135d697f175fc9933f1d23fb67b0cc6c7d)

Change-Id: I0a44a2e7cad0739ed7d56d2d67ab58058716de6b
BUG: 1353814
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/14876
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee