glusterfs.git/xlators/mgmt/glusterd/src/glusterd-rebalance.c, branch v3.12.9

glusterd: disable rpc_clnt_t after relalance process disconnection

2017-08-25T19:00:50+00:00

Problem:
glusterd continues to connect to rebalance process even after
the socket connection has disconnected.

Solution:
rpc_clnt_disable() disables the rpc_clnt_t object and disarms
all relevant timers and drops refs to the rpc_clnt_t object
and the transport as well.

>Reviewed-on: https://review.gluster.org/18114
>Reviewed-by: MOHIT AGRAWAL 
>Tested-by: Atin Mukherjee 
>Reviewed-by: Atin Mukherjee 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>(cherry picked from commit a894d44427649e99d4344a241dc2f9d584a9a691)

Change-Id: I981d6f1cc0087037f1927062c2770a4d5026a619
BUG: 1484885
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/18117
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

Revert "glusterd: disallow rebalance & remove-brick on a sharded volume"

2017-06-13T08:51:39+00:00

This reverts commit 8375b3d70d5c6268c6770b42a18b2e1bc09e411e.

Now that some of the users have confirmed rebalance works fine without
causing corruption of VMs, time to revert the CLI restriction.

Change-Id: I45493fcbb1f25fd0fff27b2b3526c42642ccb464
BUG: 1460585
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/17506
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

glusterd: socketfile & pidfile related fixes for brick multiplexing feature

2017-05-09T01:30:01+00:00

Problem: While brick-muliplexing is on after restarting glusterd, CLI is
         not showing pid of all brick processes in all volumes.

Solution: While brick-mux is on all local brick process communicated through one
          UNIX socket but as per current code (glusterd_brick_start) it is trying
          to communicate with separate UNIX socket for each volume which is populated
          based on brick-name and vol-name.Because of multiplexing design only one
          UNIX socket is opened so it is throwing poller error and not able to
          fetch correct status of brick process through cli process.
          To resolve the problem write a new function glusterd_set_socket_filepath_for_mux
          that will call by glusterd_brick_start to validate about the existence of socketpath.
          To avoid the continuous EPOLLERR erros in  logs update socket_connect code.

Test:     To reproduce the issue followed below steps
          1) Create two distributed volumes(dist1 and dist2)
          2) Set cluster.brick-multiplex is on
          3) kill glusterd
          4) run command gluster v status
          After apply the patch it shows correct pid for all volumes

BUG: 1444596
Change-Id: I5d10af69dea0d0ca19511f43870f34295a54a4d2
Signed-off-by: Mohit Agrawal 
Reviewed-on: https://review.gluster.org/17101
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: disallow rebalance & remove-brick on a sharded volume

2017-05-04T20:49:39+00:00

Change-Id: Idfbdbc61ca18054fdbf7556f74e195a63cd8a554
BUG: 1447630
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/17160
Reviewed-by: Raghavendra Talur 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Amar Tumballi 
CentOS-regression: Gluster Build System

xlator: do not call dlclose() when debugging

2017-04-07T17:17:12+00:00

Valgrind can not show the symbols if a .so after calling dlclose(). The
unhelpful ??? in the output gets resolved properly with this change:

  ==25170== 344 bytes in 1 blocks are definitely lost in loss record 233 of 324
  ==25170==    at 0x4C29975: calloc (vg_replace_malloc.c:711)
  ==25170==    by 0x52C7C0B: __gf_calloc (mem-pool.c:117)
  ==25170==    by 0x12B0638A: ???
  ==25170==    by 0x528FCE6: __xlator_init (xlator.c:472)
  ==25170==    by 0x528FE16: xlator_init (xlator.c:498)
  ==25170==    by 0x52DA8D6: glusterfs_graph_init (graph.c:321)
  ==25170==    by 0x52DB587: glusterfs_graph_activate (graph.c:695)
  ==25170==    by 0x5046407: glfs_process_volfp (glfs-mgmt.c:79)
  ==25170==    by 0x5043B9E: glfs_volumes_init (glfs.c:281)
  ==25170==    by 0x5044FEC: glfs_init_common (glfs.c:986)
  ==25170==    by 0x50451A7: glfs_init@@GFAPI_3.4.0 (glfs.c:1031)

By not calling dlclose(), the dynamically loaded .so is still available
upon program exit, and Valgrind is able to resolve the symbols. This
will add an additional leak, so dlclose() is called for normal builds,
but skipped when configuring with "./configure --enable-valgrind" or
passing the "run-with-valgrind" xlator option.

URL: http://valgrind.org/docs/manual/faq.html#faq.unhelpful
Change-Id: I2044e21b1b8fcce32ad1a817fdd795218f967731
BUG: 1425623
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16809
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Samikshan Bairagya 
Reviewed-by: Kaleb KEITHLEY

glusterd : log improvements in glusterd_op_stage_rebalance ()

2017-02-24T09:43:10+00:00

Include volume names in the respective staging failure error logs in
rebalance staging

Change-Id: Iaaab12a552930dd5274fbecec78f5735f883ab6b
BUG: 1426509
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/16746
Reviewed-by: Gaurav Yadav 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: run many bricks within one glusterfsd process

2017-01-31T00:13:58+00:00

This patch adds support for multiple brick translator stacks running
in a single brick server process.  This reduces our per-brick memory usage by
approximately 3x, and our appetite for TCP ports even more.  It also creates
potential to avoid process/thread thrashing, and to improve QoS by scheduling
more carefully across the bricks, but realizing that potential will require
further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global option.  By
default it's off, and bricks are started in separate processes as before.  If
multiplexing is enabled, then *compatible* bricks (mostly those with the same
transport options) will be started in the same process.

Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
BUG: 1385758
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/14763
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

dht/rebalance Estimate time to complete rebalance

2017-01-20T04:01:47+00:00

The estimates will be logged to the rebalance log on running
gluster v rebalance  status

Change-Id: I9d51b139cd4c8dfde1ff2c2050720ae606c13fc6
BUG: 1396004
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15893
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

glusterd: Revert 368e26f454fe35477e46dc698fa6b8c3c608ea8d

2016-09-26T17:54:27+00:00

The earlier fix prevented rebalance from
starting even if only a single brick of a
replica set was down.

Change-Id: I8c9a7d4c1fe33f7749568357462f29796e1b832a
BUG: 1376671
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15517
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Atin Mukherjee