glusterfs.git/xlators/mgmt, branch v3.5beta1

glusterd: Save/restore/sync rebalance dict

2014-01-03T11:11:04+00:00

A dictionary was added to store additional information of a rebalance
process, like the bricks being removed in case of a rebalance started
by remove-brick. This dictionary wasn't being stored/restored or synced
during volume sync, leading to errors like a volume status command
failing. These issues have been fixed in this patch. The rebalance dict
is now stored/restored and also exported/imported during volume sync.

Also, this makes sure that the rebalance dict is only create on
remove-brick start. This adds a bricks decommissioned status to the
information imported/exported during volume sync.

BUG: 1040809
Change-Id: I46cee3e4e34a1f3266a20aac43368854594f01bc
Signed-off-by: Kaushal M 
Backport-of: http://review.gluster.org/6492
Reviewed-on: http://review.gluster.org/6524
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: rebalance to ref volinfo before starting

2013-12-23T15:00:37+00:00

        Backport of http://review.gluster.org/6522

Change-Id: Ib316897dcbd0748bfb3bfcda186b9fe30c07f80f
BUG: 1038051
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6570
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: make volinfo a refcnt'ed object.

2013-12-23T15:00:24+00:00

        Backport of http://review.gluster.org/6521

Add glusterd_volinfo_remove(..) which removes @volinfo from the list
of volumes in the cluster and performs an unref on @volinfo

Change-Id: I5f546ca58f61bc334ab1bab4c51c4a21e1f66161
BUG: 1038051
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6569
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: ignore failure to stop a stopped service.

2013-12-23T14:59:09+00:00

        Backport of http://review.gluster.org/6525

kill(2) returns -1 with errno set to ESRCH when the pid of the process
being killed doesn't exist. Failing glusterd_brick_stop on a stopped
brick could result in volume-stop failing, in commit phase.
This fix prevents that from happening.

Change-Id: I00f46fa06e489a671efbb8e4119f545f8ccea329
BUG: 1038051
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6568
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Fix stopping of stale rebalance processes

2013-12-23T14:58:28+00:00

        Backport of http://review.gluster.org/6531

Trying to stop rebalance process via RPC using the GD_SYNCOP macro,
could lead to glusterd crashing. In case of an implicit volume update,
which happens when a peer comes back up, the stop function would be
called in the epoll thread. This would lead to glusterd crashing as the
epoll thread doesn't have synctasks for the GD_SYNCOP macro to make use
of.

Instead of using the RPC method, we now terminate the rebalance process
by kill(). The rebalance process has been designed to be resistant to
interruption, so this will not lead to any data corruption.

Also, when checking for stale rebalance task, make sure that the old
task-id is not null.

Change-Id: I54dd93803954ee55316cc58b5877f38d1ebc40b9
BUG: 1044327
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6567
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

rpc,glusterd: Use rpc_clnt notifyfn to cleanup mydata

2013-12-23T14:58:18+00:00

        Backport of http://review.gluster.org/5512

rpc:
- On a RPC_TRANSPORT_CLEANUP event, rpc_clnt_notify calls the registered
  notifyfn with a RPC_CLNT_DESTROY event. The notifyfn should properly
  cleanup the saved mydata on this event.
- Break the reconnect chain when an rpc client is disabled. This will
  prevent new disconnect events which can lead to crashes.

glusterd:
- Added support for RPC_CLNT_DESTROY in glusterd_brick_rpc_notify
- Use a common glusterd_rpc_clnt_unref() function throught glusterd in
  place of rpc_clnt_unref(). This function correctly gives up the
  big-lock before performing the unref.

Change-Id: I93230441c5089039643fc9f5632477ef1b695348
BUG: 962619
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6566
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Save/restore/sync rebalance dict

2013-12-23T14:58:03+00:00

        Backport of http://review.gluster.org/6492

A dictionary was added to store additional information of a rebalance
process, like the bricks being removed in case of a rebalance started
by remove-brick. This dictionary wasn't being stored/restored or synced
during volume sync, leading to errors like a volume status command
failing. These issues have been fixed in this patch. The rebalance dict
is now stored/restored and also exported/imported during volume sync.

Also, this makes sure that the rebalance dict is only create on
remove-brick start. This adds a bricks decommissioned status to the
information imported/exported during volume sync.

Change-Id: I56fed23dc2de80a96648055fe705e9c3ffd55227
BUG: 1040809
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6565
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Improve rebalance handling during volume sync

2013-12-23T14:57:52+00:00

        Backport of http://review.gluster.org/6334

Glusterd will now correctly copy existing rebalance information when a
volinfo is updated during volume sync. If the existing rebalance
information was stale, then any existing rebalance process will be
termimnated. A new rebalance process will be started only if there is no
existing rebalance process. The rebalance process will not be started if
the existing rebalance session had completed, failed or been stopped.

Change-Id: I68c5984267c188734da76770ba557662d4ea3ee0
BUG: 1036464
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6564
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Aggregate tasks status in 'volume status [tasks]'

2013-12-23T14:56:34+00:00

        Backport of http://review.gluster.org/6230
Previously, glusterd used to just send back the local status of a task
in a 'volume status [tasks]' command. As the rebalance operation is
distributed and asynchronus, this meant that different peers could give
different status values for a rebalance or remove-brick task.

With this patch, all the peers will send back the tasks status as a part
of the 'volume status' commit op, and the origin peer will aggregate
these to arrive at a final status for the task.

The aggregation is only done for rebalance or remove-brick tasks. The
replace-brick task will have the same status on all the peers (see
comment in glusterd_volume_status_aggregate_tasks_status() for more
information) and need not be aggregated.

The rebalance process has 5 states,
 NOT_STARTED - rebalance process has not been started on this node
 STARTED - rebalance process has been started and is still running
 STOPPED - rebalance process was stopped by a 'rebalance/remove-brick
           stop' command
 COMPLETED - rebalance process completed successfully
 FAILED - rebalance process failed to complete successfully
The aggregation is done using the following precedence,
 STARTED > FAILED > STOPPED > COMPLETED > NOT_STARTED

The new changes make the 'volume status tasks' command a distributed
command as we need to get the task status from all peers.

The following tests were performed,
- Start a remove-brick task and do a status command on a peer which
  doesn't have the brick being removed. The remove-brick status was
  given correctly as 'in progress' and 'completed', instead of 'not
  started'
- Start a rebalance task, run the status command. The status moved to
  'completed' only after rebalance completed on all nodes.

Also, change the CLI xml output code for rebalance status to use the
same algorithm for status aggregation.

Change-Id: Ifd4aff705aa51609a612d5a9194acc73e10a82c0
BUG: 1027094
Signed-off-by: Krishnan Parthasarathi 
 http://review.gluster.org/6230
Reviewed-on: http://review.gluster.org/6562
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: create rpc obj for rebalance only if absent

2013-12-06T00:44:06+00:00

Change-Id: Iff305023577ff92a8f43f24dafcf201f86805769
BUG: 1038051
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6424
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati