glusterfs.git/xlators/mgmt/glusterd/src/glusterd-op-sm.c, branch v3.5.4beta1

glusterd: Aggregate tasks status in 'volume status [tasks]'

2013-12-23T14:56:34+00:00

        Backport of http://review.gluster.org/6230
Previously, glusterd used to just send back the local status of a task
in a 'volume status [tasks]' command. As the rebalance operation is
distributed and asynchronus, this meant that different peers could give
different status values for a rebalance or remove-brick task.

With this patch, all the peers will send back the tasks status as a part
of the 'volume status' commit op, and the origin peer will aggregate
these to arrive at a final status for the task.

The aggregation is only done for rebalance or remove-brick tasks. The
replace-brick task will have the same status on all the peers (see
comment in glusterd_volume_status_aggregate_tasks_status() for more
information) and need not be aggregated.

The rebalance process has 5 states,
 NOT_STARTED - rebalance process has not been started on this node
 STARTED - rebalance process has been started and is still running
 STOPPED - rebalance process was stopped by a 'rebalance/remove-brick
           stop' command
 COMPLETED - rebalance process completed successfully
 FAILED - rebalance process failed to complete successfully
The aggregation is done using the following precedence,
 STARTED > FAILED > STOPPED > COMPLETED > NOT_STARTED

The new changes make the 'volume status tasks' command a distributed
command as we need to get the task status from all peers.

The following tests were performed,
- Start a remove-brick task and do a status command on a peer which
  doesn't have the brick being removed. The remove-brick status was
  given correctly as 'in progress' and 'completed', instead of 'not
  started'
- Start a rebalance task, run the status command. The status moved to
  'completed' only after rebalance completed on all nodes.

Also, change the CLI xml output code for rebalance status to use the
same algorithm for status aggregation.

Change-Id: Ifd4aff705aa51609a612d5a9194acc73e10a82c0
BUG: 1027094
Signed-off-by: Krishnan Parthasarathi 
 http://review.gluster.org/6230
Reviewed-on: http://review.gluster.org/6562
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cli: use proper copy to set node-name

2013-11-26T19:55:46+00:00

Previously node-name is set to point to node-uuid which could cause
memory leak.  This is fixed by having memory copy of node-uuid.

BUG: 1012296
Change-Id: I3b638ec289d5b167c6e752ef1ba41f41efacb9da
Signed-off-by: Bala.FA 
Reviewed-on: http://review.gluster.org/6330
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cli/glusterd: Changes to quota command Quota feature

2013-11-26T18:25:27+00:00

 re-work.

Following are the cli commands that are new/re-worked:
======================================================

volume quota  {enable|disable|list [ ...]|remove | default-soft-limit } |
volume quota  {limit-usage   []} |
volume quota  {alert-time|soft-timeout|hard-timeout} {}
volume status [all |  [nfs|shd||quotad]] [detail|clients|mem|inode|fd|callpool]
volume statedump  [nfs|quotad] [all|mem|iobuf|callpool|priv|fd|inode|history]

glusterd changes:
=================
* Quota limits are now set as extended attributes by glusterd from
  the aux mount created by the cli.
* The gfids of the directories on which quota limits are set
  for a given volume are stored in
  /var/lib/glusterd/vols//quota.conf file in binary format,
  and whose cksum and version is stored in
  /var/lib/glusterd/vols//quota.cksum.

Original-author: Krutika Dhananjay 
Original-author: Krishnan Parthasarathi 

BUG: 969461
Change-Id: If32bba36c67f9c2a30417af9c6389045b2b7c13b
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6003
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cli: fix possible memory leaks

2013-11-21T21:08:16+00:00

BUG: 955548
Change-Id: Iae410712e7e6d7a76cd537c77f1919e3b4cdf6bb
Signed-off-by: Bala.FA 
Reviewed-on: http://review.gluster.org/6328
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System

cli: add peerid to volume status xml output

2013-11-15T07:31:32+00:00

This patch adds  tag to bricks and nfs/shd like services to
volume status xml output.

BUG: 955548
Change-Id: I9aaa9266e4d56f632235eaeef565e92d757c0694
Signed-off-by: Bala.FA 
Reviewed-on: http://review.gluster.org/6162
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

bd_map: Remove bd_map xlator

2013-11-13T19:38:28+00:00

Remove bd_map xlator and CLI related changes.

Change-Id: If7086205df1907127c1a1fa4ba603f1c48421d09
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5747
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd : Improved quota volume reset command

2013-10-28T18:39:21+00:00

Quota volume reset command without "force"
option fixed, doesn't fail anymore. It resets
unprotected fields and not the protected ones.

Also, an appropriate message is provided to the user
for the following cases :
1. only unprotected fields are reset, "force" option
should be used to reset protected fields.
2. Both protected and unprotected fields are reset.
3. No field was reset, "force" option required.

Test case for the same also added.

Change-Id: I24e8f1be87b79ccd81bf6f933e00608b861c7a16
BUG: 1022905
Signed-off-by: Anuradha 
Reviewed-on: http://review.gluster.org/6135
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Vijay Bellur

libglusterfs: Add monotonic clocking counter for timer thread

2013-10-15T07:14:57+00:00

gettimeofday() returns the current wall clock time and timezone.
Using these functions in order to measure the passage of time
(how long an operation took) therefore seems like a no-brainer.

This time suffer's from some limitations:

a. They have a low resolution: “High-performance” timing by
definition, requires clock resolutions into the microseconds
or better.

b. They can jump forwards and backwards in time: Computer
clocks all tick at slightly different rates, which causes
the time to drift. Most systems have NTP enabled which
periodically adjusts the system clock to keep them in sync
with “actual” time. The adjustment can cause the clock to
suddenly jump forward (artificially inflating your timing
numbers) or jump backwards (causing your timing calculations
to go negative or hugely positive). In such cases timer
thread could go into an infinite loop.

From 'man gettimeofday':
----------
..
..
The time returned by gettimeofday() is affected by discontinuous
jumps in the system time (e.g., if the system administrator manually
changes the system time).  If you need a monotonically increasing
clock, see clock_gettime(2).
..
..
----------

Rationale:

For calculating interval timing for Timer thread, all that’s
needed should be clock as a simple counter that increments
at a stable rate.

This is necessary to avoid the jumps which are caused by using
"wall time", this counter must be monotonic that can never
“tick” backwards, ever.

Change-Id: I701d31e71a85a73d21a6c5cd15583e7a5a645eeb
BUG: 1017993
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/6070
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: [Feature] Command implementation to get heal-count

2013-10-14T21:41:54+00:00

Currently to know the number of files to be healed, either user
has to go to backend and check the number of entries present in
indices/xattrop directory. But if a volume consists of large
number of bricks, going to each backend and counting the number
of entries is a time-taking task. Otherwise user can give
gluster volume heal vol-name info command but with this
approach if no. of entries are very hugh in the indices/
xattrop directory, it will comsume time.

So as a feature, new command is implemented.

Command 1: gluster volume heal vn statistics heal-count
This command will get the number of entries present in
every brick of a volume. The output displays only entries
count.

Command 2: gluster volume heal vn statistics heal-count
           replica 192.168.122.1:/home/user/brickname

           Here if we are concerned with just one replica.
So providing any one of the brick of a replica will get
the number of entries to be healed for that replica only.

Example:
Replicate volume with replica count 2.

Backend status:
--------------
[root@dhcp-0-17 xattrop]# ls -lia | wc -l
1918

NOTE: Out of 1918, 2 entries are  dummy
entries so actual no. of entries to be healed are
1916.

[root@dhcp-0-17 xattrop]# pwd
/home/user/2ty/.glusterfs/indices/xattrop

Command output:
--------------
Gathering count of entries to be healed on volume volume3 has been successful

Brick 192.168.122.1:/home/user/22iu
Status: Brick is Not connected
Entries count is not available

Brick 192.168.122.1:/home/user/2ty
Number of entries: 1916

Change-Id: I72452f3de50502dc898076ec74d434d9e77fd290
BUG: 1015990
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/6044
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr : Implementation of command "gluster volume heal vn statistics"

2013-10-14T21:41:44+00:00

"gluster volume heal volumename statistics" command gives the summary
of the afr crawl done based on the entries present in the xattrop
directory. Whenever afr crawls are attempted, the beginning time of
crawl, end time of crawl, no of files healed, heal-failed count and
number of files in split brain are shown along with the type of the
crawl. If crawl is already in progress then it will give the number
of files healed, heal failed count and number of files in split-brain
from the beginning of the crawl and instead of telling the end time of
the crawl, "CRAWL IN PROGRESS" message will be shown.

Output format:
command: "gluster volume heal volume-name statistics"
Output:
Gathering afr crawl statistics crawl statistics on volume volume-name
has been successful
------------------------------------------------

Crawl statistics for brick no 0
Hostname of brick 192.168.122.248

Starting time of crawl: Wed Jul 10 15:52:38 2013

Ending time of crawl: Wed Jul 10 15:52:38 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

Starting time of crawl: Wed Jul 10 15:52:38 2013

Ending time of crawl: Wed Jul 10 15:52:38 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

------------------------------------------------

Crawl statistics for brick no 1
Hostname of brick 192.168.122.1

Starting time of crawl: Wed Jul 10 15:52:42 2013

Ending time of crawl: Wed Jul 10 15:52:42 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

Starting time of crawl: Wed Jul 10 15:52:42 2013

Ending time of crawl: Wed Jul 10 15:52:42 2013

Type of crawl: INDEX
No. of entries healed: 0
No. of entries in split-brain: 0
No. of heal failed entries: 0

--------------------------------------------------

Change-Id: I10bf9d10b005741db9973fb1352e0dd59ed99aa9
BUG: 949400
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/4790
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati