glusterfs.git/xlators/mgmt/glusterd, branch v3.11dev

glusterd: bypass add-brick validation with force

2017-01-19T03:50:33+00:00

Commit c916a2f added a validation to restrict add-brick operation if a
replica configuration is changed and any of the bricks belonging to the
volume is down. However we should bypass this validation with a force
option if users really want to have add-brick to go through at the sake
of the corner cases of data loss issue.

The original problem of add-brick getting failed when layout is not set
will still be a problem with a force option as the issue has to be taken
care in the DHT layer.

Change-Id: I0ed3df91ea712f77674eb8afc6fdfa577f25a7bb
BUG: 1406411
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/16358
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System

geo-rep: Separate slave mount logs for each connection

2017-01-18T08:47:01+00:00

Geo-rep worker mounts the slave volume on the slave
node. If multiple worker connects to same slave node,
all workers share the same mount log file. This
is very difficult to debug as logs are cluttered from
different mounts. Hence creating separate mount log
file for each connection from worker. Each connection
from worker is identified uniquely using 'mastervol uuid',
'master host', 'master brickpath', 'salve vol'. The log
file name will be combination of the above.

Change-Id: I67871dc8e8ea5864e2ad55e2a82063be0138bf0c
BUG: 1412689
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16384
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

glusterd: Change the volfile to have readdir-ahead as a child

2017-01-18T04:05:11+00:00

          of dht

As mentioned in feature page http://review.gluster.org/#/c/16090/
readdir-ahead will be optionally placed below dht.

There are two options:
1. performance.readdir-ahead
2. performance.parallel-readdir

If only option is enabled, then readdir ahead is placed at its
original place as an ancestor of dht. If both the options 1 and 2
are enabled then readdir ahead is placed as a child of dht.

Also changes have been made to retain the rebalance, quotad,
snapd vol files to remain unchanged.

Change-Id: I0adf0b476fcbf91251f5a2fee2241786a3d8255a
BUG: 1401812
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16072
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

tier : Tier as a service

2017-01-17T04:49:47+00:00

tierd is implemented by separating from rebalance process.

The commands affected:

1) Attach tier will trigger this process instead of old one
2) tier start and tier start force will also trigger this process.
3) volume status [tier] will show tier daemon as a process instead
of task and normal tier status and tier detach status works.
4) tier stop implemented.
5) detach tier implemented separately along with new detach tier
status
6) volume tier volname status will work using the changes.
7) volume set works

This patch has separated the tier translator from the legacy
DHT rebalance code. It now sends the RPCs from the CLI
to glusterd separate to the DHT rebalance code.
The daemon is now a service, similar to the snapshot daemon,
and can be viewed using the volume status command.

The code for the validation and commit phase are the same
as the earlier tier validation code in DHT rebalance.

The “brickop” phase has been changed so that the status
command can use this framework.

The service management framework is now used.
DHT rebalance does not use this framework.

This service framework takes care of :

*) spawning the daemon, killing it and other such processes.
*) volume set options , which are written on the volfile.
*) restart and reconfigure functions. Restart is to restart
the daemon at two points
        1)after gluster goes down and comes up.
        2) to stop detach tier.
*) reconfigure is used to make immediate volfile changes.
By doing this, we don’t restart the daemon.
it has the code to rewrite the volfile for topological
changes too (which comes into place during add and remove brick).

With this patch the log, pid, and volfile are separated
and put into respective directories.

Change-Id: I3681d0d66894714b55aa02ca2a30ac000362a399
BUG: 1313838
Signed-off-by: hari gowtham 
Reviewed-on: http://review.gluster.org/13365
Smoke: Gluster Build System 
Tested-by: hari gowtham 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Atin Mukherjee

glusterd: remove extra space in error message

2017-01-16T16:43:28+00:00

BUG: 1402237

Change-Id: Ib6efca655555a92a0542ef6056f3357f390eeb38
Signed-off-by: Michael Adam 
Reviewed-on: http://review.gluster.org/16048
Tested-by: Atin Mukherjee 
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

glusterd: Get maximum supported op-version in a cluster

2017-01-09T05:16:12+00:00

gluster volume get  cluster.opversion gives us the current
op-version on which the cluster is operating. There is no command
that lets the user know the maximum supported op-version that the
cluster can run on.

This patch adds a new global option cluster.max-op-version, that
can be used to retrieve the maximum supported op-version in a
cluster.

Usage:
	# gluster volume get all cluster.max-op-version

Example output:

Option                                  Value
------                                  -----
cluster.max-op-version                  30900

NOTE: The only way to test this feature for now is to set the
GD_OP_VERSION_MAX macro to different values (30800 for 3.8,30900
for 3.9, and so on) and rebuild glusterd. Since the regression test
framework currently doesn't have support to simulate these tests,
there are no accompanying regression tests for this feature. It
should be possible to add tests once glusto comes in and makes it
easier to run a heterogeneous cluster.

Change-Id: I547480ee5e7912664784643e436feb198b6d16d0
BUG: 1365822
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/16283
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd: Fail add-brick on replica count change, if brick is down

2017-01-06T09:19:38+00:00

Problem:
1. Have a replica 2 volume with bricks b1 and b2
2. Before setting the layout, b1 goes down
3. Set the layout write some data, which gets populated on b2
4. b2 goes down then b1 comes up
5. Add another brick b3, and heal will take place from b1 to b3, which
   basically have no data
6. Write some data. Both b1 and b3 will mark b2 for pending writes
7. b1 goes down, and b2 comes up
8. b2 gets heald from b1. During heal it removes the data which is already
   in b2, considering that as stale data. This leads to data loss.

Solution:
1. In glusterd stage-op, while adding bricks, check whether the replica
   count is being increased
2. If yes, then check whether any of the bricks are down at that time
3. If yes, then fail the add-brick to avoid such data loss
4. Else continue the normal operation.

This check will work enen when we convert plain distribute volume to replicate

Test:
1. Create a replica 2 volume
2. Kill one brick from the volume
3. Try adding a brick to the volume
4. It should fail with all bricks are not up error
5. Cretae a distribute volume and kill one of the brick
6. Try to convert it to replicate volume, by adding bricks.
7. This should also fail.

Change-Id: I9c8d2ab104263e4206814c94c19212ab914ed07c
BUG: 1406411
Signed-off-by: karthik-us 
Reviewed-on: http://review.gluster.org/16330
Tested-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Atin Mukherjee

glusterd, cli: Get global options through volume get functionality

2016-12-31T04:47:27+00:00

Currently it is not possible to retrieve values of global options
by using the 'gluster volume get' functionality if there are no
volumes present. In order to get the global options one has to use
'gluster volume get' with a specific volume name. This usage makes
the illusion as though the option is set only on one volume, which
is incorrect. When setting the global options, 'gluster volume set'
provides a way to set them using the volume name as 'all'.

Similarly, retrieving the global options should be made possible by
using the volume name 'all' with the 'gluster volume get'
functionality. This patch adds that functionality to 'volume get'

Usage:
	# gluster volume get all

performance/readdir-ahead: mark two options as NO_DOC

2016-12-27T16:19:32+00:00

The two options are rda-high-wmark and rda-low-wmark. The impact of
these two options is yet to be fully understood and hence not
advertising these options to not run into surprises.

Change-Id: Ia537f4cc342011f0f2f3849ad6b938e247e5622d
BUG: 1356960
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/16297
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd/geo-rep: Fix geo-rep config issue

2016-12-26T12:59:44+00:00

Problem:
Geo-replication config commands fail when geo-rep
status is in 'Created'.

Cause:
During staging phase of geo-rep config, it checks
for the existence of 'monitor.pid' file. But the
pid file gets created only on start of geo-rep.
Hence it fails.

Fix:
Do not check for the existence of pid-file during
staging for config commands. It is not required.

Change-Id: I626d368b249cf0423c7f49b4284465508371f566
BUG: 1404678
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16132
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Atin Mukherjee