glusterfs.git/xlators/mgmt, branch v3.8rc2

cli/glusterd: add/remove brick fixes for arbiter volumes

2016-05-24T08:35:57+00:00

Backport of: http://review.gluster.org/14126

1.Provide a command to convert replica 2 volumes to arbiter volumes.
Existing self-heal logic will automatically heal the file hierarchy into
the arbiter brick, the progress of which can be monitored using the
heal info command.

Syntax: gluster volume add-brick  replica 3 arbiter 1


2. Add checks when removing bricks from arbiter volumes:
- When converting from arbiter to replica 2 volume, allow only arbiter
  brick to be removed.
- When converting from arbiter to plain distribute volume, allow only if
  arbiter is one of the bricks that is removed.

3. Some clean-up:
- Use GD_MSG_DICT_GET_SUCCESS instead of GD_MSG_DICT_GET_FAILED to
log messages that are not failures.
- Remove unused variable `brick_list`
- Move 'brickinfo->group' related functions to glusted-utils.

Change-Id: Ifa75d137c67ffddde7dcb8e0df0873163e713119
BUG: 1337387
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14502
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd/geo-rep: slave volume uuid to identify a geo-rep session

2016-05-23T08:13:58+00:00

Problem:
Currently, it is possible to create multiple geo-rep session from
the Master host to Slave host(s), where Slave host(s) belonging
to the same volume.

For example:
Consider Master Host M1 having volume tv1 and Slave volume tv2,
which spans across two Slave hosts S1 and S2.
Currently, it is possible to create geo-rep session from
M1(tv1) to S1(tv2) as well as from M1(tv1) to S2(tv2).

When the Slave Host is alone modified, it is identified as a new geo-rep
session (as slave host and slave volume together are identifying
Slave side).

Also, it is possible to create both root and non-root geo-rep session between
same Master volume and Slave volume. This should also be avoided.

Solution:
This multiple geo-rep session creation must be avoided and
in order to avoid, use Slave volume uuid to identify a Slave.
This way, we can identify whether a session is already created for
the same Slave volume and avoid creating again (using different host).

When the session creation is forced in the above scenario, rename
the existing geo-rep session directory with new Slave Host mentioned.

Change-Id: I9239759cbc0d15dad63c48b8cf62950bb687c7c8
BUG: 1336704
Signed-off-by: Saravanakumar Arumugam 
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/13111
Reviewed-by: Kotresh HR 
Tested-by: Kotresh HR 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
(cherry picked from commit a9128cda34b1f696b717ba09fa0ac5a929be8969)
Reviewed-on: http://review.gluster.org/14372

glusterd: copy real_path from older brickinfo during brick import

2016-05-20T09:33:10+00:00

Backport of http://review.gluster.org/14306

In glusterd_import_new_brick () new_brickinfo->real_path will not be populated
for the first time and hence if the underlying file system is bad for the same
brick, import will fail resulting in inconsistent configuration data.

Fix is to populate real_path from old brickinfo object.

Also there were many cases where we were unnecessarily calling realpath() and
that may cause in failure. For eg - if a remove brick is executed with a brick
whoose underlying file system has crashed, remove-brick fails since realpath()
call fails. We'd need to call realpath() here as the value is of no use.Hence
passing construct_realpath as _gf_false in glusterd_volume_brickinfo_get_by_brick ()
is a must in such cases.

Change-Id: I7ec93871dc9e616f5d565ad5e540b2f1cacaf9dc
BUG: 1337114
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/14306
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaushal M 
Reviewed-on: http://review.gluster.org/14411

glusterd/bitrot: Fix bit-rot scrub status

2016-05-20T06:21:02+00:00

Few of the dictionary values were not allocated and
set. This patch allocates them and set.

Backport of:
>Change-Id: Ic61c1209b238e45219793fd8bec92235adfbd013
>BUG: 1329211
>Signed-off-by: Kotresh HR 
>Reviewed-on: http://review.gluster.org/14120
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Venky Shankar 
>Reviewed-by: Atin Mukherjee 
>Signed-off-by: Kotresh HR 

BUG: 1332839
Change-Id: If54f0180aaef8ea9ea1544da925ae065becfe53e
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14202
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Venky Shankar

Revert glusterd/afr: store afr pending xattrs as a volume option

2016-05-19T04:25:32+00:00

This patch reverts changes introduced by commit
6e635284a4411b816d4d860a28262c9e6dc4bd6a in the release-3.8 branch. The
commit itself was inherited from http://review.gluster.org/#/c/12738/ in
master when the branch was created.

Reverting it for the same reason it was reverted in 3.7 branch as well:
It breaks the rolling upgrade scenario and these changes will be
required only when server side AFR materializes, possibly for gluster
4.0.

Change-Id: Ib3bb78994d7375f7c34df9897dfaf653ea909924
BUG: 1337130
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/14414
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd-ganesha : copy ganesha export configuration files during reboot

2016-05-18T13:55:39+00:00

glusterd creates export conf file for ganesha using hook script during
volume start and ganesha_manage_export() for volume set command. But this
routine is not added in glusterd restart scenario.
Consider the following case, in a three node cluster a volume got exported
via ganesha while one of the node is offline(glusterd is not running).
When the node comes back online, that volume is not exported on that node
due to the above mentioned issue.
Also I have removed unused variables from glusterd_handle_ganesha_op()
For this patch to work pcs cluster should running on that be node.

Upstream reference
>Change-Id: I5b2312c2f3cef962b1f795b9f16c8f0a27f08ee5
>BUG: 1330097
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/14063
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: soumya k 
>Reviewed-by: Atin Mukherjee 
>(cherry picked from commit f71e2fa49af185779b9f43e146effd122d4e9da0)

Change-Id: I5b2312c2f3cef962b1f795b9f16c8f0a27f08ee5
BUG: 1336801
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/14397
Smoke: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Kaleb KEITHLEY 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: assorted typos and spelling mistakes reported by Debian lintian

2016-05-18T13:55:02+00:00

Also missing bang (!) in #!/bin/bash in shell scripts.

Backport of
> http://review.gluster.org/#/c/14398/
> BUG: 1336793
> Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b

Change-Id: I003554daede9937a41010e085c74e77957c5c608
BUG: 1336794
Reported-by: Patrick Matthäi 
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14400
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

glusterd: add defence mechanism to avoid brick port clashes

2016-05-10T07:11:33+00:00

Intro:
Currently glusterd maintain the portmap registry which contains ports that
are free to use between 49152 - 65535, this registry is initialized
once, and updated accordingly as an then when glusterd sees they are been
used.

Glusterd first checks for a port within the portmap registry and gets a FREE
port marked in it, then checks if that port is currently free using a connect()
function then passes it to brick process which have to bind on it.

Problem:
We see that there is a time gap between glusterd checking the port with
connect() and brick process actually binding on it. In this time gap it could
be so possible that any process would have occupied this port because of which
brick will fail to bind and exit.

Case 1:
To avoid the gluster client process occupying the port supplied by glusterd :

we have separated the client port map range with brick port map range more @
http://review.gluster.org/#/c/13998/

Case 2: (Handled by this patch)
To avoid the other foreign process occupying the port supplied by glusterd :

To handle above situation this patch implements a mechanism to return EADDRINUSE
error code to glusterd, upon which a new port is allocated and try to restart
the brick process with the newly allocated port.

Note: Incase of glusterd restarts i.e. runner_run_nowait() there is no way to
handle Case 2, becuase runner_run_nowait() will not wait to get the return/exit
code of the executed command (brick process). Hence as of now in such case,
we cannot know with what error the brick has failed to connect.

This patch also fix the runner_end() to perform some cleanup w.r.t
return values.

Backport of:
> Change-Id: Iec52e7f5d87ce938d173f8ef16aa77fd573f2c5e
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/14043
> Tested-by: Prasanna Kumar Kalever 
> Reviewed-by: Atin Mukherjee 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: Id7d8351a0082b44310177e714edc0571ad0f7195
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14235
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

rpc: define client port range

2016-05-10T07:11:20+00:00

Problem:
when bind-insecure is 'off', all the clients bind to secure ports,
if incase all the secure ports exhaust the client will no more bind
to secure ports and tries gets a random port which is obviously insecure.

we have seen the client obtaining a port number in the range 49152-65535
which are actually reserved as part of glusterd's pmap_registry for bricks,
hence this will lead to port clashes between client and brick processes.

Solution:
If we can define different port ranges for clients incase where secure ports
exhaust, we can avoid the maximum port clashes with in gluster processes.

Still we are prone to have clashes with other non-gluster processes, but
the chances being very low, but that's a different story on its own,
which will be handled in upcoming patches.

Backportof:
> Change-Id: Ib5ce05991aa1290ccb17f6f04ffd65caf411feaf
> BUG: 1322805
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/13998
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Raghavendra G 

Change-Id: I2ab9608ddbefcdf5987d817c23dd066010148e19
BUG: 1333711
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14234
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

glusterd/bitrot: Fix bitrot xlator_type

2016-05-08T18:43:18+00:00

Fix "voltype" value for bitrot in 'glusterd_volopt_map'.
Routines such as 'gd_is_boolean_option' will use
the value of "voltype" to load the shared
library and figure out data type of specific options.
If the value of "voltype" does not match with
the actual shared library name, it will break such
routines.

>Change-Id: I36af4bb3e7043cc87dd19aea1985f3167ff82c18
>BUG: 1332465
>Signed-off-by: Kotresh HR 
>Reviewed-on: http://review.gluster.org/14183
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Atin Mukherjee 
>Reviewed-by: Kaushal M 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
(cherry picked from commit 29254449b19697b93ebf376541ea5062b4a99fcb)

Change-Id: Iae7b00272d2d5231ce46944064badacb1f09bc3b
BUG: 1332864
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14207
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Niels de Vos