glusterfs.git/xlators/mgmt/glusterd/src/glusterd-volgen.c, branch v3.12.2

glusterd: fix client io-threads option for replicate volumes

2017-10-09T12:21:08+00:00

Backport of https://review.gluster.org/#/c/18430/

Problem:
Commit ff075a3d6f9b142911d25c27fd209838782bfff0 disabled loading
client-io-threads for replicate volumes (it was set to on by default in
commit e068c1997314046658dd502e9118dab32decf879) due to performance
issues but in doing so, inadvertently failed to load the xlator even if
the user explicitly enabled the option using the volume set command.
This was despite returning returning sucess for the volume set.

Fix:
Modify the check in perfxl_option_handler() and add checks in volume
create/add-brick/remove-brick code paths, tying it all to
GD_OP_VERSION_3_12_2.

Change-Id: Ib612973a999a7da818cc926f5c2601b1f0794fcf
BUG: 1499158
Signed-off-by: Ravishankar N

glusterd: disallow volume specific options to be set with all as volume name

2017-08-21T14:28:10+00:00

All the .validate_fn defined in volume map entry table refers to volinfo
object. And if we end up in trying to set a volume level option cluster
wide glusterd results into a crash.

>Reviewed-on: https://review.gluster.org/18052
>Smoke: Gluster Build System 
>Reviewed-by: Prashanth Pai 
>Reviewed-by: mohammed rafi  kc 
>Reviewed-by: Gaurav Yadav 
>CentOS-regression: Gluster Build System 
>(cherry picked from commit 01abf7ee37702407403afcf9aa6c9019a0316e1d)

Change-Id: I7c877aee0ff5c8c1d8c95662fdc8c8923355ae7b
BUG: 1482804
Signed-off-by: Atin Mukherjee 
Reviewed-on: https://review.gluster.org/18060
Reviewed-by: Prashanth Pai 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

logging: localtime logging, cmdline, volume set option

2017-08-03T12:04:50+00:00

Despite the fact that appliances generally use UTC, some
users really want log entries in localtime.

fixes gluster/glusterfs#272

feature page: https://review.gluster.org/17807

Backport from master https://review.gluster.org/#/c/16911/

Change-Id: I5fbf2c3eedd9eb128fb3f851dd67b2f4081c8bba
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/17928
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

posix: option to handle the shared bricks for statvfs()

2017-07-31T15:34:58+00:00

Currently 'storage/posix' xlator has an option called option
`export-statfs-size no`, which exports zero as values for few
fields in `struct statvfs`. In a case of backend brick shared
between multiple brick processes, the values of these variables
should be `field_value / number-of-bricks-at-node`. This way,
even the issue of 'min-free-disk' etc at different layers would
also be handled properly when the statfs() sys call is made.

Fixes #241

> Reviewed-on: https://review.gluster.org/17618
> Reviewed-by: Jeff Darcy 
> Reviewed-by: Atin Mukherjee 
> (cherry picked from commit febf5ed4848ad705a34413353559482417c61467)

Change-Id: I2e320e1fdcc819ab9173277ef3498201432c275f
Signed-off-by: Amar Tumballi 
Reviewed-on: https://review.gluster.org/17903
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

index: Do not proceed with init if brick is not mounted

2017-06-19T05:16:53+00:00

..or else when a volume start force is given, we end up creating
/brick-path/.glusterfs/indices folder and various subdirs under it and
eventually starting the brick process.

As a part of this patch, glusterd_get_index_basepath() is added in
glusterd, who will then use it to create the basepath during
volume-create, add-brick, replace-brick and reset-brick. It also uses this
function to set the 'index-base' xlator option for the index translator.

Change-Id: Id018cf3cb6f1e2e35b5c4cf438d1e939025cb0fc
BUG: 1457202
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/17426
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Pranith Kumar Karampuri

SELinux : implementation of SELinux translator

2017-05-03T09:34:11+00:00

The patch implement a part of SELinux translator to support setting
SELinux contexts on files in a glusterfs volume.

URL: https://github.com/gluster/glusterfs-specs/blob/master/accepted/SELinux-client-support.md

Change-Id: Id8916bd8e064ccf74ba86225ead95f86dc5a1a25
BUG: 1318100
Fixes : #55
Signed-off-by: Manikandan Selvaganesh 
Signed-off-by: Jiffin Tony Thottan 
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/13762
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Manikandan Selvaganesh 
Reviewed-by: Atin Mukherjee

Halo Replication feature for AFR translator

2017-05-02T10:23:53+00:00

Summary:
Halo Geo-replication is a feature which allows Gluster or NFS clients to write
locally to their region (as defined by a latency "halo" or threshold if you
like), and have their writes asynchronously propagate from their origin to the
rest of the cluster.  Clients can also write synchronously to the cluster
simply by specifying a halo-latency which is very large (e.g. 10seconds) which
will include all bricks.

In other words, it allows clients to decide at mount time if they desire
synchronous or asynchronous IO into a cluster and the cluster can support both
of these modes to any number of clients simultaneously.

There are a few new volume options due to this feature:
  halo-shd-latency:  The threshold below which self-heal daemons will
  consider children (bricks) connected.

  halo-nfsd-latency: The threshold below which NFS daemons will consider
  children (bricks) connected.

  halo-latency: The threshold below which all other clients will
  consider children (bricks) connected.

  halo-min-replicas: The minimum number of replicas which are to
  be enforced regardless of latency specified in the above 3 options.
  If the number of children falls below this threshold the next
  best (chosen by latency) shall be swapped in.

New FUSE mount options:
  halo-latency & halo-min-replicas: As descripted above.

This feature combined with multi-threaded SHD support (D1271745) results in
some pretty cool geo-replication possibilities.

Operational Notes:
- Global consistency is gaurenteed for synchronous clients, this is provided by
  the existing entry-locking mechanism.
- Asynchronous clients on the other hand and merely consistent to their region.
  Writes & deletes will be protected via entry-locks as usual preventing
  concurrent writes into files which are undergoing replication.  Read operations
  on the other hand should never block.
- Writes are allowed from _any_ region and propagated from the origin to all
  other regions.  The take away from this is care should be taken to ensure
  multiple writers do not write the same files resulting in a gfid split-brain
  which will require resolution via split-brain policies (majority, mtime &
  size).  Recommended method for preventing this is using the nfs-auth feature to
  define which region for each share has RW permissions, tiers not in the origin
  region should have RO perms.

TODO:
- Synchronous clients (including the SHD) should choose clients from their own
  region as preferred sources for reads.  Most of the plumbing is in place for
  this via the child_latency array.
- Better GFID split brain handling & better dent type split brain handling
  (i.e. create a trash can and move the offending files into it).
- Tagging in addition to latency as a means of defining which children you wish
  to synchronously write to

Test Plan:
- The usual suspects, clang, gcc w/ address sanitizer & valgrind
- Prove tests

Reviewers: jackl, dph, cjh, meyering

Reviewed By: meyering

Subscribers: ethanr

Differential Revision: https://phabricator.fb.com/D1272053

Tasks: 4117827

Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
BUG: 1428061
Signed-off-by: Kevin Vigor 
Reviewed-on: http://review.gluster.org/16099
Reviewed-on: https://review.gluster.org/16177
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/bit-rot-stub: bring in optional versioning

2017-04-19T03:29:46+00:00

* As of now bit-rot-stub does versioning always. This leads
  lots of getxattr calls being made in lookups. So make
  object versioning optional.

Change-Id: I83713e45ae59fb28004bb3cfa008f2d69edebbfa
BUG: 1359599
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/14442
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

dht: Add readdir-ahead in rebalance graph if parallel-readdir is on

2017-04-18T06:16:11+00:00

Issue:
The value of linkto xattr is generally the name of the dht's
next subvol, this requires that the next subvol of dht is not
changed for the life time of the volume. But with parallel
readdir enabled, the readdir-ahead loaded below dht, is optional.
The linkto xattr for first subvol, when:
- parallel readdir is enabled : "-readdir-head-0"
- plain distribute volume : "-client-0"
- distribute replicate volume : "-afr-0"

The value of linkto xattr is "-readdir-head-0" when
parallel readdir is enabled, and is "-client-0" if
its disabled. But the dht_lookup takes care of healing if it
cannot identify which linkto subvol, the xattr points to.

In dht_lookup_cbk, if linkto xattr is found to be "-client-0"
and parallel readdir is enabled, then it cannot understand the
value "-client-0" as it expects "-readdir-head-0".
In that case, dht_lookup_everywhere is issued and then the linkto file
is unlinked and recreated with the right linkto xattr. The issue is
when parallel readdir is enabled, mount point accesses the file
that is currently being migrated. Since rebalance process doesn't
have parallel-readdir feature, it expects "-client-0"
where as mount expects "-readdir-head-0". Thus at some point
either the mount or rebalance will fail.

Solution:
Enable parallel-readdir for rebalance as well and then do not
allow enabling/disabling parallel-readdir if rebalance is in
progress.

Change-Id: I241ab966bdd850e667f7768840540546f5289483
BUG: 1436090
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17056
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Raghavendra G

glusterd: Add validation for options rda-cache-limit rda-request-size

2017-04-11T07:42:40+00:00

Currently when prarallel readdir is enabled, setting any junk value
to rda-cache-limit and rda-request-size succeeds. This is because of
bug in the special handling of these options.

Fixing the same in this patch

Change-Id: I902cd9ac9134c158ab6f8aea4b001254a03547bd
BUG: 1439640
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17008
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System