glusterfs.git/xlators/mgmt/glusterd, branch release-3.8-fb

glusterd: fix missing/renamed options

2017-10-02T19:23:14+00:00

Change-Id: I2ca0298ee9d166f58b8730256ea76a04e547ce5d
Signed-off-by: Jeff Darcy

self-heal: fix automatic split-brain resolution options

2017-09-27T23:03:35+00:00

Differential Revision: https://phabricator.intern.facebook.com/D5927193

Change-Id: Ife04c8738b9ee721e7be9bc843b2f6d54bbb468e

Replace namespace/io-stats/io-threads with 3.6-fb versions

2017-09-15T20:47:01+00:00

This rolls up multiple patches related to namespace identificaton and
throttling/QoS.  This primarily includes the following, all by Michael
Goulet .

  io-threads: Add weighted round robin queueing by namespace
  https://phabricator.facebook.com/D5615269

  io-threads: Add per-namespaces queue sizes to IO_THREADS_QUEUE_SIZE_KEY
  https://phabricator.facebook.com/D5683162

  io-threads: Implement better slot allocation algorithm
  https://phabricator.facebook.com/D5683186

  io-threads: Only enable weighted queueing on bricks
  https://phabricator.facebook.com/D5700062

  io-threads: Update queue sizes on drain
  https://phabricator.facebook.com/D5704832

  Fix parsing (-1) as default NS weight
  https://phabricator.facebook.com/D5723383

Parts of the following patches have also been applied to satisfy
dependencies.

  io-throttling: Calculate moving averages and throttle offending hosts
  https://phabricator.fb.com/D2516161
  Shreyas Siravara 

  Hook up ODS logging for FUSE clients.
  https://phabricator.facebook.com/D3963376
  Kevin Vigor 

  Add the flag --skip-nfsd-start to skip the NFS daemon stating, even if
  it is enabled
  https://phabricator.facebook.com/D4575368
  Alex Lorca 

There are also some "standard" changes: dealing with code that moved,
reindenting to comply with Gluster coding standards, gf_uuid_xxx, etc.

This patch *does* revert some changes which have occurred upstream since
3.6; these will be re-applied as apppropriate on top of this new base.

Change-Id: I69024115da7a60811e5b86beae781d602bdb558d
Signed-off-by: Jeff Darcy

Add options to disable new features

2017-09-14T19:43:21+00:00

Summary: @sshreyas thought the best idea to roll out these new features in the default-off state. This diff adds a few options and modifies tests to make sure that this is done.

Test Plan: The brick restart test works fine, but now it's default disabled on all bricks.

Reviewers: sshreyas, jdarcy

Reviewed By: jdarcy

Subscribers: sshreyas, #posix_storage

Differential Revision: https://phabricator.intern.facebook.com/D5653138

Porting note: includes disconnected-reqs option; retart-bricks inapplicable

Change-Id: I332339894d3cbfafdabeb8592e95c37f30f9751a
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18291
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Get glusterfs to output p50, p90, and p95 latencies

2017-09-13T19:32:29+00:00

Summary:
  [done] separate p99 dumping into general funcs
  [done] add p95, p90, and p50 stats
    - add p95, p90, p50 within p99, and generalize
    - rename config to dump-percentile-lantencies

Test Plan:
  make install glusterfs on dev machine.
  gluster volume create $name ...
  mount volume on /mnt/$name 
  dd if=/dev/zero of=/mnt/$name/test
  check each brick for pn printing

    /var/lib/glusterd/stats/glusterfsd__$brick.dump

Reviewers: sshreyas, kvigor, jdarcy

Reviewed By: jdarcy

Differential Revision: https://phabricator.intern.facebook.com/D5645951

Change-Id: I7bcd7201fc3753316db0ece809491a1cbdbefd32
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18278
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

added p99 support to the samples logging

2017-09-13T18:31:53+00:00

Summary:
added global and by-fop-type calculation of p99 latency
to the sampled fop data

Test Plan:
build local glusterfs mount and looked at
the stats while dd if=/dev/zero of=/mnt/fuse/groot/share1/test1 bs=5

Reviewers: sshreyas, mgoulet, jdarcy

Reviewed By: jdarcy

Subscribers: jdarcy

Differential Revision: https://phabricator.intern.facebook.com/D5597662

Change-Id: I3f5cd9c0ea59ae4357827fcbd19bbf009e661c05
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18277
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Disable brick daemon from incorrect brick directory

2017-09-13T17:50:36+00:00

Summary:
Currently the bricks can open any mount directory from the given volume. This patch adds a provision to prevent
bricks from opening brick directories that aren't created for them. This will help with operating gluster on large
scale.

We add a new xattr GF_XATTR_BRICK_NAME to the brick directory. When we start a brick daemon, we make sure the path on
disk matches with the config provided. For backward compatibility, we ignore if there is no value for
GF_XATTR_BRICK_NAME and set the current brick daemon's path as value.

We ignore GF_XATTR_BRICK_NAME during healing and reset GF_XATTR_BRICK_NAME on brick replace.

Test Plan: Run fb-smoke

Reviewers: jdarcy, sshreyas

Reviewed By: sshreyas

Differential Revision: https://phabricator.intern.facebook.com/D5448921

Porting note: disabled some checks to deal with the snapshot case

Change-Id: I98e62033dfd07f30ad3b99ac003ce94c8d935e5f
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18275
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

glusterd: make peerfile parsing more robust

2017-09-13T17:38:36+00:00

Summary: This will now skip files in the peer directory that don't have names or contents that match what we expect for a valid peerfile, instead of blowing up the entire glusterd initialization as soon as the first unexpected thing happens.

Test Plan: Test (peer-parsing.t) included.

Reviewers: #posix_storage, kvigor

Reviewed By: kvigor

Subscribers: kvigor

Differential Revision: https://phabricator.intern.facebook.com/D5498639

Tags: gluster, posix_storage

Change-Id: Ifad9b047a828c2f76f97d0c39f305b7ec5a8ca4c
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18276
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Wait for a brick's local filesystem to be mounted before starting.

2017-09-13T14:46:59+00:00

Summary: By far the most common reason why a brick's directory might not exist is that the local filesystem on which it lives hasn't finished mounting yet.  This is unlike other checks we do, such as for a volume ID and GFID.  Some of these are normal conditions when a brick is firstcreated; others are often the result of operator/script error.  In the singular case of the directory being absent, wait a little while to see if it comes up.

Test Plan:
Create a volume.
Start/stop a volume once so everything gets initialized.
Move a brick directory out of place.
Try to start the volume.  This should pause.
Immediately move the brick directory back into place.   This should break the pause.

Reviewers: #posix_storage, sshreyas

Reviewed By: sshreyas

Subscribers: shreyas, sshreyas, ventullo, moox

Differential Revision: https://phabricator.intern.facebook.com/D5063515

Tags: gluster

Change-Id: Ied7b07b1a60f54856a67d4cdbad35bfce9e196e4
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18274
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

inodelk-count: Add stats to count the number of lock objects

2017-09-13T14:13:58+00:00

Summary:
We want to track the number of locks held by the locks xlator. One of the ways to do it would be to track the
total number of pl_lock objects in the system.

This patch tracks the total number of pl_lock object and exposes the stat via io-stats JSON dump.

Test Plan: WIP, haven't got a pass. Putting the diff to get a sense of this approach would yield what you guys are looking for?

Reviewers: kvigor, sshreyas, jdarcy

Reviewed By: jdarcy

Differential Revision: https://phabricator.intern.facebook.com/D5303071

Change-Id: I946debcbff61699ec28b4d6f243042440107a224
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18273
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System