glusterfs.git/xlators, branch release-3.8-fb

build: many rpm-build fixes

2017-12-20T17:40:19+00:00

Summary:
Highlights include:

 * Fixed GF_CONF_OPTS (dev builds) and RPM_BUILD_FLAGS (rpm builds)

 * Fixed version in configure.ac

 * Fixed handling of files only present when BUILD_FB_EXTRAS is set

 * Fixed disable-georeplication (upstream bug)

 * Fixed disable-tiering (upstream bug)

 * Removed .service files which should be generated from .in versions

 * Fixed tirpc (previously fbtirpc) references

 * Fixed init_enable problems

 * Removed delay-gen references

Test Plan: Use build.sh to build an RPM, and install it.

Differential Revision: https://phabricator.intern.facebook.com/D6611299

Change-Id: If61a4964a149f782038ea47362a82b813e6b7738
Signed-off-by: Jeff Darcy

build: merge our 3.6 and upstream 3.8 configure/specfile

2017-11-20T20:35:31+00:00

Summary:
They have a common ancestor at 3.6, but there were hundreds of
lines of changes for each file on each side of the fork. In both
cases the easiest method was to take the upstream 3.8 version and
re-apply our own changes since we branched. Some changes were
dropped (e.g. runit) and a few other files needed new changes
(e.g. pkg-version) to keep up. Then there was more hacking to
fix stealth geo-rep dependencies, enable tirpc/IPv6, and so on.

Also added buildrpm38 and makerelease38.  These should probably
not go upstream, but not sure what else to do with them.

Test Plan: Build RPMs. Install, create volumes, mount, do I/O.

Reviewers: sshreyas, #posix_storage

Reviewed By: sshreyas

Subscribers: jbacik, aquevedo, scientist, sshreyas, calvinowens, jweiner

Differential Revision: https://phabricator.intern.facebook.com/D6259797

Tasks: T20348589

Tags: posix-2017h2, gluster, posix_storage

Change-Id: I2d43fc6f7f5603293e406c21e4ec85bf19610b77
Signature: 6259797:1510694123:fc5d2975fec134a51d4b70f7f983cd71971e175a

glusterd: fix missing/renamed options

2017-10-02T19:23:14+00:00

Change-Id: I2ca0298ee9d166f58b8730256ea76a04e547ce5d
Signed-off-by: Jeff Darcy

self-heal: fix automatic split-brain resolution options

2017-09-27T23:03:35+00:00

Differential Revision: https://phabricator.intern.facebook.com/D5927193

Change-Id: Ife04c8738b9ee721e7be9bc843b2f6d54bbb468e

io-threads: re-port changes since 3.6 on top of FB version

2017-09-18T22:31:16+00:00

Includes io-threads parts of the following patches:

  9e3fea1 performance/io-threads: Exit all threads on PARENT_DOWN
  2cfb7bc performance/io-threads: Exit threads in fini() as well

Change-Id: Id7cc7720e75414fb8a3ac2db68a5fe63c459ffe2
Signed-off-by: Jeff Darcy

io-stats: re-port changes since 3.6 on top of FB version

2017-09-15T21:16:04+00:00

Includes io-stats parts of the following patches:

  1e421a5 logging: Avoid re-initing log level in io-stats
  0facb11 io-stats: Fix overwriting of client profile by the bricks
  91004b0 debug/io-stats: Disable fop stats dump by default
  62f9659 all: fix various cppcheck warnings
  e62c0fe build: export minimum symbols from xlators for correct resolution
  1d0a0d1 core: use syscall wrappers instead of direct syscalls - tail
  0773ca6 all: reduce "inline" usage
  8a9328e build: do not #include "config.h" in each file
  320455b io-stats: Fixing dereference after null check.
  28397ca Avoid conflict between contrib/uuid and system uuid
  49d6894 io-stats : null dereference coverity fix.

Change-Id: If1bdad6244e5749c6d8c456e6c64b5c5b483e273
Signed-off-by: Jeff Darcy

Replace namespace/io-stats/io-threads with 3.6-fb versions

2017-09-15T20:47:01+00:00

This rolls up multiple patches related to namespace identificaton and
throttling/QoS.  This primarily includes the following, all by Michael
Goulet .

  io-threads: Add weighted round robin queueing by namespace
  https://phabricator.facebook.com/D5615269

  io-threads: Add per-namespaces queue sizes to IO_THREADS_QUEUE_SIZE_KEY
  https://phabricator.facebook.com/D5683162

  io-threads: Implement better slot allocation algorithm
  https://phabricator.facebook.com/D5683186

  io-threads: Only enable weighted queueing on bricks
  https://phabricator.facebook.com/D5700062

  io-threads: Update queue sizes on drain
  https://phabricator.facebook.com/D5704832

  Fix parsing (-1) as default NS weight
  https://phabricator.facebook.com/D5723383

Parts of the following patches have also been applied to satisfy
dependencies.

  io-throttling: Calculate moving averages and throttle offending hosts
  https://phabricator.fb.com/D2516161
  Shreyas Siravara 

  Hook up ODS logging for FUSE clients.
  https://phabricator.facebook.com/D3963376
  Kevin Vigor 

  Add the flag --skip-nfsd-start to skip the NFS daemon stating, even if
  it is enabled
  https://phabricator.facebook.com/D4575368
  Alex Lorca 

There are also some "standard" changes: dealing with code that moved,
reindenting to comply with Gluster coding standards, gf_uuid_xxx, etc.

This patch *does* revert some changes which have occurred upstream since
3.6; these will be re-applied as apppropriate on top of this new base.

Change-Id: I69024115da7a60811e5b86beae781d602bdb558d
Signed-off-by: Jeff Darcy

nfs: Correctly reconfigure NFS options

2017-09-14T20:22:51+00:00

Summary:
A mistake was made in D2519423 where `ret` wasn't being set to `0` at the end of `nfs3_init_subvolume_options` since code was inserted between the final `ret = 0` and the return, causing the function to return phony positive ret values.
This causes the code to interpret the reconfigure function as a failure, meaning that changes can't be persisted.

This only affects the `reconfigure` path and not the `init` path, since the `reconfigure` path fails when `ret != 0` and the init path only fails when `ret == -1`...

Test Plan: See that volume options are actually being set when the `nfs` xlator is alive, instead of simply on init.

Reviewers: jdarcy, kvigor, dph, sshreyas

Reviewed By: sshreyas

Subscribers: #posix_storage

Differential Revision: https://phabricator.intern.facebook.com/D5699888

Change-Id: I89006ce3970f22a4206e58ca5630c21df536031c
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18293
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Add options to disable new features

2017-09-14T19:43:21+00:00

Summary: @sshreyas thought the best idea to roll out these new features in the default-off state. This diff adds a few options and modifies tests to make sure that this is done.

Test Plan: The brick restart test works fine, but now it's default disabled on all bricks.

Reviewers: sshreyas, jdarcy

Reviewed By: jdarcy

Subscribers: sshreyas, #posix_storage

Differential Revision: https://phabricator.intern.facebook.com/D5653138

Porting note: includes disconnected-reqs option; retart-bricks inapplicable

Change-Id: I332339894d3cbfafdabeb8592e95c37f30f9751a
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18291
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

Get glusterfs to output p50, p90, and p95 latencies

2017-09-13T20:22:11+00:00

Summary:
  [done] separate p99 dumping into general funcs
  [done] add p95, p90, and p50 stats
    - add p95, p90, p50 within p99, and generalize
    - rename config to dump-percentile-lantencies

Test Plan:
  make install glusterfs on dev machine.
  gluster volume create $name ...
  mount volume on /mnt/$name 
  dd if=/dev/zero of=/mnt/$name/test
  check each brick for pn printing

    /var/lib/glusterd/stats/glusterfsd__$brick.dump

Reviewers: sshreyas, kvigor, jdarcy

Reviewed By: jdarcy

Differential Revision: https://phabricator.intern.facebook.com/D5645951

Change-Id: Ic8ada48d9772bf2d5b3a2ba3c845d91d4e03c9d3
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/18279
Reviewed-by: Jeff Darcy 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System