glusterfs.git/glusterfsd/src, branch release-3.8-fb

Replace namespace/io-stats/io-threads with 3.6-fb versions

2017-09-15T20:47:01+00:00

This rolls up multiple patches related to namespace identificaton and
throttling/QoS.  This primarily includes the following, all by Michael
Goulet .

  io-threads: Add weighted round robin queueing by namespace
  https://phabricator.facebook.com/D5615269

  io-threads: Add per-namespaces queue sizes to IO_THREADS_QUEUE_SIZE_KEY
  https://phabricator.facebook.com/D5683162

  io-threads: Implement better slot allocation algorithm
  https://phabricator.facebook.com/D5683186

  io-threads: Only enable weighted queueing on bricks
  https://phabricator.facebook.com/D5700062

  io-threads: Update queue sizes on drain
  https://phabricator.facebook.com/D5704832

  Fix parsing (-1) as default NS weight
  https://phabricator.facebook.com/D5723383

Parts of the following patches have also been applied to satisfy
dependencies.

  io-throttling: Calculate moving averages and throttle offending hosts
  https://phabricator.fb.com/D2516161
  Shreyas Siravara 

  Hook up ODS logging for FUSE clients.
  https://phabricator.facebook.com/D3963376
  Kevin Vigor 

  Add the flag --skip-nfsd-start to skip the NFS daemon stating, even if
  it is enabled
  https://phabricator.facebook.com/D4575368
  Alex Lorca 

There are also some "standard" changes: dealing with code that moved,
reindenting to comply with Gluster coding standards, gf_uuid_xxx, etc.

This patch *does* revert some changes which have occurred upstream since
3.6; these will be re-applied as apppropriate on top of this new base.

Change-Id: I69024115da7a60811e5b86beae781d602bdb558d
Signed-off-by: Jeff Darcy

Repair more cluster tests in FB IPv6 environment

2017-01-07T00:13:26+00:00

Summary:

'transport.address-family' option is not universally supported,
which leads to woe using the IPV4 only cluster tests in FB IPv6
default build.

Two fixes here: first, pass transport.address-family to brick
deamons on startup; and second, honor the option when attaching
to management deamon.

Test Plan:
./tests/basic/mgmt_v3-locks.t in FB environment.

Previously failed every time, now succeeds

Reviewers:

Subscribers:

Tasks:

Blame Revision:

Change-Id: I2918004ff9f9d962d10684e3d3d0783eae6cb838
Signed-off-by: Kevin Vigor 
Reviewed-on: http://review.gluster.org/16353
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shreyas Siravara

Merge remote-tracking branch 'origin/release-3.8' into merge-3.8-again

2017-01-05T20:21:20+00:00

Change-Id: I844adf2aef161a44d446f8cd9b7ebcb224ee618a
Signed-off-by: Kevin Vigor

glusterfsd : fix null pointer dereference in glusterfs_handle_barrier

2017-01-02T13:09:08+00:00

>Reviewed-on: http://review.gluster.org/16043
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Vijay Bellur 

Change-Id: Iab86a3c4970e54c22d3170e68708e0ea432a8ea4
BUG: 1402697
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/16066
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

Repair cluster prove tests for FB environment

2016-12-21T18:58:54+00:00

Summary:

Several prove tests use the 'launch_cluster' function to set up a
clustered volume. This replies on using multiple local IP
addresses, one for each server. Since IPV6 provides only ::1 as
a local address, as opposed to IPv4's complete 127.x.x.x subnet,
this cannot work in a pure IPv6 environment.

However, FB systems do at least have enough IPv4 stack to talk
locally, so fix launch_cluster to work properly when default
transport is IPv6.

To do this:

1) explicitly set transport.address-family volume option to inet in
   launch_cluster().

2) teach glusterd to honor transport.address-family when connecting
   to peer glusterds in glusterd_friend_rpc_create(). Previously
   transport.address-family was used only for binding local socket,
   not for communicating with peers.

Test Plan:

prove -f --timer ./tests/basic/glusterd/arbiter-volume-probe.t

Reviewers:

Subscribers:

Tasks:

Blame Revision:

Change-Id: I077d8549dcdbe4919ac7df34856a4b2d1428cdb6
Signed-off-by: Kevin Vigor 
Reviewed-on: http://review.gluster.org/16225
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Shreyas Siravara 
Smoke: Gluster Build System

glusterd: Retry volfile server multiple times

2016-12-09T17:10:01+00:00

Summary:
- Retry the volfile server when the initial connection fails. The
  default connect attempts is currently 200.
- This is a port D2174716 & D3792748 to 3.8.

Test Plan: Tested retry functionality on devserver.

Reviewed By: rwareing

Signed-off-by: Shreyas Siravara 

Change-Id: I22810d52b43107cc156483649fc160612677858a
Reviewed-on: http://review.gluster.org/16077
Tested-by: Shreyas Siravara 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kevin Vigor

glusterfsd: Continuous errors are getting in mount logs while glusterd is down

2016-11-23T10:38:15+00:00

Problem: when glusterd is down, getting the continuous mgmt_rpc_notify errors
         messages in the volume mount log for every 3 seconds,it will
         consume disk space.

Solution: To reduce the frequency of error messages use GF_LOG_OCCASIONALLY.

> BUG: 1388877
> Change-Id: I6cf24c6ddd9ab380afd058bc0ecd556d664332b1
> Signed-off-by: Mohit Agrawal 
> Reviewed-on: http://review.gluster.org/15732
> NetBSD-regression: NetBSD Build System 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Jeff Darcy 
> (cherry picked from commit 7874ed245bcc80658547992205f8396f4dd3c76a)

Change-Id: I32c7a2271333686b27b43e54a334c58d3da60a1d
BUG: 1394108
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15822
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Prashanth Pai

fuse: usage text is wrong for use-readdirp mount default

2016-10-18T07:02:25+00:00

Backport of http://review.gluster.org/#/c/14622/

Change-Id: I0020cd1862a9af11898813b6b31bd7f9125936d2
Bug: 1384345
Signed-off-by: Ankit Raj 
Reviewed-on: http://review.gluster.org/15635
Tested-by: ankitraj
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterfsd/main: fix OOM adjustment for older kernels

2016-10-12T11:01:28+00:00

Milind Changire reported that GlusterFS fails to build on RHEL5
because linux/oom.h is unavailable.

Milind's initial patch disables OOM adjustment completely
for those environments that do not have this header. However,
I'd take another approach that:

1) checks for linux/oom.h in compile-time and defines necessary
constants if the header is not present;
2) checks for available OOM API in /proc in run-time and uses it
accordingly.

This allows OOM to be adjusted properly on RHEL5 (the kernel is pretty new
to present /proc API for that) as well as RHEL6 (the kernel has many thing
backported including new /proc API).

> Reviewed-on: http://review.gluster.org/15587
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Niels de Vos 
> Tested-by: Oleksandr Natalenko 

Change-Id: I1bc610586872d208430575c149a7d0c54bd82370
BUG: 1383694
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15623
Tested-by: Oleksandr Natalenko 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

glusterfsd: explicitly turn on encryption for volfile fetch

2016-09-28T14:55:34+00:00

Problem: In case of encrypted transport RPC clients not able to
         reconnect.due to this daemon(glustershd etc) not able to
         fetch volfile and not started.

Solution: After turn on encryption explictly to fetch volfile
          issue is resolved.

> Change-Id: I58e1fe7f5edf0abb5732432291ff677e81429b79
> BUG: 1333317
> Signed-off-by: Kaushal M 
> Reviewed-on: http://review.gluster.org/14253
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
> (cherry picked from commit 60d235515e582319474ba7231aad490d19240642)

Change-Id: I15193837dc692b0cd7df942843bcf27a1c47e695
BUG: 1379216
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/15567
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System