glusterfs.git/xlators/mgmt/glusterd/src, branch v6.0

glusterd: glusterd memory leak while running "gluster v profile" in a loop

2019-03-12T20:51:03+00:00

Problem: glusterd has memory leak while running "gluster v profile"
         in a loop

Solution: Resolve leak code path to avoid leak

> Change-Id: Id608703ff6d0ad34ed8f921a5d25544e24cfadcd
> fixes: bz#1685414
> (Cherry pick from commit 9374484917466dff4688d96ff7faa0de1c804a6c)
> (Reviewed on link https://review.gluster.org/#/c/glusterfs/+/22301/)

Change-Id: I1ca118265f97b188f94b3d5cff649ec36cb18ca0
fixes: bz#1685771
Signed-off-by: Mohit Agrawal

core: make compute_cksum function op_version compatible

2019-03-08T14:45:41+00:00

Problem: commit 5a152a changed the mechansim of computing the
checksum. In heterogeneous cluster, peers are running into
rejected state because we have different cksum computation
mechansims in upgraded and non-upgraded nodes.

Solution: add a check for op-version so that all the nodes
in the cluster follow the same mechanism for computing the
cksum.

fixes: bz#1684029

> Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
> BUG: bz#1685120
> Signed-off-by: Sanju Rakonde 
> (cherry picked from commit 073444b693b7a91c42963512e0fdafb57ad46670)

Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193

glusterd: remove experimental xlator options from glusterd-volume-set.c

2019-02-27T03:39:55+00:00

experimental xlators have been removed from the codebase. But we
missed to remove the options related to experimental xlators from
the codebase. This patch removes those options.

fixes: bz#1683506
Change-Id: I3fa7e14c6cd8ebde5cebc8d2b0cb2409bf37c1ae
Signed-off-by: Sanju Rakonde 
(cherry picked from commit 5cddd4d758014fe116d9c130632eada2ecded88c)

performance/md-cache: introduce an option to control invalidation of inodes

2019-02-18T14:44:27+00:00

Explicit invalidation by calling inode_invalidate is necessary when
same (meta)data is shared/access across multiple mounts. Without an
explicit inode_invalidate call, caches in the mount which didn't
witness writes wouldn't be aware of changes as writes wouldn't have
passed through them. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, explicit inode
invalidation can be disabled for this case which gives a huge
performance boost for workloads that write data and then immediately
read the data they just wrote. Note that otherwise, local writes
(which pass through the cache) will change ctime and cause unnecessary
invalidations.

The name of the option that controls this behavior is
"performance.global-cache-invalidation". This option is global and it
purges caches both in glusterfs and kernel stack for native FUSE
mounts. For non-native FUSE mounts, it purges cache only from
glusterfs stack. This option is effective only when
performance.stat-prefetch is on.

Note that there is a similar option "performance.cache-invalidation",
but the scope of that option is limited to quick-read and md-cache.

Change-Id: I462bb4b65ff9aae1f6ba76f50b1f2f94fb10323b
Signed-off-by: Raghavendra Gowdappa 
updates: bz#1674364
(cherry picked from commit 2b5aa4489de2017a03bcb6ec8986286f0c76a670)

glusterd: improve logging

2019-02-11T16:24:39+00:00

glusterd_resolve_all_bricks failure log should highlight the brick
identifier.

Change-Id: I035b4650ef6a14bb1e1221d3bad1c40f9d43dbdd
fixes: bz#1673972
Signed-off-by: Atin Mukherjee 
(cherry picked from commit 12af2067a82e37079e76723d3e25ba1c72ca078a)

glusterd: get-state command should not fail if any brick is gone bad

2019-02-05T14:40:25+00:00

Problem: get-state command will error out, if any of the underlying
brick(s) of volume(s) in the cluster go bad.

It is expected that get-state command should not error out, but
should generate an output successfully.

Solution: In glusterd_get_state(), a statfs call is made on the
brick path for every bricks of the volumes to calculate the total
and free memory available. If any of statfs call fails on any
brick, we should not error out and should report total memory and free
memory of that brick as 0.

This patch also handles a statfs failure scenario in
glusterd_store_retrieve_bricks().

fixes: bz#1672205

Change-Id: Ia9e8a1d8843b65949d72fd6809bd21d39b31ad83
Signed-off-by: Sanju Rakonde

glusterd: manage upgrade to current master

2019-02-04T03:47:15+00:00

Scenarios tested:

* Upgrade the node when there are stripe / tiering and regular
type of volumes are present.
  - All volumes are started fine (as the change was not on brick volfile)
  - For tier, the functionality may not even work, as changetimerecorder
    is not present.
  - 'gluster volume info' properly shows as 'NOT SUPPORTED' for stripe and
    tier type of volume.

* Upgrade in a rolling upgrade scenario, where an old version is
able to connect to higher master.
  - on a normal volume, if the volfile-server was new, the newer client
    volfiles needed to have utime xlator conditionally.
  - with this one change, all other changes seem to work fine.

Change-Id: Ib2d3b69dafa02b2c695a735b13c1aa70aba07cb8
updates: bz#1635688
Signed-off-by: Amar Tumballi

features/sdfs: disable by default

2019-01-29T13:54:22+00:00

With the feature enabled, some of the performance testing results,
specially those which create millions of small files, got approximately
4x regression compared to version before enabling this.

On master without this patch:  765 creates/sec
On master with this patch   : 3380 creates/sec

Also there seems to be regression caused by this in 'ls -l' workload.

On master without this patch:  3030 files/sec
On master with this patch   : 16610 files/sec

This is a feature added to handle multiple clients parallely operating
(specially those which race for file creates with same name) on a single
namespace/directory. Considering that is < 3% of Gluster's usecase right
now, it makes sense to disable the feature by default, so we don't
penalize the default users who doesn't bother about this usecase.
Also note that the client side translators, specially, distribute,
replicate and disperse already handle the issue upto 99.5% of the cases
without SDFS, so it makes sense to keep the feature disabled by default.

Credits: Shyamsunder  for running the tests and
getting the numbers.

Change-Id: Iec49ce1d82e621e9db25eb633fcb1d932e74f4fc
Updates: bz#1670031
Signed-off-by: Amar Tumballi

rpc: use address-family option from vol file

2019-01-22T13:47:19+00:00

This patch helps enable IPv6 connections in the cluster.
The default address-family is IPv4 without using this option explicitly.

When address-family is set to "inet6" in the /etc/glusterfs/glusterd.vol
file, the mount command-line also needs to have
-o xlator-option="transport.address-family=inet6" added to it.

This option also gets added to the brick command-line.
Snapshot and gfapi use-cases should also use this option to pass in the
inet6 address-family.

Change-Id: I97db91021af27bacb6d7578e33ea4817f66d7270
fixes: bz#1635863
Signed-off-by: Milind Changire

locks/fencing: Add a security knob for fencing

2019-01-22T05:23:44+00:00

There is a low level security issue with fencing since one client
can preempt another client's lock.

This patch does not completely eliminate the issue of a client
misbehaving, but certainly it adds a security layer for default use cases
that does not need fencing.

Change-Id: I55cd15f2ed1ae0f2556e3d27a2ef4bc10fdada1c
updates: #466
Signed-off-by: Susant Palai