glusterfs.git, branch v7.0rc1

core: fix memory allocation issues

2019-09-16T02:02:00+00:00

Two problems have been identified that caused that gluster's memory
usage were twice higher than required.

1. An off by 1 error caused that all objects allocated from the memory
   pools were taken from a pool bigger than required. Since each pool
   corresponds to a size equal to a power of two, this was wasting half
   of the available memory.

2. The header information used for accounting on each memory object was
   not taken into consideration when searching for a suitable memory
   pool. It was added later when each individual block was allocated.
   This made this space "invisible" to memory accounting.

Credits: Thanks to Nithya Balachandran for identifying this problem and
         testing this patch.

>Fixes: bz#1722802
Change-Id: I90e27ad795fe51ca11c13080f62207451f6c138c
>Signed-off-by: Xavi Hernandez 
>(cherry picked from commit 1716a907da1a835b658740f1325033d7ddd44952)

Fixes: bz#1748774
Change-Id: I90e27ad795fe51ca11c13080f62207451f6c138c
Signed-off-by: Xavi Hernandez

core/syncop: Bail out if frame creation fails

2019-09-15T13:20:25+00:00

There could be cases (either due to insufficient memory or
corrupted mem-pool) due to which frame creation fails. Bail out
with error in such cases.

This is the backport of below mainline fix -
> Fixes: bz#1748448
> review url: https://review.gluster.org/#/c/glusterfs/+/23350/

Change-Id: I8cc0a5852f6f04d2bac991e4eb79ecb42577da11
Fixes: bz#1751556
Signed-off-by: Soumya Koduri

glusterd: fix use-after-free of a dict_t

2019-09-15T06:32:20+00:00

A dict was passed to a function that calls dict_unref() without taking
any additional reference. Given that the same dict is also used after
the function returns, this was causing a use-after-free situation.

To fix the issue, we simply take an additional reference before calling
the function.

> Fixes: bz#1723890
> Change-Id: I98c6b76b08fe3fa6224edf281a26e9ba1ffe3017
> Signed-off-by: Xavi Hernandez 
> (cherry picked from commit f36086db87aae24c10abde434f081d78b942735e)

Fixes: bz#1752245
Change-Id: I98c6b76b08fe3fa6224edf281a26e9ba1ffe3017
Signed-off-by: Mohit Agrawal

afr/lookup: Pass xattr_req in while doing a selfheal in lookup

2019-09-11T05:04:47+00:00

We were not passing xattr_req when doing a name self heal
as well as a meta data heal. Because of this, some xdata
was missing which causes i/o errors

Backport of > https://review.gluster.org/#/c/glusterfs/+/23024/


>Change-Id: Ibfb1205a7eb0195632dc3820116ffbbb8043545f
>Fixes: bz#1728770
>Signed-off-by: Mohammed Rafi KC 

Fixes: bz#1749305
Change-Id: Ibfb1205a7eb0195632dc3820116ffbbb8043545f
Signed-off-by: Mohammed Rafi KC 
(cherry picked from commit d026f0bcfd301712e4f0671ccf238f43f2e6dd30)

rpc: Update address family if it is not provide in cmd-line arguments

2019-09-09T08:48:23+00:00

Problem: After enabling transport-type to inet6 and passed ipv6
         transport.socket.bind-address in glusterd.vol clients are
         not started.

Solution: Need to update address-family based on remote-address for
          all gluster client process

> Change-Id: Iaa3588cd87cebc45231bfd675745c1a457dc9b31
> Fixes: bz#1747746
> Credits: Amgad Saleh 
> Signed-off-by: Mohit Agrawal 
> (cherry picked from commit 80b8cfe3f1386606bada97a76a0cad7acdf6b877)

Change-Id: Iaa3588cd87cebc45231bfd675745c1a457dc9b31
Fixes: bz#1749664
Credits: Amgad Saleh 
Signed-off-by: Mohit Agrawal

glusterd: IPV6 hostname address is not parsed correctly

2019-09-06T08:05:10+00:00

Problem: IPV6 hostname address is not parsed correctly in function
         glusterd_check_brick_order

Solution: Update the code to parse hostname address

> Change-Id: Ifb2f83f9c6e987b2292070e048e97eeb51b728ab
> Fixes: bz#1747746
> Credits: Amgad Saleh 
> Signed-off-by: Mohit Agrawal 
> (cherry picked from commit 6563ffb04d7ba51a89726e7c5bbb85c7dbc685b5)

Change-Id: Ifb2f83f9c6e987b2292070e048e97eeb51b728ab
Fixes: bz#1749664
Credits: Amgad Saleh 
Signed-off-by: Mohit Agrawal

tests: fix spurious failure of bug-1402841.t-mt-dir-scan-race.t

2019-09-05T04:00:59+00:00

Problem:
Since commit 600ba94183333c4af9b4a09616690994fd528478, shd starts
healing as soon as it is toggled from disabled to enabled. This was
causing the following line in the .t to fail on a 'fast' machine (always
on my laptop and sometimes on the jenkins slaves).

EXPECT_NOT "^0$" get_pending_heal_count $V0

because by the time shd was disabled, the heal was already completed.

Fix:
Increase the no. of files to be healed and make it a variable called
FILE_COUNT, should we need to bump it up further because the machines
become even faster. Also created pending metadata heals to increase the
time taken to heal a file.

fixes: bz#1749155
Change-Id: I5a26b08e45b8c19bce3c01ce67bdcc28ed48198d
Signed-off-by: Ravishankar N 
(cherry picked from commit 724c657995a2e148243eeb78c68b620c6d7714a5)

doc: documented about fips-mode-rchecksum

2019-09-04T07:04:28+00:00

Updated release notes to document about fips-mode-rchecksum.
fixes: bz#1703322

Change-Id: Id6707fca6fc2dbc251f6e00e635a63d9e31f88f7
Signed-off-by: Rinku Kothiya

[RFC] change get_real_filename implementation to use ENOATTR instead of ENOENT

2019-09-03T13:21:37+00:00

get_real_filename is implemented as a virtual extended attribute to help
Samba implement the case-insensitive but case preserving SMB protocol
more efficiently. It is implemented as a getxattr call on the parent directory
with the virtual key of "get_real_filename:" by looking for a
spelling with different case for the provided file/dir name ()
and returning this correct spelling as a result if the entry is found.
Originally (05aaec645a6262d431486eb5ac7cd702646cfcfb), the
implementation used the ENOENT errno to return the authoritative answer
that  does not exist in any case folding.

Now this implementation is actually a violation or misuse of the defined
API for the getxattr call which returns ENOENT for the case that the dir
that the call is made against does not exist and ENOATTR (or the synonym
ENODATA) for the case that the xattr does not exist.

This was not a problem until the gluster fuse-bridge was changed
to do map ENOENT to ESTALE in 59629f1da9dca670d5dcc6425f7f89b3e96b46bf,
after which we the getxattr call for get_real_filename returned an
ESTALE instead of ENOENT breaking the expectation in Samba.

It is an independent problem that ESTALE should not leak out to user
space but is intended to trigger retries between fuse and gluster.
But nevertheless, the semantics seem to be incorrect here and should
be changed.

This patch changes the implementation of the get_real_filename virtual
xattr to correctly return ENOATTR instead of ENOENT if the file/directory
being looked up is not found.

The Samba glusterfs_fuse vfs module which takes advantage of the
get_real_filename over a fuse mount will receive a corresponding change
to map ENOATTR to ENOENT. Without this change, it will still work
correctly, but the performance optimization for nonexisting files is
lost. On the other hand side, this change removes the distinction
between the old not-implemented case and the implemented case.
So Samba changed to treat ENOATTR like ENOENT will not work correctly
any more against old servers that don't implement get_real_filename.
I.e. existing files will be reported as non-existing

Change-Id: I971b427ab8410636d5d201157d9af70e0d075b67
fixes: bz#1745914
Signed-off-by: Michael Adam 
(cherry picked from commit dc1b87fcfef08c9497b0c02b2410c9d18bbc2dba)

afr: wake up index healer threads

2019-08-30T05:04:56+00:00

...whenever shd is re-enabled after disabling or there is a change in
`cluster.heal-timeout`, without needing to restart shd or waiting for the
current `cluster.heal-timeout` seconds to expire.

See BZ 1743988 for more details.

Change-Id: Ia5ebd7c8e9f5b54cba3199c141fdd1af2f9b9bfe
fixes: bz#1747301
Reported-by: Glen Kiessling 
Signed-off-by: Ravishankar N 
(cherry picked from commit 600ba94183333c4af9b4a09616690994fd528478)