glusterfs.git/xlators/mgmt/glusterd, branch v8.1

glusterd: Increase buffer length to save multiple hostnames in peer file

2020-08-19T18:11:28+00:00

Problem: At the time of handling friend update request glusterd updates peer
         file and if DNS has returned multiple hostnames for the same IP, glusterd
         saves all hostnames in peer file.In commit 1fa089e7a2b180e0bdcc1e7e09a63934a2a0c0ef
         We changed the approach to save all key value pairs in single shot.
         In case of a buffer is not having space to store the hostnames glusterd
         writes partial hostname in peer file.

Solution: To avoid the failure increase the buffer length

Change-Id: Iee969d165333e9c5ba69431d474c541b8f12d442
Fixes: #1407
Signed-off-by: Mohit Agrawal 
(cherry picked from commit 6e8e73a06d71382f8f6e3cd83fe72692d19e66ba)

syncop: improve scaling and implement more tools

2020-05-30T05:17:12+00:00

The current scaling of the syncop thread pool is not working properly
and can leave some tasks in the run queue more time than necessary
when the maximum number of threads is not reached.

This patch provides a better scaling condition to react faster to
pending work.

Condition variables and sleep in the context of a synctask have also
been implemented. Their purpose is to replace regular condition
variables and sleeps that block synctask threads and prevent other
tasks to be executed.

The new features have been applied to several places in glusterd.

Change-Id: Ic50b7c73c104f9e41f08101a357d30b95efccfbf
Fixes: #1116
Signed-off-by: Xavi Hernandez

mgmt/glusterd: use stat() syscall wrapper

2020-04-13T14:50:29+00:00

Found with 0-symbol-check.t:

./tests/basic/0symbol-check.t ..
1..2
./xlators/mgmt/glusterd/src/.libs/glusterd_la-glusterd-volume-set.o should call sys_stat, not stat
ok   1 [     40/  41011] <  40> 'find . -name *.o -exec ./tests/basic/symbol-check.sh {} \;'
not ok   2 [     11/      1] <  42> '[ ! -e ./.symbol-check-errors ]' -> ''
Failed 1/2 subtests

Change-Id: I8962f487cd88738a1f7a962049d513712687088c
Fixes: #1160
Signed-off-by: Dmitry Antipov

mgmt/glusterd: Reduce log level of repetitive log

2020-04-06T01:38:39+00:00

Noticed that the following message repeats quite a bit in log files when
an external monitoring tool queries gluster for list of volumes
periodically:

"Received get vol req"

As there's not much value in having this log message at log level INFO,
changing the log level to DEBUG to make glusterd.log a bit quieter.

Change-Id: I4e791fc65b9a4f813d295e7b2b6a05f3c0782e69
Updates: #1000
Signed-off-by: Vijay Bellur

glusterd/snapshot: Add a warning message when a volume config changes

2020-03-12T11:27:16+00:00

While changing a volume configuration, there is a chance that the
brick layout on the disk might be changed. If snapshot is present
on such volumes that will effects it's working. So this patch adds
a warning message if snapshot is present while a volume config change
happen.

Change-Id: I7256863fef734841fce0bc9ad94d5d201b1813d5
Fixes: bz#1812144
Signed-off-by: Mohammed Rafi KC

mgmt/glusterd: Adding validation for statedump path

2020-03-09T06:25:19+00:00

Description of problem:
server.statedump-path is the path where statedumps are stored,
by default it is /var/run/gluster. And can be set to any valid
directory path. It was observed that server.statedump-path was
also accepting file, non-existent file and non-existent paths
as well. And statedump command was successful even when
statedumps with all the invalid paths.
a. A file
b. A non-existent path

Solution:
Added a validation function in gluster-volume-set.c which will
allow volume set to success if it's a valid directory
and in all other cases, volume set should fail.

Fixes: bz#1787122

Change-Id: Ia66e2b3d35f23efc5444c829928779a79d827b42
Signed-off-by: yatipadia 
Signed-off-by: Sanju Rakonde

glusterd: Brick process fails to come up with brickmux on

2020-02-20T06:39:55+00:00

Issue:
1- In a cluster of 3 Nodes N1, N2, N3. Create 3 volumes vol1,
vol2, vol3 with 3 bricks (one from each node)
2- Set cluster.brick-multiplex on
3- Start all 3 volumes
4- Check if all bricks on a node are running on same port
5- Kill N1
6- Set performance.readdir-ahead for volumes vol1, vol2, vol3
7- Bring N1 up and check volume status
8- All bricks processes not running on N1.

Root Cause -
Since, There is a diff in volfile versions in N1 as compared
to N2 and N3 therefore glusterd_import_friend_volume() is called.
glusterd_import_friend_volume() copies the new_volinfo and deletes
old_volinfo and then calls glusterd_start_bricks().
glusterd_start_bricks() looks for the volfiles and sends an rpc
request to glusterfs_handle_attach(). Now, since the volinfo
has been deleted by glusterd_delete_stale_volume()
from priv->volumes list before glusterd_start_bricks() and
glusterd_create_volfiles_and_notify_services() and
glusterd_list_add_order is called after glusterd_start_bricks(),
therefore the attach RPC req gets an empty volfile path
and that causes the brick to crash.

Fix- Call glusterd_list_add_order() and
glusterd_create_volfiles_and_notify_services before
glusterd_start_bricks() cal is made in glusterd_import_friend_volume

Change-Id: Idfe0e8710f7eb77ca3ddfa1cabeb45b2987f41aa
Fixes: bz#1773856
Signed-off-by: Mohammed Rafi KC

volgen: make thin-arbiter name unique in 'pending-xattr' option

2020-02-12T16:16:46+00:00

Thin-arbiter module makes use of 'pending-xattr' name for the translator
as the filename which gets created in thin-arbiter node. By making this
unique, we can host single thin-arbiter node for multiple clusters.

Updates: #763
Change-Id: Ib3c732e7e04e6dba229e71ae3e64f1f3cb6d794d
Signed-off-by: Amar Tumballi

feature/changelog: Avoid thread creation if xlator is not enabled

2020-02-09T12:42:56+00:00

Problem:
Changelog creates threads even if the changelog is not enabled

Background:
Changelog xlator broadly does two things
  1. Journalling - Cosumers are geo-rep and glusterfind
  2. Event Notification for registered events like (open, release etc) -
     Consumers are bitrot, geo-rep

The existing option "changelog.changelog" controls journalling and
there is no option to control event notification and is enabled by
default. So when bitrot/geo-rep is not enabled on the volume, threads
and resources(rpc and rbuf) related to event notifications consumes
resources and cpu cycle which is unnecessary.

Solution:
The solution is to have two different options as below.
 1. changelog-notification : Event notifications
 2. changelog : Journalling

This patch introduces the option "changelog-notification" which is
not exposed to user. When either bitrot or changelog (journalling)
is enabled, it internally enbales 'changelog-notification'. But
once the 'changelog-notification' is enabled, it will not be disabled
for the life time of the brick process even after bitrot and changelog
is disabled. As of now, rpc resource cleanup has lot of races and is
difficult to cleanup cleanly. If allowed, it leads to memory leaks
and crashes on enable/disable of bitrot or changelog (journal) in a
loop. Hence to be safer, the event notification is not disabled within
lifetime of process once enabled.

Change-Id: Ifd00286e0966049e8eb9f21567fe407cf11bb02a
Updates: #475
Signed-off-by: Mohit Agrawal

bitrot: Make number of signer threads configurable

2020-02-07T06:30:19+00:00

The number of signing process threads (glfs_brpobj)
is set to 4 by default. The recommendation is to set
it to number of cores available. This patch makes it
configurable as follows

gluster vol bitrot  signer-threads 

fixes: bz#1797869
Change-Id: Ia883b3e5e34e0bc8d095243508d320c9c9c58adc
Signed-off-by: Kotresh HR