glusterfs.git/xlators/mgmt/glusterd, branch v7.7

glusterd: Brick process fails to come up with brickmux on

2020-03-17T06:04:40+00:00

Issue:
1- In a cluster of 3 Nodes N1, N2, N3. Create 3 volumes vol1,
vol2, vol3 with 3 bricks (one from each node)
2- Set cluster.brick-multiplex on
3- Start all 3 volumes
4- Check if all bricks on a node are running on same port
5- Kill N1
6- Set performance.readdir-ahead for volumes vol1, vol2, vol3
7- Bring N1 up and check volume status
8- All bricks processes not running on N1.

Root Cause -
Since, There is a diff in volfile versions in N1 as compared
to N2 and N3 therefore glusterd_import_friend_volume() is called.
glusterd_import_friend_volume() copies the new_volinfo and deletes
old_volinfo and then calls glusterd_start_bricks().
glusterd_start_bricks() looks for the volfiles and sends an rpc
request to glusterfs_handle_attach(). Now, since the volinfo
has been deleted by glusterd_delete_stale_volume()
from priv->volumes list before glusterd_start_bricks() and
glusterd_create_volfiles_and_notify_services() and
glusterd_list_add_order is called after glusterd_start_bricks(),
therefore the attach RPC req gets an empty volfile path
and that causes the brick to crash.

Fix- Call glusterd_list_add_order() and
glusterd_create_volfiles_and_notify_services before
glusterd_start_bricks() cal is made in glusterd_import_friend_volume

> Change-Id: Idfe0e8710f7eb77ca3ddfa1cabeb45b2987f41aa
> Bug: bz#1773856
> Signed-off-by: Mohammed Rafi KC 
(cherry picked from commit 45e81aae791da9d013aba2286af44826227c05ec)

Change-Id: Idfe0e8710f7eb77ca3ddfa1cabeb45b2987f41aa
fixes: bz#1808964
Signed-off-by: Sanju Rakonde

glusterd: stop stale bricks during handshaking in brick mux mode

2020-03-16T08:25:40+00:00

This patch addresses two problems:

1. During friend handshaking, if a volume is imported due to change in
the version, the old bricks were not stopped which would lead to a
situation where bricks will run with old volfiles.

2. As part of attaching shd service in glusterd_attach_svc, there might
be a case that the volume for which we're attempting to attach a shd
service might become stale and in the process of deletion and hence in
every retrials (if the rpc connection isn't ready) check for the
existance of the volume and then only attempt the further attach
request.

patch on master: https://review.gluster.org/#/c/glusterfs/+/23042/

> Bug: bz#1733425
> Change-Id: I6bac6b871f7e31cb5bf277db979289dec196a03e
> Signed-off-by: Atin Mukherjee 
> Signed-off-by: Mohammed Rafi KC 

fixes: bz#1812849
Change-Id: I6bac6b871f7e31cb5bf277db979289dec196a03e
Signed-off-by: Sanju Rakonde

volgen: make thin-arbiter name unique in 'pending-xattr' option

2020-02-17T09:52:06+00:00

Thin-arbiter module makes use of 'pending-xattr' name for the translator
as the filename which gets created in thin-arbiter node. By making this
unique, we can host single thin-arbiter node for multiple clusters.

Updates: #763
Change-Id: Ib3c732e7e04e6dba229e71ae3e64f1f3cb6d794d
Signed-off-by: Amar Tumballi 
(cherry picked from commit 8db8202f716fd24c8c52f8ee5f66e169310dc9b1)

afr: expose cluster.optimistic-change-log to CLI.

2020-01-09T13:22:22+00:00

Backport of https://review.gluster.org/#/c/glusterfs/+/23960/

This volume option was not made avaialble to `gluster volume set` CLI.

Reported-by: epolakis(https://github.com/kinsu) in
https://github.com/gluster/glusterfs/issues/781

fixes: bz#1788785
Change-Id: I7141bdd4e53ee99e22b354edde8d023bfc0b2cd7
Signed-off-by: Ravishankar N

glusterd: fix use-after-free of a dict_t

2019-09-15T06:32:20+00:00

A dict was passed to a function that calls dict_unref() without taking
any additional reference. Given that the same dict is also used after
the function returns, this was causing a use-after-free situation.

To fix the issue, we simply take an additional reference before calling
the function.

> Fixes: bz#1723890
> Change-Id: I98c6b76b08fe3fa6224edf281a26e9ba1ffe3017
> Signed-off-by: Xavi Hernandez 
> (cherry picked from commit f36086db87aae24c10abde434f081d78b942735e)

Fixes: bz#1752245
Change-Id: I98c6b76b08fe3fa6224edf281a26e9ba1ffe3017
Signed-off-by: Mohit Agrawal

glusterd: IPV6 hostname address is not parsed correctly

2019-09-06T08:05:10+00:00

Problem: IPV6 hostname address is not parsed correctly in function
         glusterd_check_brick_order

Solution: Update the code to parse hostname address

> Change-Id: Ifb2f83f9c6e987b2292070e048e97eeb51b728ab
> Fixes: bz#1747746
> Credits: Amgad Saleh 
> Signed-off-by: Mohit Agrawal 
> (cherry picked from commit 6563ffb04d7ba51a89726e7c5bbb85c7dbc685b5)

Change-Id: Ifb2f83f9c6e987b2292070e048e97eeb51b728ab
Fixes: bz#1749664
Credits: Amgad Saleh 
Signed-off-by: Mohit Agrawal

event: rename event_XXX with gf_ prefixed

2019-08-21T06:13:38+00:00

I hit one crash issue when using the libgfapi.

In the libgfapi it will call glfs_poller() --> event_dispatch()
in file api/src/glfs.c:721, and the event_dispatch() is defined
by libgluster locally, the problem is the name of event_dispatch()
is the extremly the same with the one from libevent package form
the OS.

For example, if a executable program Foo, which will also use and
link the libevent and the libgfapi at the same time, I can hit the
crash, like:

kernel: glfs_glfspoll[68486]: segfault at 1c0 ip 00007fef006fd2b8 sp
00007feeeaffce30 error 4 in libevent-2.0.so.5.1.9[7fef006ed000+46000]

The link for Foo is:
lib_foo_LADD = -levent $(GFAPI_LIBS)
It will crash.

This is because the glfs_poller() is calling the event_dispatch() from
the libevent, not the libglsuter.

The gfapi link info :
GFAPI_LIBS = -lacl -lgfapi -lglusterfs -lgfrpc -lgfxdr -luuid

If I link Foo like:
lib_foo_LADD = $(GFAPI_LIBS) -levent
It will works well without any problem.

And if Foo call one private lib, such as handler_glfs.so, and the
handler_glfs.so will link the GFAPI_LIBS directly, while the Foo won't
and it will dlopen(handler_glfs.so), then the crash will be hit everytime.

The link info will be:
foo_LADD = -levent
libhandler_glfs_LIBADD = $(GFAPI_LIBS)

I can avoid the crash temporarily by linking the GFAPI_LIBS in Foo too like:
foo_LADD = $(GFAPI_LIBS) -levent
libhandler_glfs_LIBADD = $(GFAPI_LIBS)

But this is ugly since the Foo won't use any APIs from the GFAPI_LIBS.

And in some cases when the --as-needed link option is added(on many dists
it is added as default), then the crash is back again, the above workaround
won't work.

Backport of:
> https://review.gluster.org/#/c/glusterfs/+/23110/
> Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa
> Fixes: #699
> Signed-off-by: Xiubo Li 

Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa
updates: bz#1740519
Signed-off-by: Xiubo Li 
(cherry picked from commit 799edc73c3d4f694c365c6a7c27c9ab8eed5f260)

glusterd: don't log a warning message for tier-enabled key

2019-07-29T13:20:52+00:00

We are logging a warning message saying unknown-key for
tier-enabled kay. although the tier xlator is deprecated,
this key is left behind for handling the peer rejection
issues in a heterogeneous cluster. We need not to log if
this key is not found/recognised.

updates: bz#1727008

Change-Id: Ia68661898a618f99a240ca8d8a124ff6a65ebe9d
Signed-off-by: Sanju Rakonde 
(cherry picked from commit 7d270d05f835d41e32572501246b50181dc9be56)

glusterd/svc: update pid of mux volumes from the shd process

2019-07-24T10:29:17+00:00

For a normal volume, we are updating the pid from a the
process while we do a daemonization or at the end of the
init if it is no-daemon mode. Along with updating the pid
we also lock the file, to make sure that the process is
running fine.

With brick mux, we were updating the pidfile from gluterd
after an attach/detach request.

There are two problems with this approach.
1) We are not holding a pidlock for any file other than parent
   process.
2) There is a chance for possible race conditions with attach/detach.
   For example, shd start and a volume stop could race. Let's say
   we are starting an shd and it is attached to a volume.
   While we trying to link the pid file to the running process,
   this would have deleted by the thread that doing a volume stop.

Backport of : https://review.gluster.org/#/c/glusterfs/+/22935/
>Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a
>Updates: bz#1722541
>Signed-off-by: Mohammed Rafi KC 

Change-Id: I29a00352102877ce09ea3f376ca52affceb5cf1a
Updates: bz#1732668
Signed-off-by: Mohammed Rafi KC

glusterd: conditionally clear txn_opinfo in stage op

2019-07-20T07:40:12+00:00

...otherwise this leads to a crash when volume status is run on a
heterogeneous mode.

> Fixes: bz#1723658
> Change-Id: I0d39f412b2e5e9d3ef0a3462b90b38bb5364b09d
> Signed-off-by: Atin Mukherjee 
> (cherry picked from commit a72452fcf90679b28baec12d2769cbaa982bb4e4)

Fixes: bz#1728127
Change-Id: I0d39f412b2e5e9d3ef0a3462b90b38bb5364b09d
Signed-off-by: Atin Mukherjee