glusterfs.git/libglusterfs/src, branch v6.0rc1

glusterfsd: Multiple shd processes are spawned on brick_mux environment

2019-03-12T20:52:59+00:00

Problem: Multiple shd processes are spawned while starting volumes
         in the loop on brick_mux environment.glusterd spawn a process
         based on a pidfile and shd daemon is taking some time to
         update pid in pidfile due to that glusterd is not able to
         get shd pid

Solution: Commit cd249f4cb783f8d79e79468c455732669e835a4f changed
          the code to update pidfile in parent for any gluster daemon
          after getting the status of forking child in parent.To resolve
          the same correct the condition update pidfile in parent only
          for glusterd and for rest of the daemon pidfile is updated in
          child

> Change-Id: Ifd14797fa949562594a285ec82d58384ad717e81
> fixes: bz#1684404
> (Cherry pick from commit 66986594a9023c49e61b32769b7e6b260b600626)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22290/)

Change-Id: I9a68064d2da1acd0ec54b4071a9995ece0c3320c
fixes: bz#1683880
Signed-off-by: Mohit Agrawal

core: make compute_cksum function op_version compatible

2019-03-08T14:45:41+00:00

Problem: commit 5a152a changed the mechansim of computing the
checksum. In heterogeneous cluster, peers are running into
rejected state because we have different cksum computation
mechansims in upgraded and non-upgraded nodes.

Solution: add a check for op-version so that all the nodes
in the cluster follow the same mechanism for computing the
cksum.

fixes: bz#1684029

> Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
> BUG: bz#1685120
> Signed-off-by: Sanju Rakonde 
> (cherry picked from commit 073444b693b7a91c42963512e0fdafb57ad46670)

Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193

socket: socket event handlers now return void

2019-03-02T11:54:24+00:00

Problem:
Returning any value from socket event handlers to the event sub-system
doesn't make sense since event sub-system cannot handle socket
sub-system errors.

Solution:
Change return type of all socket event handlers to 'void'

mainline:
> Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
> Fixes: bz#1651246
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/c/glusterfs/+/22221

Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
Fixes: bz#1683900
Signed-off-by: Milind Changire 
(cherry picked from commit 776ba851c6ee6c265253d44cf1d6e4e3d4a21772)

glusterd: manage upgrade to current master

2019-02-04T03:47:15+00:00

Scenarios tested:

* Upgrade the node when there are stripe / tiering and regular
type of volumes are present.
  - All volumes are started fine (as the change was not on brick volfile)
  - For tier, the functionality may not even work, as changetimerecorder
    is not present.
  - 'gluster volume info' properly shows as 'NOT SUPPORTED' for stripe and
    tier type of volume.

* Upgrade in a rolling upgrade scenario, where an old version is
able to connect to higher master.
  - on a normal volume, if the volfile-server was new, the newer client
    volfiles needed to have utime xlator conditionally.
  - with this one change, all other changes seem to work fine.

Change-Id: Ib2d3b69dafa02b2c695a735b13c1aa70aba07cb8
updates: bz#1635688
Signed-off-by: Amar Tumballi

mount/fuse: expose auto-invalidation as a mount option

2019-02-02T03:07:35+00:00

Auto invalidation is necessary when same (meta)data is shared/access
across multiple mounts. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, fuse-auto-invalidation
can be disabled for this case which gives a huge performance boost for
workloads that write data and then immediately read the data they just
wrote.

From glusterfs --help,


      --auto-invalidation[=BOOL]   controls whether fuse-kernel can
                             auto-invalidate attribute, dentry and page-cache.
                             Disable this only if same files/directories are
                             not accessed across two different mounts
                             concurrently [default: "on"]


Details on how disabling auto-invalidation helped to reduce pgbench
init times can be found at [1]. Time taken for pgbench init of scale
8000 was 8340s. That will be an improvement of 86% (59280s vs 8340s)
with auto-invalidations turned off along with other
optimizations. Just disabling auto-invalidation contributed 56%
improvement by reducing the total time taken by 33260s.

[1] https://www.spinics.net/lists/gluster-devel/msg25907.html

Change-Id: I0ed730dba9064bd9c576ad1800170a21e100e1ce
Signed-off-by: Raghavendra Gowdappa 
updates: bz#1664934

core: make gf_thread_create() easier to use

2019-02-01T09:37:20+00:00

This patch creates a specific function to set the thread name using a
string format and a variable argument list, like printf().

This function is used to set the thread name from gf_thread_create(),
which now accepts a variable argument list to create the full name. It's
not necessary anymore to use a local array to build the name of the
thread. This is done automatically.

Change-Id: Idd8d01fd462c227359b96e98699f8c6d962dc17c
Updates: bz#1193929
Signed-off-by: Xavi Hernandez

core: move "dict is NULL" logs to DEBUG log level

2019-02-01T03:28:50+00:00

Too many logs get printed if dict_ref() and dict_unref() are passed NULL
pointer.

fixes: bz#1671213
Change-Id: I18afd849d64318f68baa7b549ee310dac0e1e786
Signed-off-by: Milind Changire

syncop: remove unnecessary call to gf_backtrace_save()

2019-01-31T15:41:30+00:00

A call to gf_backtrace_save() was done on each context switch of a
synctask. The backtrace is generated writing to the filesystem, so it
can have an important impact on latency.

The generated backtrace was not used anywhere, so it's been removed.

Change-Id: I399a93b932c5b6e981c696c72c3e1ef44710ba52
Updates: bz#1193929
Signed-off-by: Xavi Hernandez

core: heketi-cli is throwing error "target is busy"

2019-01-31T06:23:39+00:00

Problem: When rpc-transport-disconnect happens, server_connection_cleanup_flush_cbk() 
         is supposed to call rpc_transport_unref() after open-files on 
         that transport are flushed per transport.But open-fd-count is 
         maintained in bound_xl->fd_count, which can be incremented/decremented 
         cumulatively in server_connection_cleanup() by all transport 
         disconnect paths. So instead of rpc_transport_unref() happening 
         per transport, it ends up doing it only once after all the files 
         on all the transports for the brick are flushed leading to 
         rpc-leaks.

Solution: To avoid races maintain fd_cnt at client instead of maintaining
          on brick

Credits: Pranith Kumar Karampuri
Change-Id: I6e8ea37a61f82d9aefb227c5b3ab57a7a36850e6
fixes: bz#1668190
Signed-off-by: Mohit Agrawal

Multiple files: reduce work while under lock.

2019-01-29T09:27:22+00:00

Mostly, unlock before logging.
In some cases, moved different code that was not needed
to be under lock (for example, taking time, or malloc'ing)
to be executed before taking the lock.

Note: logging might be slightly less accurate in order, since it may
not be done now under the lock, so order of logs is racy. I think
it's a reasonable compromise.

Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul 

Change-Id: I2438710016afc9f4f62a176ef1a0d3ed793b4f89