glusterfs.git/xlators/protocol/server/src, branch experimental

posix: stack-buffer-overflow reported by asan

2018-12-26T02:17:16+00:00

This patch fixes buffer overflow in
$SRC/xlators/storage/posix/src/posix-inode-fd-ops.c
Memory access at offset 432 overflows "md5_checksum" variable.
SUMMARY: AddressSanitizer: stack-buffer-overflow (/lib64/libasan.so.5+0xb825a)

updates: bz#1633930

Change-Id: I46010a09161d02cdf0c69679a334ec1d3d49cffb
Signed-off-by: Harpreet Kaur

copy_file_range support in GlusterFS

2018-12-12T15:56:55+00:00

    * libglusterfs changes to add new fop

    * Fuse changes:
      - Changes in fuse bridge xlator to receive and send responses

    * posix changes to perform the op on the backend filesystem

    * protocol and rpc changes for sending and receiving the fop

    * gfapi changes for performing the fop

    * tools: glfs-copy-file-range tool for testing copy_file_range fop

      - Although, copy_file_range support has been added to the upstream
	    fuse kernel module, no release has been made yet of a kernel
        which contains the support. It is expected to come in the
        upcoming release of linux-4.20

        So, as of now, executing copy_file_range fop on a fused based
        filesystem results in fuse kernel module sending read on the
	    source fd and write on the destination fd.

	    Therefore a small gfapi based tool has been written to be able
        test the copy_file_range fop. This tool is similar (in functionality)
	    to the example program given in copy_file_range man page.

	    So, running regular copy_file_range on a fuse mount point and
	    running gfapi based glfs-copy-file-range tool gives some idea about
	    how fast, the copy_file_range (or reflink) can be.

	    On the local machine this was the result obtained.

	    mount -t glusterfs workstation:new /mnt/glusterfs
	    [root@workstation ~]# cd /mnt/glusterfs/
	    [root@workstation glusterfs]# ls
	    file
	    [root@workstation glusterfs]# cd
	    [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new
	    real  0m6.495s
	    user  0m0.000s
	    sys   0m1.439s
	    [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr
	    OPEN_SRC: opening /file is success
	    OPEN_DST: opening /rrr is success
	    FSTAT_SRC: fstat on /rrr is success
	    copy_file_range successful

        real  0m0.309s
        user  0m0.039s
        sys   0m0.017s

        This tool needs following arguments
         1) hostname
         2) volume name
         3) log file path
         4) source file path (relative to the gluster volume root)
         5) destination file path (relative to the gluster volume root)

        "glfs-copy-file-range     "

      - Added a testcase as well to run glfs-copy-file-range tool

    * io-stats changes to capture the fop for profiling

    * NOTE:

      - Added conditional check to see whether the copy_file_range syscall
        is available or not. If not, then return ENOSYS.

      - Added conditional check for kernel minor version in fuse_kernel.h
        and fuse-bridge while referring to copy_file_range. And the kernel
        minor version is kept as it is. i.e. 24. Increment it in future
        when there is a kernel release which contains the support for
        copy_file_range fop in fuse kernel module.

    * The document which contains a writeup on this enhancement can be found at
      https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit

Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367
updates: #536
Signed-off-by: Raghavendra Bhat

libglusterfs: Move devel headers under glusterfs directory

2018-12-05T21:47:04+00:00

libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR

protocol/server: support server.all-squash

2018-12-05T21:45:49+00:00

We still use gnfs on our side, so do a little work to support
server.all-squash. Just like server.root-squash, it's also a
volume wide option. Also see bz#1285126

$ gluster volume set  server.all-squash on

Note: If you enable server.root-squash and server.all-squash
at the same time, only server.all-squash works. Please refer
to following table

+---------------+-----------------+---------------------------+
|               |all_squash       | no_all_squash             |
+-------------------------------------------------------------+
|               |                 |anonuid/anongid for root   |
|root_squash    |anonuid/anongid  |useruid/usergid for no-root|
+-------------------------------------------------------------+
|no_root_squash |anonuid/anongid  |useruid/usergid            |
+-------------------------------------------------------------+

Updates bz#1285126
Signed-off-by: Xie Changlong 
Signed-off-by: Xue Chuanyu 
Change-Id: Iea043318fe6e9a75fa92b396737985062a26b47e

rpc: bump up server.event-threads

2018-12-04T06:38:55+00:00

Problem:
A single event-thread causes performance issues in the system.

Solution:
Bump up event-threads to 2 to make the system more performant.
This helps in making the system more responsive and helps avoid the
ping-timer-expiry problem as well. However, setting the event-threads
to 2 is not the only thing required to avoid ping-timer-expiry issues.

Change-Id: Idb0fd49e078db3bd5085dd083b0cdc77b59ddb00
fixes: bz#1653277
Signed-off-by: Milind Changire

server: Resolve memory leak path in server_init

2018-12-03T11:34:35+00:00

Problem: 1) server_init does not cleanup allocate resources
            while it is failed before return error
         2) dict leak at the time of graph destroying

Solution: 1) free resources in case of server_init is failed
          2) Take dict_ref of graph xlator before destroying
             the graph to avoid leak

Change-Id: I9e31e156b9ed6bebe622745a8be0e470774e3d15
fixes: bz#1654917
Signed-off-by: Mohit Agrawal

Multiple xlator .h files: remove unused private gf_* memory types.

2018-11-30T11:51:18+00:00

It seems there were quite a few unused enums (that in turn
cause unndeeded memory allocation) in some xlators.
I've removed them, hopefully not causing any damage.

Compile-tested only!

updates: bz#1193929
Signed-off-by: Yaniv Kaul 

Change-Id: I8252bd763dc1506e2d922496d896cd2fc0886ea7

rpcsvc: provide each request handler thread its own queue

2018-11-29T01:19:12+00:00

A single global per program queue is contended by all request handler
threads and event threads. This can lead to high contention. So,
reduce the contention by providing each request handler thread its own
private queue.

Thanks to "Manoj Pillai" for the idea of pairing a
single queue with a fixed request-handler-thread and event-thread,
which brought down the performance regression due to overhead of
queuing significantly.

Thanks to "Xavi Hernandez" for discussion on
how to communicate the event-thread death to request-handler-thread.

Thanks to "Karan Sandha" for voluntarily running
the perf benchmarks to qualify that performance regression introduced
by ping-timer-fixes is fixed with this patch and patiently running
many iterations of regression tests while RCAing the issue.

Thanks to "Milind Changire" for patiently running
the many iterations of perf benchmarking tests while RCAing the
regression caused by ping-timer-expiry fixes.

Change-Id: I578c3fc67713f4234bd3abbec5d3fbba19059ea5
Fixes: bz#1644629
Signed-off-by: Raghavendra Gowdappa

libglusterfs: rename macros roof and floor to not conflict with math.h

2018-11-28T15:11:59+00:00

Change-Id: I666eeb63ebd000711b3f793b948d4e0c04b1a242
Signed-off-by: Raghavendra Gowdappa 
Updates: bz#1644629

core: create a constant for default network timeout

2018-11-23T10:05:02+00:00

A new constant named GF_NETWORK_TIMEOUT has been defined and all
references to the hard-coded timeout of 42 seconds have been
replaced with this constant.

Change-Id: Id30f5ce4f1230f9288d9e300538624bcf1a6da27
fixes: bz#1652852
Signed-off-by: Xavi Hernandez