Problem: commit c34e4161f3cb6539ec83a9020f3d27eb4759a975 set log-level
         per xlator during reconfigure only for a brick process not for
         the client process.

Solution: 1) Change per xlator log-level only if brick_mux is enabled.To make sure
             about brick multiplex introudce a flag brick_mux at ctx->cmd_args.

Note: There are two other changes done with this patch
      1) Ignore client-log-level option to attach a brick with
         already running brick if brick_mux is enabled
      2) Add a log to print pid of the running process to make easier
         debugging

Change-Id: I39e85de778e150d0685cd9a79425ce8b4783f9c9
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
Fixes: bz#1696046

This patch implements a thread pool that is wait-free for adding jobs to
the queue and uses a very small locked region to get jobs. This makes it
possible to decrease contention drastically. It's based on wfcqueue
structure provided by urcu library.

It automatically enables more threads when load demands it, and stops
them when not needed. There's a maximum number of threads that can be
used. This value can be configured.

Depending on the workload, the maximum number of threads plays an
important role. So it needs to be configured for optimal performance.
Currently the thread pool doesn't self adjust the maximum for the
workload, so this configuration needs to be changed manually.

For this reason, the global thread pool has been made optional, so that
volumes can still use the thread pool provided by io-threads.

To enable it for bricks, the following option needs to be set:

   config.global-threading = on

This option has no effect if bricks are already running. A restart is
required to activate it. It's recommended to also enable the following
option when running bricks with the global thread pool:

   performance.iot-pass-through = on

To enable it for a FUSE mount point, the option '--global-threading'
must be added to the mount command. To change it, an umount and remount
is needed. It's recommended to disable the following option when using
global threading on a mount point:

   performance.client-io-threads = off

To enable it for services managed by glusterd, glusterd needs to be
started with option '--global-threading'. In this case all daemons, like
self-heal, will be using the global thread pool.

Currently it can only be enabled for bricks, FUSE mounts and glusterd
services.

The maximum number of threads for clients and bricks can be configured
using the following options:

   config.client-threads
   config.brick-threads

These options can be applied online and its effect is immediate most of
the times. If one of them is set to 0, the maximum number of threads
will be calcutated as #cores * 2.

Some distributions use a very old userspace-rcu library (version 0.7)
for this reason, some header files from version 0.10 have been copied
into contrib/userspace-rcu and are used if the detected version is 0.7
or older.

An additional change has been made to io-threads to prevent that threads
are started when iot-pass-through is set.

Change-Id: I09d19e246b9e6d53c6247b29dfca6af6ee00a24b
updates: #532
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>

updates bz#1650403
Change-Id: Ib5a11e691599ce4bd93c1ed5aca6060592893961
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>

    * libglusterfs changes to add new fop

    * Fuse changes:
      - Changes in fuse bridge xlator to receive and send responses

    * posix changes to perform the op on the backend filesystem

    * protocol and rpc changes for sending and receiving the fop

    * gfapi changes for performing the fop

    * tools: glfs-copy-file-range tool for testing copy_file_range fop

      - Although, copy_file_range support has been added to the upstream
	    fuse kernel module, no release has been made yet of a kernel
        which contains the support. It is expected to come in the
        upcoming release of linux-4.20

        So, as of now, executing copy_file_range fop on a fused based
        filesystem results in fuse kernel module sending read on the
	    source fd and write on the destination fd.

	    Therefore a small gfapi based tool has been written to be able
        test the copy_file_range fop. This tool is similar (in functionality)
	    to the example program given in copy_file_range man page.

	    So, running regular copy_file_range on a fuse mount point and
	    running gfapi based glfs-copy-file-range tool gives some idea about
	    how fast, the copy_file_range (or reflink) can be.

	    On the local machine this was the result obtained.

	    mount -t glusterfs workstation:new /mnt/glusterfs
	    [root@workstation ~]# cd /mnt/glusterfs/
	    [root@workstation glusterfs]# ls
	    file
	    [root@workstation glusterfs]# cd
	    [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new
	    real  0m6.495s
	    user  0m0.000s
	    sys   0m1.439s
	    [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr
	    OPEN_SRC: opening /file is success
	    OPEN_DST: opening /rrr is success
	    FSTAT_SRC: fstat on /rrr is success
	    copy_file_range successful

        real  0m0.309s
        user  0m0.039s
        sys   0m0.017s

        This tool needs following arguments
         1) hostname
         2) volume name
         3) log file path
         4) source file path (relative to the gluster volume root)
         5) destination file path (relative to the gluster volume root)

        "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>"

      - Added a testcase as well to run glfs-copy-file-range tool

    * io-stats changes to capture the fop for profiling

    * NOTE:

      - Added conditional check to see whether the copy_file_range syscall
        is available or not. If not, then return ENOSYS.

      - Added conditional check for kernel minor version in fuse_kernel.h
        and fuse-bridge while referring to copy_file_range. And the kernel
        minor version is kept as it is. i.e. 24. Increment it in future
        when there is a kernel release which contains the support for
        copy_file_range fop in fuse kernel module.

    * The document which contains a writeup on this enhancement can be found at
      https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit

Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367
updates: #536
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>

Fixes: #164
Change-Id: I93ad6f0232a1dc534df099059f69951e1339086f
Signed-off-by: Amar Tumballi <amarts@redhat.com>

libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>

Summary: The json being output by the io-stats debug xlator
quotes the numbers. This is not necessary and makes parsing
in strongly typed languages more difficult.

Change-Id: I3ac13700e2c52dbdc29d0bcdd39896d7871f36fe
fixes: bz#1654521
Signed-off-by: Chris Holcombe <xfactor973@gmail.com>

The issue pertains to checking if conf is NULL in BUMP_FOP
but not providing that safety in io_stats_release when using
conf to lock and bump nr_opens.

This is now corrected to check if conf is non-NULL before
attempting the lock and bump nr_opens.

Tested with local clang analyzer to ensure this fixes the
problem.

Change-Id: Iffd6a97c2060d0a6930a8dc5914b1956c192cab1
Updates: bz#1622665
Signed-off-by: ShyamsundarR <srangana@redhat.com>

By allowing clients taking dump in a file on brick process, we are
allowing compromised clients to create io-stats dumps on server,
which can exhaust all the available inodes.

Fixes: CVE-2018-14659

Fixes: bz#1644757
Change-Id: I32bfde9d4fe646d819a45e627805b928cae2e1ca
Signed-off-by: Amar Tumballi <amarts@redhat.com>

Problem: In brick_mux environment, while a user has enabled brick-log-level
         for anyone volume, it automatically enables for other volumes
         also those are attached with same brick.

Solution: A log-level option is automatically enabled for other volumes
          because log-level saved in glusterfsd_ctx and ctx is common for
          volumes those are attached with same brick. To resolve it
          set log level for all children xlator's at the time of the graph
          reconfigure at io-stat xlator.

Change-Id: Id9a6efa05d286e0bea2d47f49292d084e7bb2fcf
fixes: bz#1640495
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>