glusterfs.git/libglusterfs, branch v3.10.8

glusterd: Gluster should keep PID file in correct location

2017-10-25T14:03:54+00:00

Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).

These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.

So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.

> Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
> BUG: 1258561
> Signed-off-by: Gaurav Kumar Garg 
> Signed-off-by: Saravanakumar Arumugam 
> Reviewed-on: https://review.gluster.org/13580
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> (Cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d)

BUG: 1491059
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
Signed-off-by: Mohit Agrawal

mount/fuse: Make event-history feature configurable

2017-10-02T12:41:57+00:00

... and disable it by default.

        Backport of:
        > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
        > Reviewed-on: https://review.gluster.org/18242
        > BUG: 1467614
        > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1495430
Signed-off-by: Krutika Dhananjay

afr: heal metadata in discover code path

2017-09-17T13:37:32+00:00

******************************************************
Backport of: https://review.gluster.org/18202
Also added loc_is_nameless() to libglusterfs since the patch that
introduced it in master was not backported to release-3.10.

Note: 18202 is a squash of  17850 and 18187 in master.
******************************************************

During graph switch, if fuse sends nameless (gfid) lookups, afr takes
the discover code path to serve it. If there are pending metadata heals,
they do not happen unless an inode refresh happens as a part of
discover (which is not guaranteed to happen always).

This patch fixes it by attempting metadata heal as a part of discover,
just like how it is done in lookup code path.

Change-Id: I87c493045b9225741cad173bf3f645848697032e
BUG: 1492010
Signed-off-by: Ravishankar N 
Reviewed-on: https://review.gluster.org/18304
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

cluster/ec: Node uuid xattr support update for EC

2017-09-13T06:41:52+00:00

Problem:
The change in EC to return list of node uuids for
GF_XATTR_NODE_UUID_KEY was causing problems with
geo-rep.

Fix:
This patch will allow to get the single node uuid
as it was doing before with the key
"GF_XATTR_NODE_UUID_KEY", and will also allow to get
the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve
the problem with geo-rep and any other features which
were depending on this.

>BUG: 1462790
>Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
>Signed-off-by: Sunil Kumar Acharya 
>Reviewed-on: https://review.gluster.org/17594
>Smoke: Gluster Build System 
>Reviewed-by: Xavier Hernandez 
>Reviewed-by: Pranith Kumar Karampuri 
>CentOS-regression: Gluster Build System 
>Signed-off-by: Sunil Kumar Acharya 

BUG: 1487647
Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
Signed-off-by: Sunil Kumar Acharya 
Reviewed-on: https://review.gluster.org/17667
Smoke: Gluster Build System 
Reviewed-by: Xavier Hernandez 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/dht: rebalance perf enhancement

2017-08-11T17:40:29+00:00

Problem: Throttle settings "normal" and "aggressive" for rebalance
did not have performance difference.

normal mode spawns $(no. of cores - 4)/2 threads and aggressive
spawns $(no. of cores - 4) threads. Though aggressive mode has twice
the number of threads compared to that of normal mode, there was no
performance gain when switched to aggressive mode from normal mode.

RCA:
During the course of debugging the above problem, we tried assigning
migration job to migration threads spawned by rebalance, rather than
synctasks(as there is more overhead associated to manage the task
queue and threads). This gave us a significant improvement over rebalance
under synctasks. This patch does not really gurantee that there will be a
clear performance difference between normal and aggressive mode, but this
patch certainly maximized the disk utilization for 1GBfiles run.

Results:

Test enviroment:
Gluster Config:
Number of Bricks: 2 (one brick per disk(RAID-6 12 disk))
Bricks:
Brick1: server1:/brick/test1/1
Brick2: server2:/brick/test1/1
Options Reconfigured:
performance.readdir-ahead: on
server.event-threads: 4
client.event-threads: 4

1000 files with 1GB each were created/renamed such that all files will have
server1 as cached and server2 as hashed, so that all files will be migrated.

Test machines had 24 cores each.

Results  with/without synctask based migration:
-----------------------------------------------

mode                    normal(10threads)          aggressive(20threads)

timetaken               0:55:30 (h:m:s)            0:56:3 (h:m:s)
withsynctask

timetaken
with migrator           0:38:3 (h:m:s)             0:23:41 (h:m:s)
threads

From above table it can be seen that, there is a clear 2x perf gain between
rebalance with synctask vs rebalance with migrator threads.

Additionally this patch modifies the code so that caller will have the exact error
number returned by dht_migrate_file(earlier the errno meaning was overloaded). This
will help avoiding scenarios where migration failure due to ENOENT, can result in
rebalance abort/failure.

> Change-Id: I8904e2fb147419d4a51c1267be11a08ffd52168e
> BUG: 1420166
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/16427
> Smoke: Gluster Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Signed-off-by: Susant Palai 

Change-Id: I8904e2fb147419d4a51c1267be11a08ffd52168e
BUG: 1473134
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17834
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

refcount: typecast function for calling on free

2017-08-11T11:22:43+00:00

All of the functions called to free the refcounted structure are doing a
typecast from (void*) to their own type taht is being free'd. This
really is not needed and the refcount interface is made a little simpler
without the requirement of typecasting.

With this small improvement in the API, all callers are updated too.

Cherry picked from commit f2ca301bd741e3e3f076cd3f72fcd377bcef2a1a:
> Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
> BUG: 1416889
> Signed-off-by: Niels de Vos 
> Reviewed-on: https://review.gluster.org/16471
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Xavier Hernandez 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 

Backport note: This patch makes it easier to backport changes that use
               gf_refcount_t. There is no functional change.

Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1
BUG: 1471870
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17913
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

fuse: memory leak fixes

2017-07-19T21:33:34+00:00

Fix fuse ctx memory leak in case an error occurs and the cleanup path
is different than usual. Also fix a memory leak in logging if
eh_save_history() fails.

Cherry picked from commit 5ee383fed9f6408d303aa539dda071275021f8e4:
> Change-Id: I7ec967c807b0ed91184e5b958be70702215c46c9
> BUG: 1470220
> Signed-off-by: Danny Couture 
> Reviewed-on: https://review.gluster.org/17759
> Reviewed-by: Niels de Vos 
> Smoke: Gluster Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Prashanth Pai 
> Reviewed-by: Amar Tumballi 
> Tested-by: Amar Tumballi 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I7ec967c807b0ed91184e5b958be70702215c46c9
BUG: 1471028
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/17758
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

core: assorted typos and spelling mistakes from Debian lintian

2017-07-01T15:58:24+00:00

Plus minor readability improvements.

Reported-by: pmatthaei@debian.org
master BUG: 1466785
master https://review.gluster.org/17660
release-3.11 BUG: 1466801
release-3.11 https://review.gluster.org/17661
Change-Id: I5393819a2fc9f240a19811143bb57b127df717cf
BUG: 1466852
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/17663
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur

cluster/afr: Returning single and list of node uuids from AFR

2017-07-01T15:55:31+00:00

Problem:
The change in afr to return list of node uuids was causing problems
with geo-rep.

Fix:
This patch will allow to get the single node uuid as it was doing
before with the key "GF_XATTR_NODE_UUID_KEY", and will also allow
to get the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve the problem with
geo-rep and any other feature which were depending on this.

> Change-Id: I09885dac6dfca127be94b708470c8c2941356f9a
> BUG: 1462790
> Signed-off-by: karthik-us 
> Reviewed-on: https://review.gluster.org/17576
> Smoke: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Kotresh HR 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit 475ec9928ef96b63a0bfa859a9ae68709275033c)

Change-Id: I5e741a48a426ee9a3cc69612051e0e9bcf33b500
BUG: 1464078
Signed-off-by: karthik-us 
Reviewed-on: https://review.gluster.org/17603
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

rpc: add options to manage socket keepalive lifespan

2017-06-20T04:58:25+00:00

Problem:
Default values for handling socket timeouts for brick responses are
insufficient for aggressive applications such as databases.

Solution:
Add 1:1 gluster options for keepalive, keepalive-idle,
keepalive-interval and keepalive-timeout as per the socket level options
available as per tcp(7) man page.

Default values for options are NOT agressive and continue to be values
which result in default timeout when only the keep alive option is
turned on.

These options are Linux specific and will not be applicable to the
*BSDs.

mainline:
> BUG: 1426059
> Signed-off-by: Milind Changire 
> Reviewed-on: https://review.gluster.org/16731
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit 6b8df081b46ac4f485c86a5052fc30472e74bfbb)

Change-Id: I2a08ecd949ca8ceb3e090d336ad634341e2dbf14
BUG: 1452038
Signed-off-by: Milind Changire 
Reviewed-on: https://review.gluster.org/17330
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur