glusterfs.git/xlators/performance, branch v3.10.4

perf/ioc: Fix race causing crash when accessing freed page

2017-05-31T08:05:17+00:00

ioc_inode_wakeup does not lock the ioc_inode for the duration
of the operation, leaving a window where ioc_prune could find
a NULL waitq and hence free the page which ioc_inode_wakeup later
tries to access.

Thanks to Mohit for the analysis.

credit: moagrawa@redhat.com

> BUG: 1456385
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17410
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
Change-Id: I54b064857e2694826d0c03b23f8014e3984a3330
BUG: 1457054
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17423
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Talur 
CentOS-regression: Gluster Build System

performance/read-ahead: prevent stale data being returned to application.

2017-05-12T19:58:07+00:00

Assume that fd is shared by two application threads/processes.

T0 read is triggered from app-thread t1 and read call passes through
   write-behind.
T1 app-thread t2 issues a write. The page on which read from t1 is
   waiting is marked stale
T2 write-behind caches write and indicates to application as write
   complete.
T3 app-thread t2 issues read to same region. Since, there is already a
   page for that region (created as part of read at T0), this read
   request waits on that page to be filled (though it is stale, which
   is a bug).
T4 read (triggered at T0) completes from brick (with write still
   pending). Now both read requests from t1 and t2 are served this data
   (though data is stale from app-thread t2's perspective - which is a
   bug)
T5 write is flushed to brick by write-behind.

Fix is to not to serve data from a stale page, but instead initiate a
fresh read to back-end.

>Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
>BUG: 1414242
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/7447
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Csaba Henk 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Zhou Zhengping 
>Reviewed-by: Amar Tumballi 

(cherry picked from commit 2ff39c5cbea6fbda0d7a442f55e6dc2a72efb171)
Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
BUG: 1449313
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/17222
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

dht: The xattrs sent in readdirp should be sent in opendir aswell

2017-04-27T10:49:47+00:00

As readdir-ahead can be loaded as a child of dht, dht has to specify
the xattrs it is intrested in, as part of opendir call itself.

>Reviewed-on: https://review.gluster.org/16902
>Smoke: Gluster Build System 
>Reviewed-by: Raghavendra G 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>(cherry picked from commit 0f71338e1d7c0b70f4fe3b19c68612fe730d9de2)

Change-Id: I012ef96cc143b0cef942df78aa7150d85ec38606
BUG: 1435942
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16947
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra G

reddir-ahead: Fix EOD propagation problem

2017-04-13T15:38:18+00:00

In readdirp fop, op_errno is overloaded to indicate the EOD detection.
If op_errno contains ENOENT, then it indicates that there are no
further entries pending read in the directory. Currently NFS uses the
ENOENT to identify the EOD.

Issue:
NFS clients issues a 4K buffer for readdirp, readdir-ahead converts it
to 128K buffer as its reading ahead. If there are 100 entries in the
bricks, 128K can get all 100 and store in readdir-ahead, but only 23
entries that can be fit in 4K will be sent to NFS. Since the whole
100 entries were read from brick, the op_errno is set to ENOENT, and
the op_errno is propagated as is when sent to NFS. Hence NFS client
in reading 23 entries thinks it reached EOD.

Solution:
Do not propogate ENOENT errno, unless all the entries are read
from the readdir ahead buffer.

> Reviewed-on: https://review.gluster.org/16953
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit 61f76f318faed395660f5bbcfe39616b39c158f0)

Change-Id: I4f173a77b21ab9e98ae35e291a45b8fc0cde65bd
BUG: 1439148
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17001
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

build: miscellaneous spelling fixes

2017-04-03T01:36:51+00:00

Debian builds detected spelling issues with GlusterFS 3.10.1. Instead of
carrying the patch in the Debian sources, let's include the fixes here
too.

Change-Id: I38db6adf142f7ec247bffd47aa1e6ff1a0c49e00
Reviewed-on-master: https://review.gluster.org/16973
Reported-by: Patrick Matthäi 
BUG: 1437854
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16974
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Kaleb KEITHLEY

glusterd, readdir-ahead: Fix backward incompatibility

2017-02-19T14:18:08+00:00

Backport of https://review.gluster.org/#/c/16657/

Issue:
Any opion is spcified in two places: In the options[] of xlator
itself and glusterd-volume-set.c. The default value of this option
can be specified in both the places. If its specified only in xlator
then the volfile generated will not have the option and default value,
it will be assigned during graph initialization.
With patch [1] the option rda-request-size was changed from INT to SIZET
type, and default was changed from 131072 to 128KB, but was specified
only in the readdir-ahead.c. Thus with this patch alone the volfile
entry for readdir-ahead looks like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    subvolumes patchy-read-ahead
end-volume

With patch [2], the default of option rda-request-size was specified
in glusterd-volume-set.c as well(as it was necessary fr parallel readdir).
With this patch the readdir entry in the volfile will look like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    option rda-cache-limit 10MB
    option rda-request-size 128KB
    option parallel-readdir off
    subvolumes patchy-read-ahead
end-volume


Now consider the server has both these patches and client doesn't.
Server will generate a volfile with entry:

The old clients which thought the option rda-request-size is of type
INT will now recieve the value 128KB which it willn't understand,
and hence fail the mount.

The issue is seen only with the combination of [1] and [2].

Solution:
Instead of specifying 128KB as default in glusterd we specify 131072
so that the old clients will interpret as INT and new ones as 128KB

Credits: Raghavendra G

> Reviewed-on: https://review.gluster.org/16657
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 

Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a
BUG: 1423412
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16658
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee

readdir-ahead: Remove unnecessary logging

2017-02-17T16:56:00+00:00

dict_get_int can return < 0 when key is not found is a valid case.
Hence no need to log.

> Reviewed-on: https://review.gluster.org/16654
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Jeff Darcy  

Change-Id: If0795b0f178adbb94b10efc563506993f7411962
BUG: 1423429
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16659
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

performance/decompounder: Have terminal value in options[]

2017-02-17T11:46:15+00:00

Absence of terminal values is leading to buffer-over-flow errors in
address sanitizer.

 >BUG: 1422152
 >Change-Id: I769c0e4b5bbb3ef2849b8d1097b9def522ae08d9
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: https://review.gluster.org/16615
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Jeff Darcy 
 >(cherry picked from commit fca31ae5faa15a02e64143b9efb7e19c3b907c2d)

Change-Id: Ifec6f5b96cfe4c26a039f88649284bcdad5321f6
BUG: 1423070
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/16652
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

md-cache: initialize mdc_counter.lock

2017-02-16T17:53:25+00:00

add missed LOCK_INIT to fix INCREMENT_ATOMIC on
conf->mdc_counter.lock when pthread_spin_* using

Cherry picked from commit 22f02d8f1dcdf176744ab1536cb23a5fcd291243:
> Change-Id: I680bd6f41e3b8a1852ed969bf6794cbf4c1ccdd4
> BUG: 1417913
> Signed-off-by: Vitaly Lipatov 
> Reviewed-on: https://review.gluster.org/16515
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Poornima G 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I680bd6f41e3b8a1852ed969bf6794cbf4c1ccdd4
BUG: 1417915
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16640
Smoke: Gluster Build System 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

performance/write-behind: access stub only if available during

2017-02-02T17:36:43+00:00

statedump

>Change-Id: Ia5dd718458a5e32138012f81f014d13fc6b28be2
>BUG: 1415115
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/16440
>Reviewed-by: N Balachandran 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 

Change-Id: Ia5dd718458a5e32138012f81f014d13fc6b28be2
BUG: 1418623
Signed-off-by: Raghavendra G 
(cherry picked from commit 85d7f1d1ee24ac400d4aa223478727643532693a)
Reviewed-on: https://review.gluster.org/16519
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan