glusterfs.git/xlators/performance, branch v3.10.9

performance/write-behind: fix bug while handling short writes

2017-12-26T10:40:19+00:00

The variabled "fulfilled" in wb_fulfill_short_write is not reset to 0
while handling every member of the list.

This has some interesting consequences:

* If we break from the loop while processing last member of the list
  head->winds, req is reset to head as the list is a circular
  one. However, head is already fulfilled and can potentially be
  freed. So, we end up adding a freed request to wb_inode->todo
  list. This is the RCA for the crash tracked by the bug associated
  with this patch (Note that we saw "holder" which is freed in todo
  list).

* If we break from the loop while processing any of the last but one
  member of the list head->winds, req is set to next member in the
  list, skipping the current request, even though it is not entirely
  synced. This can lead to data corruption.

The fix is very simple and we've to change the code to make sure
"fulfilled" reflects whether the current request is fulfilled or not
and it doesn't carry history of previous requests in the list.

>Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8
>BUG: 1528558
>Signed-off-by: Raghavendra G 

(cherry picked from commit 0bc22bef7f3c24663aadfb3548b348aa121e3047)
Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8
BUG: 1529096
Signed-off-by: Raghavendra G

md-cache: Use correct xattr keynames for virtual glusterfs ACLs.

2017-10-25T14:05:54+00:00

The "glusterfs.posix_acl." prefix does not catch the glusterfs posix acl
xattr keynames which are

	* "glusterfs.posix.acl" and
	* "glusterfs.posix.default_acl"

Using the GF_POSIX_ACL_ACCESS and GF_POSIX_ACL_DEFAULT defines directly
is the savest option.

Guenther

> Change-Id: I5aba64b26b6cbec850ea02316dd9f069400e857f
> BUG: 1476295
> Signed-off-by: Guenther Deschner 
> Reviewed-on: https://review.gluster.org/17909
> Reviewed-by: Michael Adam 
> Smoke: Gluster Build System 
> Reviewed-by: Niels de Vos 
> Tested-by: Niels de Vos 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit 5fe8555800cbc9818e7c976f63499795a378cd8d)
> Signed-off-by: Günther Deschner 

Change-Id: I5aba64b26b6cbec850ea02316dd9f069400e857f
BUG: 1499890
Signed-off-by: Günther Deschner

md-cache: avoid checking the xattr value buffer with string functions.

2017-10-25T14:00:03+00:00

xattrs may very well contain binary, non-text data with leading 0
values. Using strcmp for checking empty values is not the appropriate
thing to do: In the best case, it might treat a binary xattr value
starting with 0 from being cached (and hence also from being reported
back with xattr). In the worst case, we might read beyond the end
of a data blob that does contain any zero byte.

We fix this by checking the length of the data blob and checking
the first byte against 0 if the length is one.

> Signed-off-by: Guenther Deschner 
> Pair-Programmed-With: Michael Adam 
> Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
> BUG: 1476324
> Reviewed-on: https://review.gluster.org/17910
> Reviewed-by: Michael Adam 
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> Tested-by: Poornima G 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit ab4ffdac9dec1867f2d9b33242179cf2b347319d)

Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11
BUG: 1499893
Signed-off-by: Günther Deschner

performance/io-cache: update inode contexts of each entry in readdirplus

2017-08-11T11:09:11+00:00

io-cache stores read-cache in inode which is currently created only in
lookup. But, with readdirplus and md-cache absorbing lookups, io-cache
need not receive a lookup before a fop like readv.

>Change-Id: I6eba995b0a90d4d5055a4aef0489707b852da1b8
>BUG: 1474180
>Signed-off-by: Raghavendra G 
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/5029
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 

(cherry picked from commit b90e12134af85635199750967c326761d6c06e86)
Change-Id: I6eba995b0a90d4d5055a4aef0489707b852da1b8
BUG: 1475638
Signed-off-by: Raghavendra G 
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/17891
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

perf/ioc: Fix race causing crash when accessing freed page

2017-05-31T08:05:17+00:00

ioc_inode_wakeup does not lock the ioc_inode for the duration
of the operation, leaving a window where ioc_prune could find
a NULL waitq and hence free the page which ioc_inode_wakeup later
tries to access.

Thanks to Mohit for the analysis.

credit: moagrawa@redhat.com

> BUG: 1456385
> Signed-off-by: N Balachandran 
> Reviewed-on: https://review.gluster.org/17410
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 
Change-Id: I54b064857e2694826d0c03b23f8014e3984a3330
BUG: 1457054
Signed-off-by: N Balachandran 
Reviewed-on: https://review.gluster.org/17423
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Talur 
CentOS-regression: Gluster Build System

performance/read-ahead: prevent stale data being returned to application.

2017-05-12T19:58:07+00:00

Assume that fd is shared by two application threads/processes.

T0 read is triggered from app-thread t1 and read call passes through
   write-behind.
T1 app-thread t2 issues a write. The page on which read from t1 is
   waiting is marked stale
T2 write-behind caches write and indicates to application as write
   complete.
T3 app-thread t2 issues read to same region. Since, there is already a
   page for that region (created as part of read at T0), this read
   request waits on that page to be filled (though it is stale, which
   is a bug).
T4 read (triggered at T0) completes from brick (with write still
   pending). Now both read requests from t1 and t2 are served this data
   (though data is stale from app-thread t2's perspective - which is a
   bug)
T5 write is flushed to brick by write-behind.

Fix is to not to serve data from a stale page, but instead initiate a
fresh read to back-end.

>Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
>BUG: 1414242
>Signed-off-by: Raghavendra G 
>Reviewed-on: https://review.gluster.org/7447
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Csaba Henk 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Zhou Zhengping 
>Reviewed-by: Amar Tumballi 

(cherry picked from commit 2ff39c5cbea6fbda0d7a442f55e6dc2a72efb171)
Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
BUG: 1449313
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/17222
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

dht: The xattrs sent in readdirp should be sent in opendir aswell

2017-04-27T10:49:47+00:00

As readdir-ahead can be loaded as a child of dht, dht has to specify
the xattrs it is intrested in, as part of opendir call itself.

>Reviewed-on: https://review.gluster.org/16902
>Smoke: Gluster Build System 
>Reviewed-by: Raghavendra G 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>(cherry picked from commit 0f71338e1d7c0b70f4fe3b19c68612fe730d9de2)

Change-Id: I012ef96cc143b0cef942df78aa7150d85ec38606
BUG: 1435942
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16947
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra G

reddir-ahead: Fix EOD propagation problem

2017-04-13T15:38:18+00:00

In readdirp fop, op_errno is overloaded to indicate the EOD detection.
If op_errno contains ENOENT, then it indicates that there are no
further entries pending read in the directory. Currently NFS uses the
ENOENT to identify the EOD.

Issue:
NFS clients issues a 4K buffer for readdirp, readdir-ahead converts it
to 128K buffer as its reading ahead. If there are 100 entries in the
bricks, 128K can get all 100 and store in readdir-ahead, but only 23
entries that can be fit in 4K will be sent to NFS. Since the whole
100 entries were read from brick, the op_errno is set to ENOENT, and
the op_errno is propagated as is when sent to NFS. Hence NFS client
in reading 23 entries thinks it reached EOD.

Solution:
Do not propogate ENOENT errno, unless all the entries are read
from the readdir ahead buffer.

> Reviewed-on: https://review.gluster.org/16953
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> (cherry picked from commit 61f76f318faed395660f5bbcfe39616b39c158f0)

Change-Id: I4f173a77b21ab9e98ae35e291a45b8fc0cde65bd
BUG: 1439148
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17001
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

build: miscellaneous spelling fixes

2017-04-03T01:36:51+00:00

Debian builds detected spelling issues with GlusterFS 3.10.1. Instead of
carrying the patch in the Debian sources, let's include the fixes here
too.

Change-Id: I38db6adf142f7ec247bffd47aa1e6ff1a0c49e00
Reviewed-on-master: https://review.gluster.org/16973
Reported-by: Patrick Matthäi 
BUG: 1437854
Signed-off-by: Niels de Vos 
Reviewed-on: https://review.gluster.org/16974
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Kaleb KEITHLEY

glusterd, readdir-ahead: Fix backward incompatibility

2017-02-19T14:18:08+00:00

Backport of https://review.gluster.org/#/c/16657/

Issue:
Any opion is spcified in two places: In the options[] of xlator
itself and glusterd-volume-set.c. The default value of this option
can be specified in both the places. If its specified only in xlator
then the volfile generated will not have the option and default value,
it will be assigned during graph initialization.
With patch [1] the option rda-request-size was changed from INT to SIZET
type, and default was changed from 131072 to 128KB, but was specified
only in the readdir-ahead.c. Thus with this patch alone the volfile
entry for readdir-ahead looks like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    subvolumes patchy-read-ahead
end-volume

With patch [2], the default of option rda-request-size was specified
in glusterd-volume-set.c as well(as it was necessary fr parallel readdir).
With this patch the readdir entry in the volfile will look like:
volume patchy-readdir-ahead
    type performance/readdir-ahead
    option rda-cache-limit 10MB
    option rda-request-size 128KB
    option parallel-readdir off
    subvolumes patchy-read-ahead
end-volume


Now consider the server has both these patches and client doesn't.
Server will generate a volfile with entry:

The old clients which thought the option rda-request-size is of type
INT will now recieve the value 128KB which it willn't understand,
and hence fail the mount.

The issue is seen only with the combination of [1] and [2].

Solution:
Instead of specifying 128KB as default in glusterd we specify 131072
so that the old clients will interpret as INT and new ones as 128KB

Credits: Raghavendra G

> Reviewed-on: https://review.gluster.org/16657
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Shyamsundar Ranganathan 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Atin Mukherjee 

Change-Id: I0c269a5890957fd8a38e9a05bdec088645a6688a
BUG: 1423412
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/16658
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee