glusterfs.git/tests/bugs/shard, branch v4.2dev

features/shard: Upon FSYNC from upper layers, wind fsync on all changed shards

2018-03-05T04:53:03+00:00

Change-Id: Ib74354f57a18569762ad45a51f182822a2537421
BUG: 1468483
Signed-off-by: Krutika Dhananjay

features/shard: Fix shard inode refcount when it's part of priv->lru_list.

2018-03-02T05:26:00+00:00

For as long as a shard's inode is in priv->lru_list, it should have a non-zero
ref-count. This patch achieves it by taking a ref on the inode when it
is added to lru list. When it's time for the inode to be evicted
from the lru list, a corresponding unref is done.

Change-Id: I289ffb41e7be5df7489c989bc1bbf53377433c86
BUG: 1468483
Signed-off-by: Krutika Dhananjay

features/shard: Change default shard-block-size to 64MB

2017-09-14T03:23:15+00:00

Change-Id: I55fa87e07136cff10b0d725ee24dd3151016e64e
BUG: 1489823
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/18243
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
Tested-by: Sunil Kumar Acharya 
CentOS-regression: Gluster Build System

features/shard: Return aggregated size in stbuf of LINK fop

2017-09-06T02:31:09+00:00

Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0
BUG: 1488546
Signed-off-by: Krutika Dhananjay 
Reviewed-on: https://review.gluster.org/18209
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

features/shard: Remove ctx from LRU in shard_forget

2017-06-30T05:05:15+00:00

Problem:
There is a race when the following two commands are executed on the mount in
parallel from two different terminals on a sharded volume,
which leads to use-after-free.

Terminal-1:
while true; do dd if=/dev/zero of=file1 bs=1M count=4; done

Terminal-2:
while true; do cat file1 > /dev/null; done

In the normal case this is the life-cycle of a shard-inode
1) Shard is added to LRU when it is first looked-up
2) For every operation on the shard it is moved up in LRU
3) When "unlink of the shard"/"LRU limit is hit" happens it is removed from LRU

But we are seeing a race where the inode stays in Shard LRU even after it is
forgotten which leads to Use-after-free and then some memory-corruptions.

These are the steps:
1) Shard is added to LRU when it is first looked-up
2) For every operation on the shard it is moved up in LRU

Reader-handler                                    Truncate-handler
1) Reader handler needs shard-x to be read.     1) Truncate has just deleted shard-x
2) In shard_common_resolve_shards(), it does
   inode_resolve() and that leads to
   a hit in LRU, so it is going to call
__shard_update_shards_inode_list() to move the
   inode to top of LRU
						2) shard-x gets unlinked from the itable
						   and inode_forget(inode, 0) is called
						   to make sure the inode can be purged
						   upon last unref
3) when __shard_update_shards_inode_list() is
   called it finds that the inode is not in LRU
   so it adds it back to the LRU-list

Both these operations complete and call inode_unref(shard-x) which leads to the inode
getting freed and forgotten, even when it is in Shard LRU list. When more inodes are
added to LRU, use-after-free will happen and it leads to undefined behaviors.

Fix:
I see that the inode can be removed from LRU even by the protocol layers like gfapi/gNFS
when LRU limit is reached. So it is better to add a check in shard_forget() to remove itself
from LRU list if it exists.

BUG: 1466037
Change-Id: Ia79c0c5c9d5febc56c41ddb12b5daf03e5281638
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17644
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Krutika Dhananjay

features/shard: Handle offset in appending writes

2017-05-27T16:00:50+00:00

When a file is opened with append, all writes are appended at the end of file
irrespective of the offset given in the write syscall. This needs to be
considered in shard size update function and also for choosing which shard to
write to.

At the moment shard piggybacks on queuing from write-behind
xlator for ordering of the operations. So if write-behind is disabled and
two parallel appending-writes come both of which can increase the file size
beyond shard-size the file will be corrupted.

BUG: 1455301
Change-Id: I9007e6a39098ab0b5d5386367bd07eb5f89cb09e
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17387
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: run many bricks within one glusterfsd process

2017-01-31T00:13:58+00:00

This patch adds support for multiple brick translator stacks running
in a single brick server process.  This reduces our per-brick memory usage by
approximately 3x, and our appetite for TCP ports even more.  It also creates
potential to avoid process/thread thrashing, and to improve QoS by scheduling
more carefully across the bricks, but realizing that potential will require
further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global option.  By
default it's off, and bricks are started in separate processes as before.  If
multiplexing is enabled, then *compatible* bricks (mostly those with the same
transport options) will be started in the same process.

Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
BUG: 1385758
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/14763
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

tests: Enable all gfapi test cases

2016-07-21T06:15:55+00:00

Change-Id: I32bfec4af91348d96dc3e81a9d5c9cad599f821b
Bug: 1358594
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14748
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Talur

features/shard: Don't modify readv size

2016-06-06T15:12:55+00:00

For o-direct reads application sends aligned size which needs to be
sent as is, otherwise o-direct writes where the file-size is not
aligned fails.

Change-Id: I097418ad92eda6c835d7352a3d2e53ea9d8e2424
BUG: 1342298
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14623
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

features/shard: Get hard-link-count in {unlink,rename}_cbk before deleting shards

2016-05-20T14:40:00+00:00

Change-Id: I0606b74f11f5412c4d9af44a6505635ed9022c15
BUG: 1335858
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14334
Reviewed-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System