glusterfs.git/xlators/storage, branch v3.7.13

libglusterfs: Implement API that provides page-aligned iobufs

2016-06-30T04:37:08+00:00

        Backport of: http://review.gluster.org/14672

One of the consumers of a page aligned buffer would be posix's
readv fop on O_DIRECT fds. Today the way it works is by getting
a page-aligned buffer through calloc, pread()ing into this buffer
and then copying its contents into a newly created iobuf's ptr.
This results in an extra memcpy() which can be avoided if we could
implement an api that would return an iobuf whose ptr is
page-aligned. That way the iobuf->ptr can be directly passed to
sys_pread() as a parameter by posix translator.

Change-Id: Ie44c300dc773fef8e1669d609987d47dbd340ac2
BUG: 1351024
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14826
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Pranith Kumar Karampuri

cluster/dht: Fix unsafe iteration on inode->fd_list

2016-06-24T10:20:05+00:00

When DHT traverses the inode->fd_list, it does that in an unsafe
way that can generate races with fd_unref() called from other threads.

This patch fixes this problem taking the inode->lock and adding a
reference to the fd while it's being used outside of the mutex
protected region.

A minor change in storage/posix has been done to also access the
inode->fd_list in a safe way.

Backport of:
> Change-Id: I10d469ca6a8f76e950a8c9779ae9c8b70f88ef93
> BUG: 1344340
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: http://review.gluster.org/14682
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I10d469ca6a8f76e950a8c9779ae9c8b70f88ef93
BUG: 1346751
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/14734
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

dht/afr/client/posix: Fail mkdir without gfid-req

2016-06-17T06:52:55+00:00

Do not allow directory creations without gfids as
after the directories are created, operations
on them fail anyway. So it is better to fail mkdir.

 >BUG: 1317361
 >Change-Id: I8f8e3b38bbded1960b7215bac0432500f7e78038
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/13690
 >Smoke: Gluster Build System 
 >Reviewed-by: Krutika Dhananjay 
 >CentOS-regression: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >(cherry picked from commit b246b07896fefb261c9fb07f3f29f0d03b81b88d)

Change-Id: Ibf9c84add7265e3e1755a37958e1de38307624b2
BUG: 1332372
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14188
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core, shard: Make shards inherit main file's O_DIRECT flag if present

2016-06-16T16:04:10+00:00

        Backport of: http://review.gluster.org/14191

If the application opens a file with O_DIRECT, the shards'
anon fds would also need to inherit the flag. Towards this,
shard xl would be passing the odirect flag in the @flags parameter
to the WRITEV fop. This will be used in anon fd resolution
and subsequent opening by posix xl.

Change-Id: I3a0593fa46cc25e390a5762a0354b469c2a1532d
BUG: 1342903
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14663
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

posix, shard: Use page-aligned buffer for o-direct reads

2016-06-15T11:15:52+00:00

        Backport of: http://review.gluster.org/#/c/14639/

and also make shard_readv_do() pass the correct flags when
the original fd is opened with O_DIRECT.

Change-Id: I253f74459656b047de652b6aace487c0e2ac71f6
BUG: 1342903
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14650
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Even anonymous fds must have fd->flags set

2016-06-08T09:44:08+00:00

        Backport of: http://review.gluster.org/10219

We do not set the same flags to anonymous fd that posix uses to open
an anonymous fd in the backend. If there are any xlators which rely on
these flags for their operation they may not work well.

Add proper flags to anonymous fds at the time of their creation and
refer to them for subsequent operations.

Change-Id: I60d5b04ab851c31fbe4e5e74bee1af1b14daa52e
BUG: 1342903
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/14662
Tested-by: Krutika Dhananjay 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

storage/posix: Print offset,size and gfid too when readv fails

2016-06-04T02:00:21+00:00

... for better debuggability.

        Backport of: http://review.gluster.org/14271

Here's a sample log:

[2016-05-31 05:18:34.893951] E [MSGID: 113040] [posix.c:3089:posix_readv]
0-dis-posix: read failed on gfid=3744d34b-f798-4ce2-b421-cc632a9903bd,
fd=0x7f8b3c015abc, offset=131072 size=131072 [Invalid argument]

Change-Id: Ied15223b25dbde9ffc40af5012d96164a32fb7c3
BUG: 1342348
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14625
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System

features/shard: Get hard-link-count in {unlink,rename}_cbk before deleting shards

2016-05-24T06:08:01+00:00

        Backport of http://review.gluster.org/#/c/14334/

Change-Id: I41321d8b00a10f1bd5b0a7b008f673b1aa240d0c
BUG: 1337837
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14450
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

posix: Set correct d_type for readdirp() calls

2016-05-11T11:09:37+00:00

dirent.d_type can contain the type of the directory entry. The 'd_type'
struct member in dirent is present in Linux and many BSD flavours.
However, filling d_type with correct value requires support from the
underlying filesystem. If not, d_type is set to DT_UNKNOWN. XFS added
support for d_type as part of their newer version 5 on-disk format.
However, this requires Linux >= 3.15, xfsprogs >= 3.2.0 and the bricks
to be formatted using the new format.

This patch enables posix xlator to set d_type to the right value even
when the underlying filesystem does not support it. d_type can be set
using information previously fetched by stat() on the dir entry.
This will aid FUSE applications to leverage d_type to avoid the expense
of calling lstat() if further actions depend on the type of the file.

Refer `man 3 readdir` and `man 2 getdents`

> Change-Id: Ic5a262fe4c64122726b4fae2d1bea375c559ca04
> Signed-off-by: Prashanth Pai 
> Reviewed-on: http://review.gluster.org/14095
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit 77def44d497d090ef3f393b6d9403c1a29dcf993)

Change-Id: I8de1e643dbe88c57eb7a946357283f46c30ae701
BUG: 1332397
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/14176
Smoke: Gluster Build System 
Tested-by: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

dht: add "nuke" functionality for efficient server-side deletion

2016-04-26T08:21:47+00:00

This is a backport of the following two patches (of which the second is a
trivial adjustment to a timeout for a test added by the first).

	http://review.gluster.org/13878
	http://review.gluster.org/13935

This turns a special xattr into an rmdir with flags set.  When that hits
the posix translator on the server side, that causes the file/directory
to be moved into the special "landfill" directory.  From there, the
posix janitor thread will take care of deleting it entirely on the
server side - traversing it recursively if necessary.  A couple of
secondary issues were fixed to make this effective.

 * FUSE now ensures that setxattr values are NUL terminated.

 * The janitor thread now gets woken up immediately when something is
   placed in 'landfill' instead of only when file descriptors need to be
   closed.

 * The default landfill-emptying interval was reduced to 10s.

To use the feature, issue a setxattr something like this:

   setfattr -n glusterfs.dht.nuke -v "" /mnt/glusterfs/vol/some_dir

The value doesn't actually matter; the mere receipt of a request with
this key is sufficient.  Some day it might be useful to allow setting a
required value as a sort of password, so that only those who know it can
access the underlying special functionality.

Change-Id: I4132a30d1faa53a6682399ad1d9041e2c4519951
BUG: 1330241
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/14065
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G