glusterfs.git/libglusterfs/src, branch v3.8.1

libglusterfs: Implement API that provides page-aligned iobufs

2016-06-29T09:53:40+00:00

        Backport of: http://review.gluster.org/14672

One of the consumers of a page aligned buffer would be posix's
readv fop on O_DIRECT fds. Today the way it works is by getting
a page-aligned buffer through calloc, pread()ing into this buffer
and then copying its contents into a newly created iobuf's ptr.
This results in an extra memcpy() which can be avoided if we could
implement an api that would return an iobuf whose ptr is
page-aligned. That way the iobuf->ptr can be directly passed to
sys_pread() as a parameter by posix translator.

Change-Id: I7f64419461db387e4ab6b0e03c90471744aa56e1
BUG: 1351025
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14825
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Negate all but O_DIRECT flag if present on anon fds

2016-06-28T05:57:11+00:00

        Backport of: http://review.gluster.org/14665

This is to prevent any unforeseen problems that might arise due to
writevs and readvs being wound with @flag parameter containing
O_TRUNC or O_APPEND especially wrt translators like sharding and ec
where O_TRUNC write or O_APPEND write on individual shards/fragments
is not the same as O_TRUNC write or O_APPEND write as expected by the
application.

 >Change-Id: Ib0110731d6099bc888b7ada552a2abbea8e76051
 >BUG: 1342903
 >Signed-off-by: Krutika Dhananjay 
 >Reviewed-on: http://review.gluster.org/14735
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Pranith Kumar Karampuri 
 >CentOS-regression: Gluster Build System 

Change-Id: I7ffa4fa366f727f7e345ab0bf4c8eb009710074b
BUG: 1347553
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14755
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core, shard: Make shards inherit main file's O_DIRECT flag if present

2016-06-27T15:17:55+00:00

        Backport of: http://review.gluster.org/14191

If the application opens a file with O_DIRECT, the shards'
anon fds would also need to inherit the flag. Towards this,
shard xl would be passing the odirect flag in the @flags parameter
to the WRITEV fop. This will be used in anon fd resolution
and subsequent opening by posix xl.

 >Change-Id: I3a0593fa46cc25e390a5762a0354b469c2a1532d
 >BUG: 1342903
 >Signed-off-by: Krutika Dhananjay 
 >Reviewed-on: http://review.gluster.org/14663
 >Smoke: Gluster Build System 
 >CentOS-regression: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Jeff Darcy 

Change-Id: Ibfc164aa7f9eecd6993255f1c03557f2ec35ac8c
BUG: 1347553
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14754
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Fix invalid __fd_unref() call

2016-06-13T10:16:17+00:00

__fd_unref() doesn't do any cleanup, so it cannot be called to release
fd references, specially if it's the last reference.

The code has been changed to avoid a call to this function.

In the previous version we always tried to keep the newest fd in the
ec_lock_t structure. However this is not necessary. We'll always keep
one reference to an open file on the same inode. It's irrelevant if
the reference is new or old.

The function __fd_unref() has also been removed from fd.h to avoid being
used in the future since it's useless as it's defined now.

Backport of http://review.gluster.org/14683

Change-Id: Ia728777fc8e464758d5ea4d3bf020f0603919039
BUG: 1344421
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/14684
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Niels de Vos

protocol: Add framework to send transaction id with recall

2016-06-10T11:03:54+00:00

Backport of: http://review.gluster.org/#/c/14647/

Issue:
The upcall(cache invalidation/recall) event is sent from the bricks
to clients. In AFR/EC setup, it can so happen that all the bricks
will send the upcall for the same event, and if AFR/EC doesn't filter
out these duplicate notifications, the logic above cluster xlators
can fail.

Solution:
Use transaction id to filter out duplicate notifications.

This patch adds framework for duplicate notifications.
AFR/EC can build up on this patch for deduping the notifications

Change-Id: I66b08e63b8799bc5932f2b2545376138a5701168
BUG: 1337638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14648
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

libglusterfs (timer): race conditions, illegal mem access, mem leak

2016-06-08T12:15:20+00:00

While investigating gfapi memory consumption with valgrind, valgrind
reported several memory access issues.

Also see the timer 'registry' being recreated (shortly) after being
freed during teardown due to the way it's currently written.

Passing ctx as data to gf_timer_proc() is prone to memory access
issues if ctx is freed before gf_timer_proc() terminates. (And in
fact this does happen, at least in valgrind.) gf_timer_proc() doesn't
need ctx for anything, it only needs ctx->timer, so just pass that.

Nothing ever calls gf_timer_registry_init(). Nothing outside of
timer.c that is. Making it and gf_timer_proc() static.

backport mainline:
> http://review.gluster.org/14247
> BUG: 1333925

Change-Id: Ia28454dda0cf0de2fec94d76441d98c3927a906a
BUG: 1342620
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14644
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

posix/lock: implement meta-lock/unlock functionality

2016-06-06T06:33:53+00:00

problem : The lock state needs to be protected when rebalance is reading the
lock state on the source. Otherwise there will be locks left unmigrated.
Hence, to synchronize incoming lock requests with lock-migration, meta lock
is needed. Any new lock request will be queued if there is an active meta-lock
and with sucessful lock migration, will be unwound with EREMOTE, so that
dht module can wind the request to the correct destination.

On a successful lock migration, "pl_inode->migrated" flag is enabled. Hence,
any further request would be unwound with EREMOTE and will be redirected to
new destination.

More details can be found here:
https://github.com/gluster/glusterfs-specs/blob/master/accepted/Lock-Migration.md

design discussion:
https://www.gluster.org/pipermail/gluster-devel/2016-January/048088.html

Change-Id: Ief033d5652b5ca4ba6f499110a521cae283d6aba
BUG: 1338501
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14462
Tested-by: Kotresh HR 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G 
CentOS-regression: Gluster Build System

glusterfsd/main: Add ability to set oom_score_adj

2016-06-01T18:14:43+00:00

Give the administrator a possibility to set oom_score_adj for glusterfs
process. Applies to Linux only.

This is a backport of cb8f5e01f639cb6e8715b33bb725210cb0493887.

Change-Id: Iff13c2f4cb28457871c6ebeff6130bce4a8bf543
BUG: 1341697
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/14399
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/14605
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

glusterd: set the maximum op-version for 3.8.0

2016-05-30T08:56:43+00:00

release-3.8 was branched after the op-version was increased for version
4.0.0. The maximum op-version for GlusterFS 3.8.0 should be 30800.

BUG: 1327174
Change-Id: Ia5796219d8fb115a680e6080f164d2c399d728a3
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14551
Reviewed-by: Shyamsundar Ranganathan 
NetBSD-regression: NetBSD Build System 
Tested-by: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System

core: assorted spelling mistakes reported by Debian

2016-05-28T00:28:07+00:00

See also
> Change-Id: I567a4be8f0f31f6285550f243fe802895f6bc43b

Cherry picked from commit 868058b8e0162318801dc3006401ed28449d9268:
> Reported-by: Patrick Matthäi 
> BUG: 1336793
> Change-Id: Icb9a6ff94d86663a5bca4ba931d810439c02556e
> Signed-off-by: Kaleb S KEITHLEY 
> Reviewed-on: http://review.gluster.org/14526
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Jeff Darcy 

Change-Id: Icb9a6ff94d86663a5bca4ba931d810439c02556e
BUG: 1336794
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14549
Reviewed-by: Kaleb KEITHLEY 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System