glusterfs.git/libglusterfs, branch v3.8.2

dht/rebalance: allocate migrator thread pool dynamically

2016-08-10T10:21:54+00:00

Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a
crash or memory corruption can be seen.

Fix: Make the migratior thread pool dynamic.

> Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
> BUG: 1362069
> Reviewed-on: http://review.gluster.org/15000
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b)

Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1362069
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15061
Smoke: Gluster Build System 
Tested-by: N Balachandran 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

libglusterfs: fix glusterd statedump crash

2016-08-07T11:53:48+00:00

Backport of http://review.gluster.org/14987

commit 3c04a91 removed setting typeStr to NULL if num_allocs is set to 0, this
has caused this regression. Code has been put back like earlier and to avoid
statedump printing all the NULL values check is modified to see skip the records
if num_allocs is 0 instead of total_allocs

>Reviewed-on: http://review.gluster.org/14987
>NetBSD-regression: NetBSD Build System 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: N Balachandran 
>Reviewed-by: Prashanth Pai 
>Reviewed-by: Jeff Darcy 

Change-Id: Ib8bcc2fba908e88cf52b641c3f6bcba74f5e667c
BUG: 1364329
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15091
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Niels de Vos

features/ctr: Check for NULL local

2016-07-26T15:14:02+00:00

This is a defensive fix to prevent a crash reported
during a rename operation. This is not reproducible
under normal circumstances.

This patch also moves ctr-messages.h to the src dir
of the changetimerecorder xlator.

Backported from master: http://review.gluster.org/#/c/14964/

Change-Id: If459f46d3ce9258f595d0ca7ef55942bf466d767
BUG: 1360122
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15009
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

rpc/socket.c: Modify approach to cleanup threads of socket_poller in socket_spawn.

2016-07-21T14:24:48+00:00

Problem: Current approach to cleanup threads of socket_poller is not appropriate.

Solution: Enable detach flag at the time of thread creation in socket_spawn.

Fix: Write a new wrapper(gf_create_detach_thread) to create detachable thread
     instead of store thread ids in a queue.

Test: Fix is verfied on gluster process, To test the patch followed below procedure
      Enable the client.ssl and server.ssl option on the volume
      Start the volume and count anon segment in pmap output for glusterd process
      pmap -x  | grep "\[ anon \]" | wc -l
      Stop the volume and check again count of anon segment it should not increase.

Backport of commit 2ee48474be32f6ead2f3834677fee89d88348382

> Signed-off-by: Mohit Agrawal 
> Change-Id: Ib8f7ec7504ec8f6f74b45ce6719b6fb47f9fdc37
> BUG: 1336508
> Reviewed-on: http://review.gluster.org/14694
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1354395
Change-Id: Ibdbbae508d9dda2fd36220a9b1e47f7776336929
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14891
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

afr, index: Clean up stale directory and file indices in granular entry sh

2016-07-15T13:43:13+00:00

	Backport of: http://review.gluster.org/14832

Specifically when a directory tree is removed (rm -rf)
while a brick is down, both the directory index and the
name indices of the files and subdirs under it will remain.
Self-heal will need to pick up these and remove them.

Towards this, afr sh will now also crawl indices/entry-changes
and call an rmdir on the dir if the directory index is stale.

On the brick side, rmdir fop has been implemented for index xl,
which would delete the directory index and its contents if present
in a synctask.

Change-Id: I08f45201adca56737ec2be1aab5433aebaefefd0
BUG: 1355609
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14920
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Jeff Darcy

libglusterfs: Implement API that provides page-aligned iobufs

2016-06-29T09:53:40+00:00

        Backport of: http://review.gluster.org/14672

One of the consumers of a page aligned buffer would be posix's
readv fop on O_DIRECT fds. Today the way it works is by getting
a page-aligned buffer through calloc, pread()ing into this buffer
and then copying its contents into a newly created iobuf's ptr.
This results in an extra memcpy() which can be avoided if we could
implement an api that would return an iobuf whose ptr is
page-aligned. That way the iobuf->ptr can be directly passed to
sys_pread() as a parameter by posix translator.

Change-Id: I7f64419461db387e4ab6b0e03c90471744aa56e1
BUG: 1351025
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14825
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Negate all but O_DIRECT flag if present on anon fds

2016-06-28T05:57:11+00:00

        Backport of: http://review.gluster.org/14665

This is to prevent any unforeseen problems that might arise due to
writevs and readvs being wound with @flag parameter containing
O_TRUNC or O_APPEND especially wrt translators like sharding and ec
where O_TRUNC write or O_APPEND write on individual shards/fragments
is not the same as O_TRUNC write or O_APPEND write as expected by the
application.

 >Change-Id: Ib0110731d6099bc888b7ada552a2abbea8e76051
 >BUG: 1342903
 >Signed-off-by: Krutika Dhananjay 
 >Reviewed-on: http://review.gluster.org/14735
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Pranith Kumar Karampuri 
 >CentOS-regression: Gluster Build System 

Change-Id: I7ffa4fa366f727f7e345ab0bf4c8eb009710074b
BUG: 1347553
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14755
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core, shard: Make shards inherit main file's O_DIRECT flag if present

2016-06-27T15:17:55+00:00

        Backport of: http://review.gluster.org/14191

If the application opens a file with O_DIRECT, the shards'
anon fds would also need to inherit the flag. Towards this,
shard xl would be passing the odirect flag in the @flags parameter
to the WRITEV fop. This will be used in anon fd resolution
and subsequent opening by posix xl.

 >Change-Id: I3a0593fa46cc25e390a5762a0354b469c2a1532d
 >BUG: 1342903
 >Signed-off-by: Krutika Dhananjay 
 >Reviewed-on: http://review.gluster.org/14663
 >Smoke: Gluster Build System 
 >CentOS-regression: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Jeff Darcy 

Change-Id: Ibfc164aa7f9eecd6993255f1c03557f2ec35ac8c
BUG: 1347553
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14754
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/ec: Fix invalid __fd_unref() call

2016-06-13T10:16:17+00:00

__fd_unref() doesn't do any cleanup, so it cannot be called to release
fd references, specially if it's the last reference.

The code has been changed to avoid a call to this function.

In the previous version we always tried to keep the newest fd in the
ec_lock_t structure. However this is not necessary. We'll always keep
one reference to an open file on the same inode. It's irrelevant if
the reference is new or old.

The function __fd_unref() has also been removed from fd.h to avoid being
used in the future since it's useless as it's defined now.

Backport of http://review.gluster.org/14683

Change-Id: Ia728777fc8e464758d5ea4d3bf020f0603919039
BUG: 1344421
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/14684
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Niels de Vos

protocol: Add framework to send transaction id with recall

2016-06-10T11:03:54+00:00

Backport of: http://review.gluster.org/#/c/14647/

Issue:
The upcall(cache invalidation/recall) event is sent from the bricks
to clients. In AFR/EC setup, it can so happen that all the bricks
will send the upcall for the same event, and if AFR/EC doesn't filter
out these duplicate notifications, the logic above cluster xlators
can fail.

Solution:
Use transaction id to filter out duplicate notifications.

This patch adds framework for duplicate notifications.
AFR/EC can build up on this patch for deduping the notifications

Change-Id: I66b08e63b8799bc5932f2b2545376138a5701168
BUG: 1337638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14648
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G