glusterfs.git/libglusterfs/src, branch v3.8.3

cluster/afr: Prevent split-brain when bricks are brought off and on in cyclic order

2016-08-22T10:22:36+00:00

        Backport of: http://review.gluster.org/15080

When the bricks are brought offline and then online in cyclic
order while writes are in progress on a file, thanks to inode
refresh in write txns, AFR will mostly fail the write attempt
when the only good copy is offline. However, there is still a
remote possibility that the file will run into split-brain if
the brick that has the lone good copy goes offline *after* the
inode refresh but *before* the write txn completes (I call it
in-flight split-brain in the patch for ease of reference),
requiring intervention from admin to resolve the split-brain
before the IO can resume normally on the file. To get around this,
the patch does the following things:
i) retains the dirty xattrs on the file
ii) avoids marking the last of the good copies as bad (or accused)
    in case it is the one to go down during the course of a write.
iii) fails that particular write with the appropriate errno.

This way, we still have one good copy left despite the split-brain situation
which when it is back online, will be chosen as source to do the heal.

> Change-Id: I9ca634b026ac830b172bac076437cc3bf1ae7d8a
> BUG: 1363721
> Signed-off-by: Krutika Dhananjay 
> Reviewed-on: http://review.gluster.org/15080
> Tested-by: Pranith Kumar Karampuri 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Ravishankar N 
> Reviewed-by: Oleksandr Natalenko 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit fcb5b70b1099d0379b40c81f35750df8bb9545a5)

Change-Id: I157f1025aebd6624fa3d412abc69a4ae6f2fe9e0
BUG: 1367272
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15221
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

core: use for makedev(3), major(3), minor(3)

2016-08-22T08:43:46+00:00

gnu glibc (libc) in Fedora 26 (rawhide) has turned off "compat" for
makedev(), major(), and minor(3) decls.

3.8.2 on F26 built with warnings, see:
https://kojipkgs.fedoraproject.org//packages/glusterfs/3.8.2/1.fc26/data/logs/x86_64/build.log

More recent builds of other packages (namely nfs-ganesha) indicate
that these warnings would now be errors, and, will be errors when
3.8.3 is released and built.

Cherry picked from commit c441a0ae5aca1f2977961afecb3236f90eb5b672:
> Change-Id: I3caa0d05f5de695818753e40e754da8b40d2011c
> BUG: 1367527
> Signed-off-by: Kaleb S. KEITHLEY 
> Reviewed-on: http://review.gluster.org/15182
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Niels de Vos 

Change-Id: I3caa0d05f5de695818753e40e754da8b40d2011c
BUG: 1368423
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15203
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

inode: Adjust lru_size while retiring entries in lru list

2016-08-13T08:04:01+00:00

As part of inode_table_destroy(), we first retire entries
in the lru list but the lru_size is not adjusted accordingly.
This may result in invalid memory reference in inode_table_prune
if the lru_size > lru_limit.

> Reviewed-on: http://review.gluster.org/15087
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 
> Reviewed-by: Prashanth Pai 

BUG: 1362540
Change-Id: I29ee3c03b0eaa8a118d06dc0cefba85877daf963
Signed-off-by: Soumya Koduri 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15129
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

libglusterfs/client_t: Dump the 0th client too

2016-08-11T13:04:22+00:00

> Reviewed-on: http://review.gluster.org/14704
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1365742
Change-Id: I565e81944b6670d26ed1962689dcfd147181b61e
Signed-off-by: Raghavendra G 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15124
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System

dht/rebalance: allocate migrator thread pool dynamically

2016-08-10T10:21:54+00:00

Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a
crash or memory corruption can be seen.

Fix: Make the migratior thread pool dynamic.

> Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
> BUG: 1362069
> Reviewed-on: http://review.gluster.org/15000
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b)

Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1362069
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15061
Smoke: Gluster Build System 
Tested-by: N Balachandran 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

libglusterfs: fix glusterd statedump crash

2016-08-07T11:53:48+00:00

Backport of http://review.gluster.org/14987

commit 3c04a91 removed setting typeStr to NULL if num_allocs is set to 0, this
has caused this regression. Code has been put back like earlier and to avoid
statedump printing all the NULL values check is modified to see skip the records
if num_allocs is 0 instead of total_allocs

>Reviewed-on: http://review.gluster.org/14987
>NetBSD-regression: NetBSD Build System 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: N Balachandran 
>Reviewed-by: Prashanth Pai 
>Reviewed-by: Jeff Darcy 

Change-Id: Ib8bcc2fba908e88cf52b641c3f6bcba74f5e667c
BUG: 1364329
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/15091
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Niels de Vos

features/ctr: Check for NULL local

2016-07-26T15:14:02+00:00

This is a defensive fix to prevent a crash reported
during a rename operation. This is not reproducible
under normal circumstances.

This patch also moves ctr-messages.h to the src dir
of the changetimerecorder xlator.

Backported from master: http://review.gluster.org/#/c/14964/

Change-Id: If459f46d3ce9258f595d0ca7ef55942bf466d767
BUG: 1360122
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15009
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

rpc/socket.c: Modify approach to cleanup threads of socket_poller in socket_spawn.

2016-07-21T14:24:48+00:00

Problem: Current approach to cleanup threads of socket_poller is not appropriate.

Solution: Enable detach flag at the time of thread creation in socket_spawn.

Fix: Write a new wrapper(gf_create_detach_thread) to create detachable thread
     instead of store thread ids in a queue.

Test: Fix is verfied on gluster process, To test the patch followed below procedure
      Enable the client.ssl and server.ssl option on the volume
      Start the volume and count anon segment in pmap output for glusterd process
      pmap -x  | grep "\[ anon \]" | wc -l
      Stop the volume and check again count of anon segment it should not increase.

Backport of commit 2ee48474be32f6ead2f3834677fee89d88348382

> Signed-off-by: Mohit Agrawal 
> Change-Id: Ib8f7ec7504ec8f6f74b45ce6719b6fb47f9fdc37
> BUG: 1336508
> Reviewed-on: http://review.gluster.org/14694
> Smoke: Gluster Build System 
> Reviewed-by: Atin Mukherjee 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Jeff Darcy 

BUG: 1354395
Change-Id: Ibdbbae508d9dda2fd36220a9b1e47f7776336929
Signed-off-by: Mohit Agrawal 
Reviewed-on: http://review.gluster.org/14891
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

afr, index: Clean up stale directory and file indices in granular entry sh

2016-07-15T13:43:13+00:00

	Backport of: http://review.gluster.org/14832

Specifically when a directory tree is removed (rm -rf)
while a brick is down, both the directory index and the
name indices of the files and subdirs under it will remain.
Self-heal will need to pick up these and remove them.

Towards this, afr sh will now also crawl indices/entry-changes
and call an rmdir on the dir if the directory index is stale.

On the brick side, rmdir fop has been implemented for index xl,
which would delete the directory index and its contents if present
in a synctask.

Change-Id: I08f45201adca56737ec2be1aab5433aebaefefd0
BUG: 1355609
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14920
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Jeff Darcy

libglusterfs: Implement API that provides page-aligned iobufs

2016-06-29T09:53:40+00:00

        Backport of: http://review.gluster.org/14672

One of the consumers of a page aligned buffer would be posix's
readv fop on O_DIRECT fds. Today the way it works is by getting
a page-aligned buffer through calloc, pread()ing into this buffer
and then copying its contents into a newly created iobuf's ptr.
This results in an extra memcpy() which can be avoided if we could
implement an api that would return an iobuf whose ptr is
page-aligned. That way the iobuf->ptr can be directly passed to
sys_pread() as a parameter by posix translator.

Change-Id: I7f64419461db387e4ab6b0e03c90471744aa56e1
BUG: 1351025
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14825
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri