glusterfs.git/xlators, branch v3.8.2

storage/posix: fix inode leaks

2016-08-10T13:47:07+00:00

> Reviewed-on: http://review.gluster.org/14739
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Pranith Kumar Karampuri 
> Reviewed-by: Krutika Dhananjay 

BUG: 1365742
Change-Id: Ibd221ba62af4db17bea5c52d37f5c0ba30b60a7d
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/15127
Tested-by: Oleksandr Natalenko 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

glusterd: Fix memory leak in glusterd (un)lock RPCs

2016-08-10T10:32:28+00:00

Problem:  At the time of execute "gluster volume profile  info" command
          It does have memory leak in glusterd.

Solution: Modify the code to prevent memory leak in glusterd.

Fix    : 1) Unref dict and free dict_val buffer in glusterd_mgmt_v3_lock_peer and
            glusterd_mgmt_v3_unlock_peers.

Test   : To verify the patch run below loop to generate io traffic
         for (( i=0 ; i<=1000000 ; i++ ));
           do echo "hi Start Line " > file$i;
           cat file$i >> /dev/null;
         done

         To verify the improvement in memory leak specific to glusterd run below command

         cnt=0;while [ $cnt -le 1000 ]; do
         pmap -x  | grep total;
         gluster volume profile distributed info > /dev/null; cnt=`expr $cnt + 1`; done

         After apply this patch it will reduce leak significantly.

> Reviewed-on: http://review.gluster.org/14862
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Atin Mukherjee 
> Reviewed-by: Prashanth Pai 

BUG: 1365743
Change-Id: I52a0ca47adb20bfe4b1848a11df23e5e37c5cea9
Signed-off-by: Mohit Agrawal 
Signed-off-by: Oleksandr Natalenko 
Reviewed-on: http://review.gluster.org/15125
Reviewed-by: Atin Mukherjee 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

dht/rebalance: allocate migrator thread pool dynamically

2016-08-10T10:21:54+00:00

Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a
crash or memory corruption can be seen.

Fix: Make the migratior thread pool dynamic.

> Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
> BUG: 1362069
> Reviewed-on: http://review.gluster.org/15000
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 
(cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b)

Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1362069
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/15061
Smoke: Gluster Build System 
Tested-by: N Balachandran 
Reviewed-by: N Balachandran 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

posix: Do not move and recreate .glusterfs/unlink directory

2016-08-10T09:31:50+00:00

Problem:
At the time of start of a volume, it is checked if
.glusterfs/unlink exist or not. If it does, move it
to landfill and recreate unlink directory. If a volume
is mounted and we write data on it till we face ENOSPC,
restart of that volume fails as it will not be able to
create unlink dir. mkdir will fail with ENOSPC.
This will not allow volume to restart.

Solution:
If .glusterfs/unlink directory exist, don't move it to
landfill. Delete all the entries inside it.

master -
http://review.gluster.org/#/c/15030/

Change-Id: Icde3fb36012f2f01aeb119a2da042f761203c11f
BUG: 1364365
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15093
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

feature/bitrot: Ignore files with sticky bit set

2016-08-10T05:20:22+00:00

Backport of http://review.gluster.org/14903

Scrubber scrubs entries in backend. It is scrubbing files with sticky
bit as well. This might include linkfiles which should be skipped.  This
patch adds the check to ignore linkfiles during scrub.

Change-Id: Ic21367b37770d391326c55c659491a1e5a82335b
BUG: 1359020
Signed-off-by: Kotresh HR 
(cherry picked from commit 8c47b19fc057f08c47444ef557503e610c707128)
Reviewed-on: http://review.gluster.org/14983
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

glusterd : skip non directories inside /var/lib/glusterd/vols

2016-08-09T15:03:14+00:00

Right now glusterd won't come up if vols directory contains an invalid entry.
Instead of doing that with this change a message will be logged and then skip
that entry

Backport details: 
>Change-Id: I665b5c35291b059cf054622da0eec4db44ec5f68
>BUG: 1318591
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/13764
>Reviewed-by: Prashanth Pai 
>Reviewed-by: Atin Mukherjee 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
(cherry picked from commit 720b63c24b07ee64e1338db28de602b9abbef0a1)

Change-Id: I665b5c35291b059cf054622da0eec4db44ec5f68
BUG: 1365265
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/15113
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

posix: honour fsync flags in posix_do_zerofill

2016-08-06T05:40:25+00:00

Backport of http://review.gluster.org/#/c/15100/
commit fe1054110ac54750ca0333a727d83b14a98e165e introduced fallocate +
FALLOC_FL_ZERO_RANGE to do posix_do_zerofill but did not do an fsync if
O_SYNC or O_DSYNC fd flags were set. Prashant Pai pointed out performing
fallocate(2) does not remove the necessity to do fsyncs if durability is
desired: http://linux-fsdevel.vger.kernel.narkive.com/bDmrAUlh/fallocate-falloc-fl-punch-hole

So fixed it to honour the fd flags.

Change-Id: I27ae90b429185d0af29a5e632c4a8b242075a899
BUG: 1364497
Signed-off-by: Ravishankar N 
(cherry picked from commit cf51bc699646359be92969024a52c225e7ee55a1)
Reviewed-on: http://review.gluster.org/15103
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

nfs: allow hostnames with dashes in exports/netgroups files

2016-08-05T13:19:28+00:00

Hostnames with dashes (like "vagrant-testVM") are not correctly parsed
when reading the exports/netgroups files. This bacomes obvious when
running ./run-tests-in-vagrant.sh because it causes
tests/basic/mount-nfs-auth.t and tests/basic/netgroup_parsing.t to fail.

The regex for hostname (in exports) and the entry and hostname
(netgroups) parsing does not include the "-" sign, and hence the
hostnames are splitted at it.

Cherry picked from commit e5221d288e41d29d89d52f8deab657d2285a852c:
> BUG: 1350237
> Change-Id: I38146a283561e1fa386cc841c43fd3b1e30a87ad
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/14809
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I38146a283561e1fa386cc841c43fd3b1e30a87ad
BUG: 1357834
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14955
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: jiffin tony Thottan

cluster/tier: dont promote if estimated block consumption > hi watermark

2016-08-05T08:12:11+00:00

Add test to fail promotion if estimated block consumption grows
beyond hi watermark.

Skip file migrations until next cycle if tier_get_fs_stat() fails
in tier_migrate_using_query_file()

> Reviewed-on: http://review.gluster.org/14780
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: mohammed rafi  kc 
> Reviewed-by: Dan Lambright 

(cherry picked from commit 1f4e41e8c2f5f4af4564caba0a08996853f089f4)

Change-Id: Ice04572fa739c09109c4433e65965197482a7beb
BUG: 1362198
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/15065
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

xlator/trash : append '/' at the end in trash_notify_lookup_cbk

2016-08-05T07:51:28+00:00

In the notify function in trash xlator, a lookup is performed
to obtain path of old trash directory. The result usually contains
path without '/' at the end. The trash xlator maintains expects
'/' at the end for the values such as 'old trash dir' and
'new trash dir'. Otherwise certian checks in the code will fail.

>Change-Id: I89e02e4b249314fb6536297f959865feee182c83
>BUG: 1357397
>Signed-off-by: Jiffin Tony Thottan 
>Reviewed-on: http://review.gluster.org/14938
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Anoop C S 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Jeff Darcy 
>(cherry picked from commit d90307c1b0245e0e6a39044a28819cde520a100c)

Change-Id: I89e02e4b249314fb6536297f959865feee182c83
BUG: 1358262
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/14965
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Anoop C S 
Reviewed-by: Niels de Vos