glusterfs.git/xlators, branch v3.9rc1

feature/bitrot: Fix recovery of corrupted hardlink

2016-09-19T05:37:55+00:00

Problem:
When a file with hardlink is corrupted in ec volume,
the recovery steps mentioned was not working.
Only name and metadata was healing but not the data.

Cause:
The bad file marker in the inode context is not removed.
Hence when self heal tries to open the file for data
healing, it fails with EIO.

Background:
The bitrot deletes inode context during forget.

Briefly, the recovery steps involves following steps.
   1. Delete the entry marked with bad file xattr
      from backend. Delete all the hardlinks including
      .glusters hardlink as well.
   2. Access the each hardlink of the file including
      original from the mount.

The step 2 will send lookup to the brick where the files
are deleted from backend and returns with ENOENT. On
ENOENT, server xlator forgets the inode if there are
no dentries associated with it. But in case hardlinks,
the forget won't be called as dentries (other hardlink
files) are associated with the inode. Hence bitrot stube
won't delete it's context failing the data self heal.

Fix:
Bitrot-stub should delete the inode context on getting
ENOENT during lookup.

>Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1
>BUG: 1373520
>Signed-off-by: Kotresh HR 
>Reviewed-on: http://review.gluster.org/15408
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra Bhat 
(cherry picked from commit b86a7de9b5ea9dcd0a630dbe09fce6d9ad0d8944)

Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1
BUG: 1374567
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15434
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Aravinda VK

dht/events: Added rebalance events

2016-09-17T10:51:48+00:00

The rebalance process will now send an event when it is
complete.
Also fixed a problem where the run-time was not always
set causing spurious rebalance failure events to be sent.

> Change-Id: Ib445171c78c9560940022bca20c887d31a9bb1ca
> BUG: 1371874
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/15501
> Reviewed-by: Raghavendra G 
> Reviewed-by: Shyamsundar Ranganathan 
(cherry picked from commit e872917dbf4563703b8f396d7d30f8168ed646cd)

Change-Id: Ia24038c04548a98aac899df10414585b988caa76
BUG: 1376477
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/15520
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

ganesha/glusterd : Correct the path for ganesha conf dir in Makefile

2016-09-17T10:44:00+00:00

The value for ganesha conf dir in Makefile.am is mistakenly entered as
following :
DCONFDIR=\"/$(runstatedir)/gluster/shared_storage/nfs-ganesha\"

Here value for runstatedir is seems to "NULL" which results wrong path
for ganesha configuration directory

Upstream reference :
> Change-Id: I0b7ebd8e2503de0cb79b601553c4405d0d1fd711
> BUG: 1355956
> Signed-off-by: Jiffin Tony Thottan 
> Reviewed-on: http://review.gluster.org/15355
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: soumya k 
> Reviewed-by: Niels de Vos 
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> (cherry picked from commit 7e18e16f9f62ed95acded0fb3f2a8784087f59c0)

Change-Id: I0b7ebd8e2503de0cb79b601553c4405d0d1fd711
BUG: 1376874
Signed-off-by: Jiffin Tony Thottan 
Reviewed-on: http://review.gluster.org/15521
Reviewed-by: soumya k 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee

Tier: failing detach commit on detach failure and in-progress

2016-09-15T20:49:38+00:00

        back-port of: http://review.gluster.org/#/c/15438/

PROBLEM: if detach status has failed or if it remains in progress
we allow detach commit to happen. only detach force should
be allowed.

FIX: check the detach status for failure or inprogress and disallow
with the apt error message.

>Change-Id: Ib97d540fec67717bb55c18d133187c665cf69ef1
>BUG: 1374584
>Signed-off-by: hari gowtham 
>Reviewed-on: http://review.gluster.org/15438
>Smoke: Gluster Build System 
>Tested-by: hari gowtham 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 
Signed-off-by: hari gowtham 

Change-Id: I4f8c3859121ee0827df57348069a243a2502889a
BUG: 1375570
Reviewed-on: http://review.gluster.org/15493
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Dan Lambright

storage/posix: Integrate important events with gf_event

2016-09-15T04:18:56+00:00

        Backport of http://review.gluster.org/15342

BUG: 1375914
Change-Id: Icc23b22de02e45d2b0f8c6339ee628b439a4ffa8
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15497
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N

system/posix-acl: Unwind with NULL xdata on error

2016-09-14T04:43:59+00:00

In posix-acl when there are errors xdata that comes as part of input is used to
unwind which can be used as response xdata which may lead to problems as the
keys in the input will match with keys in the output but the values the
response xdata may expect can be completely different.

For example, we see that dht sends DHT_IATT_IN_XDATA_KEY in setxattr
which will be unwound with the same key in the xdata-response which
dht thinks is valid response and fills stbuf with invalid values
leading to EIO

> BUG: 1374093
> Change-Id: I6b77a1fa1ee99cb62e181e1db2e6fea73f6eaaa3
> Signed-off-by: Pranith Kumar K 
> Reviewed-on: http://review.gluster.org/15421
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Niels de Vos 
(cherry picked from commit c9271ff14d3efa8279cf67907548b3f43970d4fb)

Change-Id: I6b77a1fa1ee99cb62e181e1db2e6fea73f6eaaa3
BUG: 1374639
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15474
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Add support for hardware acceleration

2016-09-10T02:19:13+00:00

This patch implements functionalities for fast encoding/decoding
using hardware support. Currently optimized x86_64, SSE and AVX is
added.

Additionally this patch implements a caching mecanism for inverse
matrices to reduce computation time, as well as a new method for
computing the inverse that takes quadratic time instead of cubic.

Finally some unnecessary memory copies have been eliminated to
further increase performance.

 >Change-Id: I26c75f26fb4201bd22b51335448ea4357235065a
 >BUG: 1289922
 >Signed-off-by: Xavier Hernandez 
 >Reviewed-on: http://review.gluster.org/12837
 >Tested-by: Pranith Kumar Karampuri 
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Pranith Kumar Karampuri 

BUG: 1374841
Change-Id: I83731663922ed11ca84536deab5737463416e1e0
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15455
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

cluster/tier: add tiering events

2016-09-09T05:48:51+00:00

Add events for:
* tier attach and detach
* tier pause and resume
* tier rising and dropping hi and lo watermarks

Update eventskeygen.py with tiering events.
Update cli help with:
* attach: add optional force argument
* detach: make force available as non-optional argument on its own

> Reviewed-on: http://review.gluster.org/15232
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

(cherry picked from commit cd23242b8953040b231f402c334f16520ac8029f)

Change-Id: I43990d3a8742151a4a7889bafa19cb572fe661bd
BUG: 1374324
Signed-off-by: Milind Changire 
Reviewed-on: http://review.gluster.org/15431
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/ec: Add events for EC translator

2016-09-08T12:21:52+00:00

This patch will generates events in following
cases which will be consumed by new event
framework.

Consider an EC volume with (K+M) configuration
K = Data bricks
M = Redundancy bricks

1- EVENT_EC_MIN_BRICKS_NOT_UP -
   When minimum "K" number of bricks, required
   for any ec fop, are not up.
2- EVENT_EC_MIN_BRICKS_UP
   When minimum "K" number of bricks, required
   for any ec fop, are up.

>Change-Id: I0414b8968c39740a171e5aa14b087afd524d574f
>BUG: 1371470
>Signed-off-by: Ashish Pandey 
>Reviewed-on: http://review.gluster.org/15348
>Tested-by: Pranith Kumar Karampuri 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

Change-Id: I0414b8968c39740a171e5aa14b087afd524d574f
BUG: 1374167
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15425
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol-server: generate events

2016-09-08T07:59:22+00:00

This patch add/generate the events seen from/in the protocol server
side, which will be consumed by the new eventing framework.

-------------------------------------------------------------------------
|          event                |            description                |
|-------------------------------|---------------------------------------|
| EVENT_CLIENT_CONNECT          |          new client connected         |
|-------------------------------|---------------------------------------|
| EVENT_CLIENT_AUTH_REJECT      |     client cannot be authenticated    |
|-------------------------------|---------------------------------------|
| EVENT_CLIENT_DISCONNECT       |        client had disconnected        |
-------------------------------------------------------------------------

Thanks to "Raghavendra Talur"  for all the help

Backport of:
> Change-Id: I4fda83fae7747507f64d81a7645cc3a8a2fbaeae
> BUG: 1369403
> Signed-off-by: Prasanna Kumar Kalever 
> Reviewed-on: http://review.gluster.org/15294
> Tested-by: Prasanna Kumar Kalever 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I4fda83fae7747507f64d81a7645cc3a8a2fbaeae
BUG: 1373740
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/15415
Tested-by: Prasanna Kumar Kalever 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee