glusterfs.git, branch v3.7.17

doc: Update release-notes again

2016-11-03T13:22:02+00:00

Change-Id: Ia58884e333ef03feca4172dc6977f83e1df29815
Signed-off-by: Kaushal M

doc: Amending release notes for 3.7.17

2016-11-03T13:17:18+00:00

The release note for 3.7.17 has been amended to include the
following:

 - Note to Samba users regarding the possibility of this update
breaking their vfs_glusterfs module, if the following patch isn't
included in their Samba packages:
patch https://git.samba.org/?p=samba.git;a=commitdiff;h=92a0a56c3852726e0812d260e043957c879aefa4

 - Include https://bugzilla.redhat.com/1369363 in list of fixed bugs

Change-Id: I5ffa1d3ffb0fb85bb3965ad77573e516eba60c07
BUG: 1385526
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/15767
Smoke: Gluster Build System 
Tested-by: Kaushal M 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaushal M

performance/write-behind: fix flush stuck by former failed writes

2016-11-03T09:28:17+00:00

the issue is happened in this case:
assume a file is opened with fd1 and fd2.
1. some WRITE opto fd1 got error, they were add back to 'todo' queue
   because of those error.
2. fd2 closed, a FLUSH op is send to write-behind.
3. FLUSH can not be unwind because it's not a legal waiter for those
   failed write(as func __wb_request_waiting_on() say). and those failed
   WRITE also can not be ended if fd1 is not closed. fd2 stuck in close
   syscall.

to resolve this issue, we can change the way we determine 2 requests is
'conflict': flush/fsync is not conflict with those write that is not
belonged to them. so __wb_pick_winds() can wind the FLUSH op.

below is some information when the stuck issue happen:
glusterdump logs:
[xlator.performance.write-behind.wb_inode]
path=/ltp-F9eG0ZSOME/rw-buffered-16436
inode=0x7fdbe8039b9c
window_conf=1048576
window_current=249856
transit-size=0
dontsync=0

[.WRITE]
request-ptr=0x7fdbe8020200
refcount=1
wound=no
generation-number=4
req->op_ret=-1
req->op_errno=116
sync-attempts=3
sync-in-progress=no
size=131072
offset=1220608
lied=-1
append=0
fulfilled=0
go=0

[.WRITE]
request-ptr=0x7fdbe8068c30
refcount=1
wound=no
generation-number=5
req->op_ret=-1
req->op_errno=116
sync-attempts=2
sync-in-progress=no
size=118784
offset=1351680
lied=-1
append=0
fulfilled=0
go=0

[.FLUSH]
request-ptr=0x7fdbe8021cd0
refcount=1
wound=no
generation-number=6
req->op_ret=0
req->op_errno=0
sync-attempts=0

gdb detail about above 3 requests:
(gdb) print *((wb_request_t *)0x7fdbe8021cd0)
$2 = {all = {next = 0x7fdbe803a608, prev = 0x7fdbe8068c30}, todo = {next
= 0x7fdbe803a618, prev = 0x7fdbe8068c40}, lie = {next = 0x7fdbe8021cf0,
    prev = 0x7fdbe8021cf0}, winds = {next = 0x7fdbe8021d00, prev =
0x7fdbe8021d00}, unwinds = {next = 0x7fdbe8021d10, prev =
0x7fdbe8021d10}, wip = {
    next = 0x7fdbe8021d20, prev = 0x7fdbe8021d20}, stub =
0x7fdbe80224dc, write_size = 0, orig_size = 0, total_size = 0, op_ret =
0, op_errno = 0,
  refcount = 1, wb_inode = 0x7fdbe803a5f0, fop = GF_FOP_FLUSH, lk_owner
= {len = 8, data = "W\322T\f\271\367y$", '\000' },
  iobref = 0x0, gen = 6, fd = 0x7fdbe800f0dc, wind_count = 0, ordering =
{size = 0, off = 0, append = 0, tempted = 0, lied = 0, fulfilled = 0,
    go = 0}}
(gdb) print *((wb_request_t *)0x7fdbe8020200)
$3 = {all = {next = 0x7fdbe8068c30, prev = 0x7fdbe803a608}, todo = {next
= 0x7fdbe8068c40, prev = 0x7fdbe803a618}, lie = {next = 0x7fdbe8068c50,
    prev = 0x7fdbe803a628}, winds = {next = 0x7fdbe8020230, prev =
0x7fdbe8020230}, unwinds = {next = 0x7fdbe8020240, prev =
0x7fdbe8020240}, wip = {
    next = 0x7fdbe8020250, prev = 0x7fdbe8020250}, stub =
0x7fdbe8062c3c, write_size = 131072, orig_size = 4096, total_size = 0,
op_ret = -1,
  op_errno = 116, refcount = 1, wb_inode = 0x7fdbe803a5f0, fop =
GF_FOP_WRITE, lk_owner = {len = 8, data = '\000' },
  iobref = 0x7fdbe80311a0, gen = 4, fd = 0x7fdbe805c89c, wind_count = 3,
ordering = {size = 131072, off = 1220608, append = 0, tempted = -1,
    lied = -1, fulfilled = 0, go = 0}}
(gdb) print *((wb_request_t *)0x7fdbe8068c30)
$4 = {all = {next = 0x7fdbe8021cd0, prev = 0x7fdbe8020200}, todo = {next
= 0x7fdbe8021ce0, prev = 0x7fdbe8020210}, lie = {next = 0x7fdbe803a628,
    prev = 0x7fdbe8020220}, winds = {next = 0x7fdbe8068c60, prev =
0x7fdbe8068c60}, unwinds = {next = 0x7fdbe8068c70, prev =
0x7fdbe8068c70}, wip = {
    next = 0x7fdbe8068c80, prev = 0x7fdbe8068c80}, stub =
0x7fdbe806746c, write_size = 118784, orig_size = 4096, total_size = 0,
op_ret = -1,
  op_errno = 116, refcount = 1, wb_inode = 0x7fdbe803a5f0, fop =
GF_FOP_WRITE, lk_owner = {len = 8, data = '\000' },
  iobref = 0x7fdbe8052b10, gen = 5, fd = 0x7fdbe805c89c, wind_count = 2,
ordering = {size = 118784, off = 1351680, append = 0, tempted = -1,
    lied = -1, fulfilled = 0, go = 0}}

you can see they are all on 'todo' queue, and FLUSH op fd is not the
same WRITE op fd.

> Change-Id: Id687f9cd3b9f281e1a97c83f1ce981ede272b8ab
> BUG: 1372211
> Signed-off-by: Ryan Ding 

Change-Id: Id687f9cd3b9f281e1a97c83f1ce981ede272b8ab
BUG: 1390840
Signed-off-by: Ryan Ding 
Reviewed-on: http://review.gluster.org/15763
Tested-by: Raghavendra G 
Reviewed-by: Raghavendra G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

snapshot/cli: Fix snapshot status xml output

2016-11-02T06:33:45+00:00

    Backport of http://review.gluster.org/#/c/14018/

snap status --xml errors out if a brick is down and
doesn't have pid. It is handled in the cli of the snap
status where "N/A" is displayed in such a scenario.
Handled the same in xml

snap status  --xml fails as the writer is
not initialised for the same. Using GF_SNAP_STATUS_TYPE_ITER
instead of GF_SNAP_STATUS_TYPE_SNAP for all snap's
status to differentiate between the two scenarios.

Added testcase volume-snapshot-xml.t to check
all snapshot commands xml outputs

> Reviewed-on: http://review.gluster.org/14018
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Rajesh Joseph 

Change-Id: I99563e8f3e84f1aaeabd865326bb825c44f5c745
BUG: 1369363
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/15290
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph

doc: Add release notes for 3.7.17

2016-10-30T17:32:08+00:00

Change-Id: I7e2bedaaa2fc4bfe425bd9384047df93a0438d04
BUG: 1385526
Signed-off-by: Samikshan Bairagya 
Reviewed-on: http://review.gluster.org/15756
Smoke: Gluster Build System 
Reviewed-by: Kaushal M 
Tested-by: Kaushal M 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

gfapi: warn when glfs_realpath() returned malloc'd memory

2016-10-28T11:40:08+00:00

glfs_realpath() may return memory allocated with malloc(). Depending on
the memory allocator that the application uses, calling free() on the
returned string can cause segmentation faults or other problems.

Functions that allocate memory, need to match the free'ing of the same
memory allocator and memory accounting. glibc/malloc and jemalloc/free
do not match together (other allocators could probably trigger these
problems as well).

Applications need to provide a pre-allocated buffer, or in case
glfs_realpath() allocates the memory, glfs_free() should be used to free
it.

Cherry picked from commit 85e959052148ec481823d55c8b91cdee36da2b43:
> Change-Id: I5d721a7425674aa700db8a7a436cbedb95a5927f
> BUG: 1370931
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/15332
> CentOS-regression: Gluster Build System 
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I5d721a7425674aa700db8a7a436cbedb95a5927f
BUG: 1383593
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15628
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

afr,ec: Heal device files with correct major, minor numbers

2016-10-27T06:23:20+00:00

Thanks a lot to xiaoping.wu@nokia.com from Nokia for the bug and the
fix.

 >BUG: 1384297
 >Change-Id: Ie443237e85d34633b5dd30f85eaa2ac34e45754c
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15728
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >CentOS-regression: Gluster Build System 

Change-Id: I28636a741592335cebcaa1abc2af8460ebc740e1
BUG: 1388949
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15736
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Xavier Hernandez

rpc/socket: Close pipe on disconnection

2016-10-26T11:38:17+00:00

Encrypted connections create a pipe, which isn't closed when the
connection disconnects. This leaks fds, and gluster eventually ends up
in a situation with fd starvation which leads to operation failures.

> Change-Id: I144e1f767cec8c6fc1aa46b00cd234129d2a4adc
> BUG: 1336371
> Signed-off-by: Kaushal M 
> Reviewed-on: http://review.gluster.org/14356
> Tested-by: MOHIT AGRAWAL 
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra G 

Change-Id: I144e1f767cec8c6fc1aa46b00cd234129d2a4adc
BUG: 1336369
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/15704
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: lidi 
CentOS-regression: Gluster Build System

gfapi: Fix few fd ref leaks

2016-10-20T14:07:05+00:00

From the code inspection, have observed that there are fd ref
leaks for few fd-based gfapi fops. 'glfs_resolve_fd' returns
a fd (either existing or migrated) with an extra ref taken.
This needs to be unref'ed at the end of the operation.

Mainline reference:
> Change-Id: Id63394e3e7deafb0c8b06444f2ae847248b126db
> BUG: 1379285
> Signed-off-by: Soumya Koduri 
> Reviewed-on: http://review.gluster.org/15573
> Reviewed-by: Niels de Vos 
(cherry picked from commit d27cffab670858d7812bbb458a0833303d009b3b)

BUG: 1379710
Change-Id: I143532a8a14febc8c4aad7a18b1d9166529b30cc
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/15604
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

system/posix-acl: Unwind with NULL xdata on error

2016-10-20T07:47:41+00:00

In posix-acl when there are errors xdata that comes as part of input is used to
unwind which can be used as response xdata which may lead to problems as the
keys in the input will match with keys in the output but the values the
response xdata may expect can be completely different.

For example, we see that dht sends DHT_IATT_IN_XDATA_KEY in setxattr
which will be unwound with the same key in the xdata-response which
dht thinks is valid response and fills stbuf with invalid values
leading to EIO

> BUG: 1374093
> Change-Id: I6b77a1fa1ee99cb62e181e1db2e6fea73f6eaaa3
> Signed-off-by: Pranith Kumar K 
> Reviewed-on: http://review.gluster.org/15421
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: jiffin tony Thottan 
> Reviewed-by: Niels de Vos 
(cherry picked from commit c9271ff14d3efa8279cf67907548b3f43970d4fb)

Change-Id: I6b77a1fa1ee99cb62e181e1db2e6fea73f6eaaa3
BUG: 1374641
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15475
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Pranith Kumar Karampuri