glusterfs.git/libglusterfs, branch v3.9.1

ec: Invalidations in disperse volume should not update the stat

2017-01-17T14:55:46+00:00

Backport of http://review.gluster.org/16329

Issue:
In disperse volume, the file is present across bricks, hence the stat
from one brick doesn't carry the valid size of the file. Therefore
the upcall from one brick updating the md-cache results in wrong size
being updated.

Fix:
If the notification is cache invalidation then, indicate md-cache that
the attributes is invalid.

>Reviewed-on: http://review.gluster.org/16329
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Xavier Hernandez 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 
(cherry picked from commit 95d07a3d2d68805d93d36a447436e27c48777939)

BUG: 1410688
Change-Id: Id89d2283478e70b62b435a8891fffc86d2be8cb2
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16341
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: fix statvfs in FreeBSD

2017-01-13T21:23:30+00:00

FreeBSD interprets statvfs' f_bsize field in a different way than Linux.

This fix modifies the value returned by statvfs() on FreeBSD to match
the expected value by Gluster.

> Change-Id: I930dab6e895671157238146d333e95874ea28a08
> BUG: 1356076
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: http://review.gluster.org/16361
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Kaleb KEITHLEY 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

Change-Id: I3a8e06bea7ce63d16e5569260bf9abdbced06dc1
BUG: 1411898
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/16398
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Kaleb KEITHLEY 
CentOS-regression: Gluster Build System

posix: make sure atime and mtime are set when calling lutimes()

2017-01-09T01:10:41+00:00

When overwriting an existing file with O_TRUNC, the 'atime' was set to
0, meaning the Epoch (01-Jan-1970 UTC). However, the 'mtime' gets
updated correcty.

In case 'atime' or 'mtime' is not passed in the 'struct iatt', the time
values passed to the systemcall are taken from the current values are
returned by lstat().

Cherry picked from commit 9bed81ada6f91f998e9abd915b18e3f06557cdcb:
> Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
> BUG: 1401777
> Reported-by: Eivind Sarto 
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/16034
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Raghavendra Bhat 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 

Change-Id: I7021b7161dcd6c9a3e515d98f6d4847533c434b3
BUG: 1411010
Reported-by: Eivind Sarto 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/16355
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

dht/md-cache: Filter invalidate if the file is made a linkto file

2017-01-05T06:10:12+00:00

Backport of http://review.gluster.org/15789

Upcall as a part of setattr, sends an invalidation and the
invalidation carries the resulting stat value. When a file
is converted to linkto files, even then an invalidation
is set and as a result the mountpoint shows the sticky
bit in the stat of the file.
eg: ---------T. 945 root root 0 Nov  8 10:14 hardlink.999

Fix:
When dht recieves a notification of sticky bit change, it updates
the flag, to indicate md-cache to send the subsequent lookup.

>Reviewed-on: http://review.gluster.org/15789
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Niels de Vos 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Susant Palai 
>Reviewed-by: Rajesh Joseph 
>(cherry picked from commit 4536f7bdf16f8286d67598eda9a46c029f0c0bf4)

Change-Id: Ic2fd7a5b196db0754f9b97072e644e6bf69da606
BUG: 1401376
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/16022
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

glusterd/geo-rep: Fix glusterd crash

2016-12-14T15:27:51+00:00

Problem:
glusterd crashes when geo-rep mountbroker setup is
created if the slave user length is more than
8 characters.

Cause:
_POSIX_LOGIN_NAME_MAX is used which is 9 including
NULL byte.

Analysis:
While the man page says it sufficient for portability, but
acutally it's not. Linux allows the creation of username
upto 32 characters by default where the max length is 256.
And NetBSD's max is 17.

Linux:
  #getconf LOGIN_NAME_MAX
   256
NetBSD:
  #getconf LOGIN_NAME_MAX
   17

Fix:
Use LOGIN_NAME_MAX instead of _POSIX_LOGIN_NAME_MAX

>Reviewed-on: http://review.gluster.org/16053
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Aravinda VK 
>Reviewed-by: Atin Mukherjee 

Change-Id: I26b7230433ecbbed6e6914ed39221a478c0266a8
BUG: 1403108
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/16084
Tested-by: Atin Mukherjee 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

cluster/ec: Fix lk-owner set race in ec_unlock

2016-12-14T12:38:26+00:00

Problem:
Rename does two locks. There is a case where when it tries to unlock it sends
xattrop of the directory with new version, callback of these two xattrops can
be picked up by two separate epoll threads. Both of them will try to set the
lk-owner for unlock in parallel on the same frame so one of these unlocks will
fail because the lk-owner doesn't match.

Fix:
Specify the lk-owner which will be set on inodelk frame which will not be over
written by any other thread/operation.

 >BUG: 1402710
 >Change-Id: I666ffc931440dc5253d72df666efe0ef1d73f99a
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/16074
 >Reviewed-by: Xavier Hernandez 
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 

BUG: 1404572
Change-Id: Iff4f0c1364e6533f3c07f192138bcd321789b4cd
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/16130
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

protocol/server: Print error-xlator name

2016-12-14T07:49:25+00:00

Problem:
At the moment from which xlator the errors are stemming from is a mystery.

Fix:
With this patch we can find on the server side which xlator first gave
the errno received by server xlator. I am not yet sure how to get this
done for client side which has lot of copy_frame()s. May be another
patch.

 >Change-Id: Ie13307b965facf2a496123e81ce0bd6756f98ac9
 >BUG: 1394548
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/15836
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >Reviewed-by: Vijay Bellur 
 >CentOS-regression: Gluster Build System 

BUG: 1396780
Change-Id: I3b293d21528da391eafd0fbaa5b451a1d3ddc237
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/15886
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra Talur

afr,dht,ec: Replace GF_EVENT_CHILD_MODIFIED with event SOME_DESCENDENT_DOWN/UP

2016-12-13T09:46:14+00:00

Backport of http://review.gluster.org/#/c/15764/

Currently these are few events related to child_up/down:
GF_EVENT_CHILD_UP :  Issued when any of the protocol client
connects.
GF_EVENT_CHILD_MODIFIED : Issued by afr/dht/ec
GF_EVENT_CHILD_DOWN : Issued when any of the protocol client
disconnects.
These events get modified at the dht/afr/ec layers. Here is a
brief on the same.

DHT:
- All the subvolumes reported once, and atleast one child came
  up, then GF_EVENT_CHILD_UP is issued
- connect GF_EVENT_CHILD_UP is issued
- disconnect GF_EVENT_CHILD_MODIFIED is issued
- All the subvolumes disconnected, GF_EVENT_CHILD_DOWN is issued

AFR:
- First subvolume came up, then GF_EVENT_CHILD_UP is issued
- Subsequent subvolumes coming up, results in GF_EVENT_CHILD_MODIFIED
- Any of the subvolumes go down, then GF_EVENT_SOME_CHILD_DOWN is issued
- Last up subvolume goes down, then GF_EVENT_CHILD_DOWN is issued

Until the patch [1] introduced GF_EVENT_SOME_CHILD_UP,
GF_EVENT_CHILD_MODIFIED was issued by afr/dht when any of the subvolumes
go up or down.

Now with md-cache changes, there is a necessity to differentiate between
child up and down. Hence, introducing GF_EVENT_SOME_DESCENDENT_DOWN/UP and
getting rid of GF_EVENT_CHILD_MODIFIED.

[1] http://review.gluster.org/12573

>Reviewed-on: http://review.gluster.org/15764
>CentOS-regression: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Smoke: Gluster Build System 
>Reviewed-by: N Balachandran 
>Reviewed-by: Pranith Kumar Karampuri  
>Reviewed-by: Rajesh Joseph 
(cherry picked from commit f7ab6c45963fa0da68acedfb14281cd2456abc68)

Change-Id: I704140b6598f7ec705493251d2dbc4191c965a58
BUG: 1396880
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15890
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

libglusterfs: Fix a read hang

2016-12-13T09:45:51+00:00

Backport of http://review.gluster.org/15923

Issue:
=====
In certain cases, there was no unwind of read
from read-ahead xlator, thus resulting in hang.

RCA:
====
In certain cases, ioc_readv() issues STACK_WIND_TAIL() instead
of STACK_WIND(). One such case is when inode_ctx for that file
is not present (can happen if readdirp was called, and populates
md-cache and serves all the lookups from cache).

Consider the following graph:
...
io-cache (parent)
   |
readdir-ahead
   |
read-ahead
...

Below is the code snippet of ioc_readv calling STACK_WIND_TAIL:
ioc_readv()
{
...
 if (!inode_ctx)
   STACK_WIND_TAIL (frame, FIRST_CHILD (frame->this),
                    FIRST_CHILD (frame->this)->fops->readv, fd,
                    size, offset, flags, xdata);
   /* Ideally, this stack_wind should wind to readdir-ahead:readv()
      but it winds to read-ahead:readv(). See below for
      explaination.
    */
...
}

STACK_WIND_TAIL (frame, obj, fn, ...)
{
  frame->this = obj;
  /* for the above mentioned graph, frame->this will be readdir-ahead
   * frame->this = FIRST_CHILD (frame->this) i.e. readdir-ahead, which
   * is as expected
   */
  ...
  THIS = obj;
  /* THIS will be read-ahead instead of readdir-ahead!, as obj expands
   * to "FIRST_CHILD (frame->this)" and frame->this was pointing
   * to readdir-ahead in the previous statement.
   */
  ...
  fn (frame, obj, params);
  /* fn will call read-ahead:readv() instead of readdir-ahead:readv()!
   * as fn expands to "FIRST_CHILD (frame->this)->fops->readv" and
   * frame->this was pointing ro readdir-ahead in the first statement
   */
  ...
}

Thus, the readdir-ahead's readv() implementation will be skipped, and
ra_readv() will be called with frame->this = "readdir-ahead" and
this = "read-ahead". This can lead to corruption / hang / other problems.
But in this perticular case, when 'frame->this' and 'this' passed
to ra_readv() doesn't match, it causes ra_readv() to call ra_readv()
again!. Thus the logic of read-ahead readv() falls apart and leads to
hang.

Solution:
=========
Modify STACK_WIND_TAIL() as:
STACK_WIND_TAIL (frame, obj, fn, ...)
{
  next_xl = obj /* resolve obj as the variables passed in obj macro
                   can be overwritten in the further instrucions */
  next_xl_fn = fn /* resolve fn and store in a tmp variable, before
                     modifying any variables */
  frame->this = next_xl;
  ...
  THIS = next_xl;
  ...
  next_xl_fn (frame, next_xl, params);
  ...
}

>Reviewed-on: http://review.gluster.org/15923
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>Reviewed-by: Rajesh Joseph 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Raghavendra G 
(Cherry picked from commit 8943c19a2ef51b6e4fa66cb57211d469fe558579)

BUG: 1399015
Change-Id: Ie662ac8f18fa16909376f1e59387bc5b886bd0f9
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15933
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

syncop: fix conditional wait bug in parallel dir scan

2016-12-12T06:51:57+00:00

Problem:
The issue as seen by the user is detailed in the BZ but what is
happening is if the no. of items in the wait queue == max-qlen,
syncop_mt_dir_scan() does a pthread_cond_wait until the launched
synctask workers dequeue the queue. But if for some reason the worker
fails, the queue is never emptied due to which further invocations of
syncop_mt_dir_scan() are blocked forever.

Fix: Made some changes to _dir_scan_job_fn

- If a worker encounters error while processing an entry, notify the
  readdir loop in syncop_mt_dir_scan() of the error but continue to process
  other entries in the queue, decrementing the qlen as and when we dequeue
  elements, and ending only when the queue is empty.

- If the readdir loop in syncop_mt_dir_scan() gets an error form the
  worker, stop the readdir+queueing of further entries.


> Reviewed-on: http://review.gluster.org/16073
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

(cherry picked from commit 2d012c4558046afd6adb3992ff88f937c5f835e4)
Change-Id: I39ce073e01a68c7ff18a0e9227389245a6f75b88
BUG: 1403187
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/16095
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri