glusterfs.git/tests, branch v3.8rc1

cluster/afr: Don't let NFS cache stat after writes

2016-05-14T14:30:21+00:00

Problem:
Afr does post-ops after write but the stat buffer it unwinds is at the
time of write, so if nfs client caches this, it will see different
ctime when it does stat on it after post-op is done. From NFS client's
perspective it thinks the file is changed. Tar which depends on this
to be correct keeps giving 'file changed as we read it' warning.
If Afr instead has to choose to unwind after post-op, eager-lock,
delayed-post-op will have to be disabled which will lead to bad
performance for all write usecases.

Fix:
Don't let client cache stat after write.

 >Change-Id: Ic6062acc6e5cdd97a9c83c56bd529ec83cee8a23
 >BUG: 1302948
 >Signed-off-by: Pranith Kumar K 
 >Signed-off-by: Anuradha Talur 
 >Reviewed-on: http://review.gluster.org/13785
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Niels de Vos 

BUG: 1335285
Change-Id: Ibef4fc80496d12acd15db57713af2e3a1c9109a7
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14300
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Niels de Vos

cluster/afr: Do heals with shd pid

2016-05-14T14:14:37+00:00

Multi-threaded healing doesn't create synctask with shd pid, this
leads to healing problems when quota exceeds.

 >BUG: 1332994
 >Change-Id: I80f57c1923756f3298730b8820498127024e1209
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/14211
 >Smoke: Gluster Build System 
 >NetBSD-regression: NetBSD Build System 
 >CentOS-regression: Gluster Build System 
 >Reviewed-by: Ravishankar N 

BUG: 1335283
Change-Id: If59d8f88d8f4a3ca6a3b6e1c9dfd594dd93f542b
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14298
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

performance/write-behind: guaranteed retry after a short write

2016-05-05T03:27:10+00:00

* Don't mark the request with a fake EIO after a short write.
* retry the remaining buffer at least once before unwinding reply to
  application. This way we capture correct error from backend (ENOSPC,
  EDQUOT etc).

Thanks to "Vijaikumar Mallikarjuna" for the test
script.

Change-Id: I73a18b39b661a7424db1a7855a980469a51da8f9
BUG: 1332789
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/14197
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

posix: Set correct d_type for readdirp() calls

2016-05-03T14:53:46+00:00

dirent.d_type can contain the type of the directory entry. The 'd_type'
struct member in dirent is present in Linux and many BSD flavours.
However, filling d_type with correct value requires support from the
underlying filesystem. If not, d_type is set to DT_UNKNOWN. XFS added
support for d_type as part of their newer version 5 on-disk format.
However, this requires Linux >= 3.15, xfsprogs >= 3.2.0 and the bricks
to be formatted using the new format.

This patch enables posix xlator to set d_type to the right value even
when the underlying filesystem does not support it. d_type can be set
using information previously fetched by stat() on the dir entry.
This will aid FUSE applications to leverage d_type to avoid the expense
of calling lstat() if further actions depend on the type of the file.

Refer `man 3 readdir` and `man 2 getdents`

> Change-Id: Ic5a262fe4c64122726b4fae2d1bea375c559ca04
> Signed-off-by: Prashanth Pai 
> Reviewed-on: http://review.gluster.org/14095
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Jeff Darcy 

(cherry picked from commit 77def44d497d090ef3f393b6d9403c1a29dcf993)

Change-Id: Iaaf2a7c85ce8deb9043772e04a76b904d8bf076e
BUG: 1332396
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/14175
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System

features/locks: Implement mandatory locks

2016-05-03T09:24:58+00:00

Initial change to fix/enable the mandatory locking support in GlusterFS
as per the following design:

https://review.gluster.org/#/c/12014/

Accordingly 'locks.mandatory-locking' option is available as part of this
change which will accept one among the following values:

* off
* file
* forced
* optimal

See design doc for more details

> Reviewed-on: http://review.gluster.org/9768
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> Reviewed-by: Poornima G 
> Reviewed-by: Raghavendra Talur 
> Reviewed-by: Rajesh Joseph 
> Reviewed-by: Pranith Kumar Karampuri 

(cherry picked from commit 4517bf8dd6de310950cc5a612955aa3a2fddb57e)

Change-Id: I14c489b3f8af5ebcbfa155a03f0c175e9558ac46
BUG: 1332162
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14149
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System

WORM/Retention Translator: Implementation of file level WORM

2016-05-02T01:05:22+00:00

To activate the file level worm feature, the features.read-only and
features.worm options should be switched "off" on the volume and
the features.worm-file-level should be switched "on". Both read-only
and worm or worm-file-level cannot be switched "on" together. The
files which are created when the worm-file-level option is set on the
volume will have their own retention profile.

If both worm and worm-file-level are "on" at that time the worm
which is the volume level worm will have priority over file level
worm. If worm-file level is switched "off" after some time and the
read-only option is switched "on" then read-only will have priority.

The current implementation allows the users to manually transmit
a file to a WORM-Retained state by removing all the write bits of
the file using the chmod command. The file will have a retention
profile which contains the state of the file, mode of retention,
and the default retention time.

The file will be made WORM-Retained for a default of 120 seconds
during which it will be immutable and undeletable and it sets the
atime of the file to the time till which it is retained.
After that period if any fop request comes for that file, will
make the transition from WORM-Retained state to WORM state, where
the file will be immutable but deletable and, it will reset
the atime to the actual atime of the file. If a WORM file needs
to be made undeletable again, it can be done by using the chmod
command with all the write bits removed.

There are two modes of retention:
1. Relax: where the retention time of a WORM-Retained file can be
   increased or decreased.
2. Enterprise: where the retention time of a WORM-Retained file
   can be increased but not be decreased.
Whenever a utime change(touch -a, -t, ...)request comes for a
file it checks the mode of retention before setting the utimes.
This is done only if the file is WORM-Retained but for a WORM file
it will change the utimes.

Lazy auto commit:
Whenever a file gets created it will store the creation time of the
file or if a file already exists then any of the next unlink, link,
truncate or rename fops will set the current time as the start time
in an xattr. The next rename/unlink/truncate/link call will check for the
auto commit period and if is is expired, then it will automatically do
the state transition. If it is a normal file then it gets converted
to WORM-Retained state. If it is a WORM-Retained file and its retention
period is expired, then it gets converted to WORM state.

Added the volume set options for the WORM translator. It allows the users
to change the default values of auto-commit-period, default-retention-period,
retention-mode. To make use of the file-level WORM first we have to set the
'worm-file' option to 'on'. The files which are created when the worm-file
option is set on the volume will get WORM-Retained. Other files will work
as usual and will not be WORMed. The auto-commit-period, retention-mode,
and the default-retention-period values for the file will be set to the values
which are set on the volume when the file is created.

Added the tests to check the basic functionalities of the WORM/Retention feature.

Change-Id: I77bd9777f9395a944d76b5cc35a5b48a3c14d148
BUG: 1326308
Signed-off-by: karthik-us 
Reviewed-on: http://review.gluster.org/13429
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: volume set changes for lock migration

2016-05-02T01:05:03+00:00

Change-Id: I48c6f9cdda47503615ba65882acd5eedf0a70c89
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14024
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quota: mark tests/basic/quota.t as bad_test

2016-05-02T01:04:17+00:00

tests/basic/quota.t regularly leaves a core behind.
Some of the failed tests:
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16439/consoleFull
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16440/console

Change-Id: Iacf42c40f768dd2939994bbb87cc7ef366ee495f
BUG: 1332045
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14136
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quota: mark inode-quota-enforcing.t as bad_test

2016-05-01T14:37:21+00:00

tests/basic/inode-quota-enforcing.t regularly leaves a core behind.

Some of the failed tests:
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16387/console
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16390/console

Change-Id: I6c9c8f3503f033a3a10d585efa2de13caf4ac610
BUG: 1332021
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14135
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quota: mark quota-ancestry-building.t as bad_test

2016-05-01T14:37:17+00:00

tests/basic/quota-ancestry-building.t regularly fails like this:

    ./tests/basic/quota-ancestry-building.t (Wstat: 0 Tests: 34 Failed: 6)
      Failed tests:  14-19
    Files=1, Tests=34, 43 wallclock secs ( 0.05 usr  0.01 sys +  2.28 cusr  2.91 csys =  5.25 CPU)
    Result: FAIL
    ./tests/basic/quota-ancestry-building.t: 2 new core files
    End of test ./tests/basic/quota-ancestry-building.t

Some of the failed tests:
 * https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16385/console
 * https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16386/console

Change-Id: Ibbae908b4f752e195e7a7bdd10b102d7bcec312b
BUG: 1332020
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14134
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos