glusterfs.git/tests, branch v3.9dev

WORM/Retention Translator: Implementation of file level WORM

2016-05-02T01:05:22+00:00

To activate the file level worm feature, the features.read-only and
features.worm options should be switched "off" on the volume and
the features.worm-file-level should be switched "on". Both read-only
and worm or worm-file-level cannot be switched "on" together. The
files which are created when the worm-file-level option is set on the
volume will have their own retention profile.

If both worm and worm-file-level are "on" at that time the worm
which is the volume level worm will have priority over file level
worm. If worm-file level is switched "off" after some time and the
read-only option is switched "on" then read-only will have priority.

The current implementation allows the users to manually transmit
a file to a WORM-Retained state by removing all the write bits of
the file using the chmod command. The file will have a retention
profile which contains the state of the file, mode of retention,
and the default retention time.

The file will be made WORM-Retained for a default of 120 seconds
during which it will be immutable and undeletable and it sets the
atime of the file to the time till which it is retained.
After that period if any fop request comes for that file, will
make the transition from WORM-Retained state to WORM state, where
the file will be immutable but deletable and, it will reset
the atime to the actual atime of the file. If a WORM file needs
to be made undeletable again, it can be done by using the chmod
command with all the write bits removed.

There are two modes of retention:
1. Relax: where the retention time of a WORM-Retained file can be
   increased or decreased.
2. Enterprise: where the retention time of a WORM-Retained file
   can be increased but not be decreased.
Whenever a utime change(touch -a, -t, ...)request comes for a
file it checks the mode of retention before setting the utimes.
This is done only if the file is WORM-Retained but for a WORM file
it will change the utimes.

Lazy auto commit:
Whenever a file gets created it will store the creation time of the
file or if a file already exists then any of the next unlink, link,
truncate or rename fops will set the current time as the start time
in an xattr. The next rename/unlink/truncate/link call will check for the
auto commit period and if is is expired, then it will automatically do
the state transition. If it is a normal file then it gets converted
to WORM-Retained state. If it is a WORM-Retained file and its retention
period is expired, then it gets converted to WORM state.

Added the volume set options for the WORM translator. It allows the users
to change the default values of auto-commit-period, default-retention-period,
retention-mode. To make use of the file-level WORM first we have to set the
'worm-file' option to 'on'. The files which are created when the worm-file
option is set on the volume will get WORM-Retained. Other files will work
as usual and will not be WORMed. The auto-commit-period, retention-mode,
and the default-retention-period values for the file will be set to the values
which are set on the volume when the file is created.

Added the tests to check the basic functionalities of the WORM/Retention feature.

Change-Id: I77bd9777f9395a944d76b5cc35a5b48a3c14d148
BUG: 1326308
Signed-off-by: karthik-us 
Reviewed-on: http://review.gluster.org/13429
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

glusterd: volume set changes for lock migration

2016-05-02T01:05:03+00:00

Change-Id: I48c6f9cdda47503615ba65882acd5eedf0a70c89
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14024
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quota: mark tests/basic/quota.t as bad_test

2016-05-02T01:04:17+00:00

tests/basic/quota.t regularly leaves a core behind.
Some of the failed tests:
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16439/consoleFull
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16440/console

Change-Id: Iacf42c40f768dd2939994bbb87cc7ef366ee495f
BUG: 1332045
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14136
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quota: mark inode-quota-enforcing.t as bad_test

2016-05-01T14:37:21+00:00

tests/basic/inode-quota-enforcing.t regularly leaves a core behind.

Some of the failed tests:
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16387/console
* https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16390/console

Change-Id: I6c9c8f3503f033a3a10d585efa2de13caf4ac610
BUG: 1332021
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14135
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

quota: mark quota-ancestry-building.t as bad_test

2016-05-01T14:37:17+00:00

tests/basic/quota-ancestry-building.t regularly fails like this:

    ./tests/basic/quota-ancestry-building.t (Wstat: 0 Tests: 34 Failed: 6)
      Failed tests:  14-19
    Files=1, Tests=34, 43 wallclock secs ( 0.05 usr  0.01 sys +  2.28 cusr  2.91 csys =  5.25 CPU)
    Result: FAIL
    ./tests/basic/quota-ancestry-building.t: 2 new core files
    End of test ./tests/basic/quota-ancestry-building.t

Some of the failed tests:
 * https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16385/console
 * https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16386/console

Change-Id: Ibbae908b4f752e195e7a7bdd10b102d7bcec312b
BUG: 1332020
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14134
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

disperse: mark bug-1304988.t as bad_test

2016-05-01T14:37:12+00:00

tests/bugs/disperse/bug-1304988.t regularly fails like this:

    [07:25:34] Running tests in file ./tests/bugs/disperse/bug-1304988.t
    tar: Removing leading `/' from member names
    ./tests/bugs/disperse/bug-1304988.t ..
    1..9
    ok 1, LINENUM:19
    ok 2, LINENUM:20
    ok 3, LINENUM:21
    ok 4, LINENUM:22
    ok 5, LINENUM:23
    ok 6, LINENUM:24
    Started rename 26042
    ok 7, LINENUM:33
    ok 8, LINENUM:34
    not ok 9 Got "in progress ::" instead of "completed", LINENUM:37
    FAILED COMMAND: completed rebalance_status_field patchy
    rebalance done...
    Failed 1/9 subtests

    Test Summary Report
    -------------------
    ./tests/bugs/disperse/bug-1304988.t (Wstat: 0 Tests: 9 Failed: 1)
      Failed test:  9
    Files=1, Tests=9, 374 wallclock secs ( 0.02 usr  0.00 sys + 34.12 cusr 15.33 csys = 49.47 CPU)
    Result: FAIL
    End of test ./tests/bugs/disperse/bug-1304988.t

Some of the failed tests:
 * https://build.gluster.org/job/rackspace-regression-2GB-triggered/20261/console
 * https://build.gluster.org/job/rackspace-regression-2GB-triggered/20262/console
 * https://build.gluster.org/job/rackspace-regression-2GB-triggered/20263/console

Change-Id: I0350a720f57f536d283c088e040680bbbe9f8f34
BUG: 1332022
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14133
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

tests: Add test cases for add/replace brick with granular entry sh

2016-05-01T13:43:19+00:00

Most of the tests borrowed from Anuradha's original replace-brick
and add-brick tests under tests/basic/afr/.

Change-Id: I874c04a6af3223e07aa6099b818ff502b6ba2a15
BUG: 1269461
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/14130
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System

quota/glusterd: enhance quota enable and disable process

2016-04-29T10:11:35+00:00

Previously quota crawl was done from the single mount point,
this is very slow process if there are huge number of files exists
in the volume

This RFE will now spawn crawl process for each brick in the
volume, and  files are looked in parallel independently for each
brick. This improves the speed of crawling process for
entire files-system

This patch also fixes below problem
* Previously, mountdir was created under '/tmp'.
  If someone tries to cleanup '/tmp'/ directory
  then it is very dangerous that we loose volume data
  So create a mount point under /var/run/gluster/tmp
  instead

* Previously, file-system crawl is performed from all the nodes,
  which is a redundant operation and performance will degrade
  The problem is fixed with this patch

Change-Id: Icabedeb44182139ace9c8106793803122388cab8
BUG: 1290766
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/12952
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Manikandan Selvaganesh 
Reviewed-by: Atin Mukherjee

features/trash: wind mkdir with special pid

2016-04-27T21:24:48+00:00

Recent changes done w.r.t handling of mkdir calls in posix translator
resulted in crashing the brick process from trash translator. This was
due to the changes made in posix translator to return EPERM for every
mkdir calls without 'gfid-req' set in dictionary. In order to avoid
gfid mismatches during directory creation from brick side trash
translator does not set 'gfid-req'. This patch is to have an exemption
for trash based on a special pid set for those mkdir calls originating
from trash translator and to reset it in callback.

This patch also includes a small optimization to the existing test case
for trash feature.

Change-Id: I59f084ac875e54342ecf2bffa6e43ebd84814153
BUG: 1317361
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/13776
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

cluster/afr: Do not fsync when durability is off

2016-04-27T15:06:35+00:00

BUG: 1329501
Change-Id: Id402c20f2fa19b22bc402295e03e7a0ea96b0c40
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/14048
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy