glusterfs.git/tests/features, branch v3.7.3

dht: Add lookup-optimize configuration option for DHT

2015-06-05T09:16:13+00:00

Currently with commit 4eaaf5 a mixed version cluster would
have issues if lookup-uhashed is set to auto, as older clients
would fail to validate the layouts if newer clients (i.e 3.7 or
upwards) create directories. Also, in a mixed version cluster
rebalance daemon would set commit hash for some subvolumes and
not for the others.

This commit fixes this problem by moving the enabling of the
functionality introduced in the above mentioned commit to a
new dht option. This option also has a op_version of 3_7_1
thereby preventing it from being set in a mixed version
cluster. It brings in the following changes,
- Option can be set only if min version of the cluster is
3.7.1 or more
- Rebalance and mkdir update the layout with the commit hashes
only if this option is set, hence ensuring rebalance works in a
mixed version cluster, and also directories created by newer
clients do not cause layout errors when read by older clients
- This option also supersedes lookup-unhased, to enable the
optimization for lookups more deterministic and not conflict
with lookup-unhashed settings.

Option added is cluster.lookup-optimize, which is a boolean.

Usage: # gluster volume set VOLNAME cluster.lookup-optimize on

Change-Id: Ifd1d4ce3f6438fcbcd60ffbfdbfb647355ea1ae0
BUG:  1225940
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/10976
Tested-by: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

tests: write log for glupy.t to the standard log directory

2015-05-26T17:35:48+00:00

There is (or was) a spurious test in glupy.t for which a separate log
file was setup. The directory where the log is saved, is not available
on NetBSD and this causes glupy.t to always fail, without a log.

Instead of hard-coding the path to the log, use "gluster --print-logdir"
to provide a LOGDIR environment variable. glupy.t now writes the log to
an existing directory.

Cherry picked from commit cfca748fef28170377b868871b0dd980c1f151f6:
> BUG: 1163543
> Change-Id: Ifa73198d06fa267856d0da9d25a4380329909124
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10801
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: Ifa73198d06fa267856d0da9d25a4380329909124
BUG: 1225077
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10920
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

dht: make lookup-unhashed=auto do something actually useful

2015-05-10T04:55:09+00:00

The key concept here is to determine whether a directory is "clean" by
comparing its last-known-good topology to the current one for the
volume.  These are stored as "commit hashes" on the directory and the
volume root respectively.  The volume's commit hash changes whenever a
brick is added or removed, and a fix-layout is done.  A directory's
commit hash changes only when a full rebalance (not just fix-layout)
is done on it.  If all bricks are present and have a directory
commit hash that matches the volume commit hash, then we can assume
that every file is in its "proper" place. Therefore, if we look for
a file in that proper place and don't find it, we can assume it's not
on any other subvolume and *safely* skip the global (broadcast to all)
lookup.

Change-Id: Id6ce4593ba1f7daffa74cfab591cb45960629ae3
BUG: 1220064
Reviewed-on-master: http://review.gluster.org/#/c/7702/
Signed-off-by: Jeff Darcy 
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/10729
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Vijay Bellur

tests: Fix spurious failures

2015-05-06T06:02:17+00:00

        Backport of: http://review.gluster.org/10480
                     http://review.gluster.org/10487
                     http://review.gluster.org/10488
                     http://review.gluster.com/10540

- Use REBALANCE_TIMEOUT in EXPECT_WITHIN
- Use fdatasync to prevent write - behind from giving success
- Add logfile to glupy
- Check aux umount is unmounted for quota tests
- Disable flush behind so that file will be closed

Change-Id: Ia2447176273557738b7a71cc25dd9b9effc16d58
BUG: 1218485
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10543
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Tests: fix spurious failures in trash.t

2015-04-26T08:54:30+00:00

Fix timing problems that cause rare spurious failures in trash.t

Backport of I673e033b53b6b4bb993c22fadbdcee725b2c1e96

BUG: 1212676
Change-Id: I319f94b7e099e189c08543da526d90d36318fe4b
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10374
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

Tests: trash.t portability fixes

2015-04-26T04:51:25+00:00

        Backport of http://review.gluster.org/#/c/10327/

Fix various portability problems in trash.t
- work around timing issues
- avoid wildcard usage only supported by bash

Original patch from Anoop C S and Jiffin Tony Thottan.

Removed LONGER_HEAL_TIMEOUT from previous patch as it
seems to run fine without it now.

BUG: 1212676
Change-Id: I0f6f484209ef4db7e0a7b733b863927cb248e73e
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10346
Reviewed-by: Vijay Bellur

Tests: portability fixes for ipc.t

2015-03-31T06:45:10+00:00

This fixes portability problems in ipc.t so that it can run on NetBSD:

1) EOPNOTSUPP value is OS-dependent. Learn it from system headers
   instead of hard-coding it in the script

2) liglusterfs embbeds its own UUID implementation. The function name
   may be the same as in built(in implementation from libc, but with
   different prototype. In that case, we must make sure python will
   use libglusterfs's version, otherwise we will crash in libc's UUID
   code. Since dlopen() does not make any guarantee on what symbol
   will be used, me need to preload libglusterfs when loading python.
   This is done using LD_PRELOAD.

3) In python code we need to load with RTLD_GLOBAL global in order
   to have dependencies loaded

4) Python's ctypes.util.find_library does not lookup LD_LIBRARy_PATH
   and may therefore miss the library. On failure, retry with less
   portable but more reliable explicit name

BUG: 1129939
Change-Id: I024cdfd03a5a42a8ec23de38a99e7349aba92ea8
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/9944
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-by: Vijay Bellur

Tests: portability fixes

2015-03-30T12:32:56+00:00

Fix various portability problems in mount-nfs-auth.t,
quota-ancestry-building.t and trash.t:

- dd bs=1M is not portable, use dd bs=1024k instead
- dd bs=1MB is not portable iether, use dd bs=1000000 instead
- After restarting NFS service, wait for it to become available
- After killing a process, wait for it to terminate
- BSD awk does not accept a=b="", use a=""; b="" instead
- NetBSD displays the original program name in paenthesis at the end
  of ps output. Strip it using sed 's/ *([^()]*)$//' is we want just
  the command
- Do no use umount $N0, which leads to many troubles solved by
  EXPECT_WITHIN $UMOUNT_TIMEOUT "Y" umount_nfs $N0
- The -p option for mkdir must be before the directory name
- du -b is not portable. Use ls -l instead.

BUG: 1129939
Change-Id: I3d44a10a37d47ebb6a263c206566487e3ffb85d8
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10033
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Anoop C S 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

tests: fix rebalance_completed

2015-03-26T18:44:52+00:00

This was causing spurious faiures in bug-884455.t and possibly
elsewhere.

Change-Id: Iad6b7515ca0c7c485300f79dcd2477efc76877f8
BUG: 1163543
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/9994
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

every/where: add GF_FOP_IPC for inter-translator communication

2015-03-17T14:02:15+00:00

Several features - e.g. encryption, erasure codes, or NSR - involve
multiple cooperating translators which sometimes need a "private" means
of communication amongst themselves.  Historically we've used virtual or
synthetic xattrs, but that's not very elegant and clutters up the
getxattr/setxattr path which must also handle real xattr requests.  This
new fop should address that.

The only argument is an int32_t "op" which should be recognized by the
target translator.  It is recommended that translators using these
feature follow some convention regarding the ops that they define, to
avoid conflicts.  Using a hash of the target translator's type string as
a base for a series of ops would probably be a good start.  Any other
information can be passed in both directions using xdata.

The default behavior for this fop, as with any other, is to pass through
to FIRST_CHILD.  That makes use of this fop "transparent" to other
translators that were written before it existed, but it also means that
it only really works with pass-through translators.  If a routing
translator (such as DHT) or a fan-out translator (such as AFR) is
involved, the IPC might not reach its intended destination unless those
translators are modified to forward IPC fops along all paths.

If an IPC gets all the way to storage/posix it is considered an error,
much like an uncaught exception.  We don't actually *do* anything in
that case, but we do log it send back an EOPNOTSUPP error.  This makes
the "unrecognized opcode" condition distinguishable from the "no IPC
support" condition (which would yield an RPC error instead) so clients
can probe for the presence of a handler for their own favorite opcode
and either use that or use old-school xattrs depending on the result.

BUG: 1158628
Signed-off-by: Venky Shankar 
Signed-off-by: Jeff Darcy 
Change-Id: I84af1b17babe5b30ec03ecf027ae37d09b873968
Reviewed-on: http://review.gluster.org/8812
Reviewed-by: Vijay Bellur