glusterfs.git/tests, branch v3.7dev

dht: fix rename race

2014-07-17T17:30:56+00:00

If two clients try to rename the same file at the same time, we
sometimes end up with *no file at all* in either the old or new
location.  That's kind of bad.  The culprit seems to be some overly
aggressive cleanup code.  AFAICT, based on today's study of the code,
the intent of the changed section is to remove any linkfile we might
have created before the actual rename.  However, what we're removing
might not be our extra link.  If we're racing with another client that's
also doing a rename, it might be the only remaining link to the user's
data.  The solution, which is good enough to pass this test but almost
certainly still not complete, is to be more selective about when we do
this unlink.  Now, we only do it if we know that, at some point, we did
in fact create the link without error (notably ENOENT on the source or
EEXIST on the destination) ourselves.

Change-Id: I8d8cce150b6f8b372c9fb813c90be58d69f8eb7b
BUG: 1117851
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8269
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

make snapview-server more compatible with NFS server

2014-07-16T09:27:50+00:00

* There was no handle based API for listxattr. With this change, glfs_h_getxattrs
  also handles the listxattr functionality by checking whether the name is NULL
  or not (like posix). But all the gfapi functions for listxattr
  (glfs_h_getxattrs AND glfs_listxattr AND glfs_flistxattr) returns the names of
  the xattrs in a buffer provided by the caller. But snapview-server has to
  return the list of xattrs in a dict itself (similar to posix xlator). But
  the buffer just contains the names of the xattrs. So for each xattr, a zero
  byte value is set (i.e. "") into the dict and sent back. Translators which
  do xattr caching (as of now md-cache which caches selinux and acl related
  xattrs) should not cache those xattrs whose value is a zero byte data ("").
  So made changes in md-cache to ignore zero byte values.

* NFS server was not linking the inodes to inode table in readdirp. This was
  leading to applications getting errors. The below set of operations would
  lead to applications getting error
  1) ls -l in one of the snaopshots (snapview-server would generate gfids for
     each entry on the fly and link the inodes associated with those entries)
  2) NFS server upon getting readdirp reply would not link the inodes of the
     entries. But it used to generate filehandles for each entry and associate
     the gfid of that entry with the filehandle and send it as part of the
     reply to nfs client.
  3) NFS client would send the filehandle of one of those entries when some
     activity is done on it.
  4) NFS server would not be able to find the inode for the gfid present in the
     filehandle (as the inode was not linked) and would go for hard resolution
     by sending a lookup on the gfid by creating a new inode.
  5) snapview-client will not able to identify whether the inode is a real inode
     existing in the main volume or a virtual inode existing in the snapshots
     as there would not be any inode context.
  6) Since the gfid upon which lookup is sent is a virtual gfid which is not
     present in the disk, lookup would fail and the application would get an
     error.

  To handle above situation, now nfs server also does inode linking in readdirp.

Change-Id: Ibb191408347b6b5f21cff72319ccee619ea77bcd
BUG: 1115949
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/8230
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

mgmt/glusterd: do not check for snapd handle in restore if uss is disabled

2014-07-16T06:38:41+00:00

Change-Id: I01afe64685a5794cce9265580c6c5de57a045201
BUG: 1119582
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/8310
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

tests: make dd less noisy

2014-07-16T06:00:48+00:00

Also fixed one case in quota.t where error output is expected.  There
are probably other similar cases which can be fixed separately.

Change-Id: If80fad0d9fcff6f8ca91d00f4f7b2d5f3fef4256
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8298
Tested-by: Gluster Build System 
Reviewed-by: Humble Devassy Chirammal 
Reviewed-by: Prashanth Pai 
Reviewed-by: Vijay Bellur

Add possibility to run single tests

2014-07-16T05:59:44+00:00

Change-Id: I9282b711c09611bd0fd4cc814f3ec34aa67d10c0
BUG: 1118453
Signed-off-by: Anders Blomdell 
Reviewed-on: http://review.gluster.org/8291
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Reviewed-by: Humble Devassy Chirammal 
Reviewed-by: Vijay Bellur

cli: Changed "rebalance start" output

2014-07-14T17:33:41+00:00

Change-Id: Ie87f1a2107b07a6e519ed894a74edf3b3e0a8340
BUG: 1063230
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/6946
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

test: utility to create files with a specific gfid

2014-07-14T13:42:50+00:00

python test utility that uses setxattr() interface to
create files with a predefined gfid (which is also used
by geo-replication). this utility aids in writing test
cases for gfid-access translator.

Change-Id: Ib11248844a094cc02dae74d969f2e9212f94ede4
BUG: 1111490
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8275
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Regression test portability: dd usage

2014-07-14T13:29:15+00:00

NetBSD, FreeBSD, and MacOS X dd(1) bs argument uses m for megabyte, while
Linux uses M. Use bs=1024k instead of bs=1M for better compatibility.

BUG: 764655
Change-Id: I603f57adbc9b31f6d634b918726437fbfce42e03
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8278
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Reviewed-by: Justin Clift 
Reviewed-by: Vijay Bellur

features/gfid-access: error handling for entry creation

2014-07-14T13:26:55+00:00

Proceed with setattr() only on a successfull entry creation.
Winding a setattr() using a freshlyOC initiated inode would
most likely fail in one translator or the other (e.g. DHT
expecting the layout information to be set in the inode
context), which is the case if the inode was not looked up.

Therefore, gfid-access handles failure entry creations and
passes the _correct_ errno back to the client instead of
continuing with setattr() call and probably returning back
incorrect errno. Also, filling up inode->gfid is required
as the new inode is not looked up and ->gfid would be
certainely required for inode operations.

Change-Id: Ie92f5647a89bf558c07710ab0400bce69d59fc31
BUG: 1111490
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8260
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur

dht: support heterogeneous brick sizes

2014-07-12T16:20:52+00:00

Calculation of layouts now considers the size of each brick, so that
smaller bricks don't get an "unfair" share of allocations and start
returning ENOSPC while the larger bricks still have plenty of space.

The observation has been made that some clients might get ENOTCONN when
trying to fetch disk-size information, and end up calculating layouts
differently.  The following meta-observations can be made.

(1) This scenario is extremely unlikely in configurations with AFR.

(2) The most likely consequence of this scenario is that some files will
be placed sub-optimally by the client with the obsolete (non-weighted)
layout.  They'll still be found anyway, so this isn't a show stopper.

(3) Without this patch it's *guaranteed* that some files will be placed
sub-optimally, because any layout that fails to account for brick sizes
is sub-optimal.

(4) We shouldn't be doing fix-layout from two nodes simultaneously
anyway.  That's inefficient at best.  Any instances of such behavior are
separate bugs, which should be fixed separately.

(5) In the most extreme edge case, two nodes doing weighted and
non-weighted layout fixes could race and end up creating an internally
inconsistent layout.  This condition is still transient; it will be
detected and repaired automatically the next time anyone fetches the
layout.  (If it's not that's also a preexisting bug that can show up in
other contexts.)

In conclusion, it's not the purpose of this patch to fix bugs elsewhere
in DHT.  Its purpose is to make life incrementally better for users who
add new hardware with larger disks etc. than the older equipment.  It's
only one part of an ongoing process to improve layout management and
repair, all the way up to support for multiple hash rings or tiering.

Change-Id: I05eb6f9eface9cdaf8622e0260c8c7f29020447f
BUG: 1114680
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8093
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur