glusterfs.git/tests/bugs, branch v3.7dev

dht: fix rename race

2014-07-17T17:30:56+00:00

If two clients try to rename the same file at the same time, we
sometimes end up with *no file at all* in either the old or new
location.  That's kind of bad.  The culprit seems to be some overly
aggressive cleanup code.  AFAICT, based on today's study of the code,
the intent of the changed section is to remove any linkfile we might
have created before the actual rename.  However, what we're removing
might not be our extra link.  If we're racing with another client that's
also doing a rename, it might be the only remaining link to the user's
data.  The solution, which is good enough to pass this test but almost
certainly still not complete, is to be more selective about when we do
this unlink.  Now, we only do it if we know that, at some point, we did
in fact create the link without error (notably ENOENT on the source or
EEXIST on the destination) ourselves.

Change-Id: I8d8cce150b6f8b372c9fb813c90be58d69f8eb7b
BUG: 1117851
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8269
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

mgmt/glusterd: do not check for snapd handle in restore if uss is disabled

2014-07-16T06:38:41+00:00

Change-Id: I01afe64685a5794cce9265580c6c5de57a045201
BUG: 1119582
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/8310
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

cli: Changed "rebalance start" output

2014-07-14T17:33:41+00:00

Change-Id: Ie87f1a2107b07a6e519ed894a74edf3b3e0a8340
BUG: 1063230
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/6946
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

Regression test portability: dd usage

2014-07-14T13:29:15+00:00

NetBSD, FreeBSD, and MacOS X dd(1) bs argument uses m for megabyte, while
Linux uses M. Use bs=1024k instead of bs=1M for better compatibility.

BUG: 764655
Change-Id: I603f57adbc9b31f6d634b918726437fbfce42e03
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8278
Tested-by: Gluster Build System 
Reviewed-by: Harshavardhana 
Reviewed-by: Justin Clift 
Reviewed-by: Vijay Bellur

features/gfid-access: error handling for entry creation

2014-07-14T13:26:55+00:00

Proceed with setattr() only on a successfull entry creation.
Winding a setattr() using a freshlyOC initiated inode would
most likely fail in one translator or the other (e.g. DHT
expecting the layout information to be set in the inode
context), which is the case if the inode was not looked up.

Therefore, gfid-access handles failure entry creations and
passes the _correct_ errno back to the client instead of
continuing with setattr() call and probably returning back
incorrect errno. Also, filling up inode->gfid is required
as the new inode is not looked up and ->gfid would be
certainely required for inode operations.

Change-Id: Ie92f5647a89bf558c07710ab0400bce69d59fc31
BUG: 1111490
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8260
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur

dht: support heterogeneous brick sizes

2014-07-12T16:20:52+00:00

Calculation of layouts now considers the size of each brick, so that
smaller bricks don't get an "unfair" share of allocations and start
returning ENOSPC while the larger bricks still have plenty of space.

The observation has been made that some clients might get ENOTCONN when
trying to fetch disk-size information, and end up calculating layouts
differently.  The following meta-observations can be made.

(1) This scenario is extremely unlikely in configurations with AFR.

(2) The most likely consequence of this scenario is that some files will
be placed sub-optimally by the client with the obsolete (non-weighted)
layout.  They'll still be found anyway, so this isn't a show stopper.

(3) Without this patch it's *guaranteed* that some files will be placed
sub-optimally, because any layout that fails to account for brick sizes
is sub-optimal.

(4) We shouldn't be doing fix-layout from two nodes simultaneously
anyway.  That's inefficient at best.  Any instances of such behavior are
separate bugs, which should be fixed separately.

(5) In the most extreme edge case, two nodes doing weighted and
non-weighted layout fixes could race and end up creating an internally
inconsistent layout.  This condition is still transient; it will be
detected and repaired automatically the next time anyone fetches the
layout.  (If it's not that's also a preexisting bug that can show up in
other contexts.)

In conclusion, it's not the purpose of this patch to fix bugs elsewhere
in DHT.  Its purpose is to make life incrementally better for users who
add new hardware with larger disks etc. than the older equipment.  It's
only one part of an ongoing process to improve layout management and
repair, all the way up to support for multiple hash rings or tiering.

Change-Id: I05eb6f9eface9cdaf8622e0260c8c7f29020447f
BUG: 1114680
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8093
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

tests: Fixed spurious failure in bug-887098-gmount-crash.t

2014-07-11T07:12:01+00:00

This script was trying to see if the mount process died by doing a
'ps ax' and a grep of the original pid in the results. After that
the pid of the first line returned by grep was compared to the
original pid.

This method can lead to false negatives because it's possible that
the original pid appears in some other part of the 'ps ax' list.

This patch uses get_mount_process_pid() from volume.rc to check if
the process is still alive.

Change-Id: I0285366e601a146793c47e9c1156a4bb36d6fcb3
BUG: 1092850
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8286
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog: prevent deadlock on thread cancellation

2014-07-11T07:08:57+00:00

helper threads (fsync, rollover) wake up periodically and perform
their respective operation under a lock (crt->lock). These threads
are also subjected to cancellation under some circumstance such as
disabling changelog. This is inherently dangerous when funtions
which are cancellation points for pthread_cancel(3) are used
in the locked region.

Consider this

         pthread_mutex_lock(&mutex);
         {
                /* ... */
                ret = fsync (fd);   <-- cancellation point
                /* ... */
         }
         pthread_mutex_unlock(&mutex);

A pthread_cancel(3) by another thread just before fsync(3) but
after pthread_mutex_lock(3) would result in the thread getting
cancelled when fsync(3) is invoked, thereby never unlocking the
mutex. Moreover, in case of changelog translator, the locked
region (under crt->lock in changelog-rt.c) is also the code
path for fop changelog updation. Therefore, unlocking the
mutex in thread cleanup handler (pthread_cleanup_pop(3)) might
prematurely release the mutex during fop updation path.

This patch fixes such problems existing in fsync and rollover
threads. Fix is to enter the locked region with cancellation
disabled and enable it after mutex unlock. Also, test for a
cancellation request early on in case none of the functions
are cancellation points.

Change-Id: I1795627a12827609c1da659d07fc1457ffa033de
BUG: 1110917
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8106
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System

glusterd/regression: Temp fix for spurious err

2014-07-10T18:12:00+00:00

As discussed in the mail,
Disabling the checking of snap brick status until
the investigation is done on the port bind issue.

Change-Id: I8854cee050de1b7f843e3d40631b6cb61fd8583e
BUG: 1112559
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/8259
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

socket/glusterd/client: enable SSL for management

2014-07-10T14:37:12+00:00

The feature is controlled by presence of the following file:

	/var/lib/glusterd/secure-access

See the comment near the definition of SECURE_ACCESS_FILE in glusterfs.h
for the rationale.  With this enabled, the following rules apply to
connections:

	UNIX-domain sockets never have SSL.

	Management-port sockets (both connecting and accepting, in
	daemons and CLI) have SSL based on presence of the file.

	Other IP sockets have SSL based on the existing client.ssl and
	server.ssl volume options.

Transport multi-threading is explicitly turned off in glusterd (it would
otherwise be turned on when SSL is) due to multi-threading issues.
Tests have been elided to avoid risk of leaving a file which will cause
all subsequent tests to run with management SSL still enabled.

IMPLEMENTATION NOTE
The implementation is a bit messy, and consists of two stages.  First we
decide whether to set the relevant fields in our context structure, based
on presence of the sentinel file OR a command-line override.  Later we
decide whether a particular connection should actually use SSL, based on the
context flags plus what kind of connection we're making[1] and what kind of
daemon we're in[2].

[1] inbound, outbound to glusterd port, other outbound
[2] glusterd, glusterfsd, other

TESTING NOTE
Instead of just running one special test for this feature, the ideal
would be to run all tests with management SSL enabled.  However, it
would be inappropriate or premature to set up an optional feature in the
patch itself.  Therefore, the method of choice is to submit a separate
patch on top, which modifies "cleanup" in include.rc to recreate the
secure-access file and associated SSL certificate/key files before each
test.

Change-Id: I0e04d6d08163893e24ec8c031748c5c447d7f780
BUG: 1114604
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/8094
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur