glusterfs.git/ufo/test/unit/common, branch v6.4

object-storage: final removal of ufo code

2013-05-10T18:03:55+00:00

See https://git.gluster.org/gluster-swift.git for the new location of
the Gluster-Swift code.

With this patch, no OpenStack Swift related RPMs are constructed.

This patch also removes the unused code that references the
user.ufo-test xattr key in the DHT translator.

Change-Id: I2da32642cbd777737a41c5f9f6d33f059c85a2c1
BUG: 961902 (https://bugzilla.redhat.com/show_bug.cgi?id=961902)
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4970
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Luis Pabon 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

object-storage: turn off stat() for container list

2013-04-12T20:48:43+00:00

Turn of stat() system calls used to fetch the file size during a
container listing operation since these system calls can swamp Gluster
and the result is most often not used.

When a GET or HEAD request is made on a container, stat() system calls
are made during the Python standard library method, os.walk, to
determine if a given directory entry is another directory to recurse
into, and then utils._update_list() will stat() each file to get it
size, and finally utils.get_container_details_from_fs() will stat()
each directory encountered.

For most installations we have seen so far, we don't need the
container listing to accurately return the size of all the objects in
the container, so we can reduce the number of stat() system calls by
not fetching the size of the object.

For now, turn it off by default, and provide an /etc/swift/fs.conf
configuration parameter to turn it back on:

  accurate_size_in_listing = yes

The default for the above is "no".

Change-Id: I7dde11e14bb32ecafa3eabb08852f1ffc4366b35
BUG: 903396
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4787
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Anand Avati

object-storage: use tox for unit tests; fix em too

2013-04-05T18:20:37+00:00

Add the ability to use tox for unit tests, since it helps us solve the
problem of supporting multiple branches that require different
versions of dependencies, and allows us to possibly support multiple
versions of python in the future.

Also fix the code to work with pre-grizzly environments, by not
requiring the constraints backport.

Also fixed the xattr support to work with both pyxattr and xattr
modules.

And fixed the ring tests to also work without a live /etc/swift
directory.

BUG: 948657 (https://bugzilla.redhat.com/show_bug.cgi?id=948657)
Change-Id: I2be79c8ef8916bb6552ef957094f9186a963a068
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4781
Reviewed-by: Alex Wheeler 
Tested-by: Alex Wheeler 
Reviewed-by: Anand Avati

object-storage: Import missing sys and errno modules.

2013-04-04T16:57:26+00:00

Import the missing modules and implemented unit test case for Glusterfs module.
Thanks to Paul Smith for pointing it out.

Change-Id: Ib04202aa0ae05c4da2ebbf11f87d6accc778f827
BUG: 905946
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4758
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Anand Avati

Adds missing functions to ring.py, and more thorough tests.

2013-04-02T23:09:44+00:00

Situation:
The function get_part_nodes is being called by Openstack-Swift's proxy/controllers/base.py:
        https://github.com/openstack/swift/blob/1.7.4/swift/proxy/controllers/base.py#L410
        https://github.com/openstack/swift/blob/1.7.6/swift/proxy/controllers/base.py#L447
As this was not implemented in the current GlusterFS version of ring.py, it was calling swift's
original get_part_nodes, which would often return the incorrect node, resulting in the incorrect
drive being associated with a request.
There is another function that the original ring.py implements -- get_other_nodes, which has to
do with replication.  Since GlusterFS is handling replication, this function should never be
called.  However, in the interest of completeness, that function is also being replaced.

Code changes:
The two functions, get_part_nodes, and get_other_nodes have been implemented to override the
default functions, and get_nodes has been updated to store information in self vars, about the
account being operated on, as the two new functions are not called with that info, and get_nodes
appears to always be called first.
The code has be refactored to all call _get_part_nodes, much like swift has refactored their code.

Reason for implementation this way:
I didn't see a better way to do it, but am open to suggestions.

Test cases:
A mock ring is created with two different devices, test and iops

test_first_device: Ensure that the first device, test, is returned for both get_nodes, and
        get_part_node, and get_more_nodes returns volume_not_in_ring.
test_invalid_device: Ensure that a request for a non-existant device returns volume_not_in_ring.
test_second_device: Same as test_first_device, but for the second device, iops instead of test.
test_second_device_with_reseller_prefix: Test that calling with the reseller prefix, AUTH_ will
        still return the correct data.

Change-Id: I2f3d526934a47b01e9c065d0edf0fbf06f300369
BUG: 924792
Signed-off-by: Alex Wheeler 
Reviewed-on: http://review.gluster.org/4748
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Jeff Darcy 
Reviewed-by: Anand Avati

object-storage: Removed the redundant REMOTE_CLUSTER option.

2013-03-21T21:07:27+00:00

Gluster cli uses the remote-host option to connect to the glusterd
and by default it uses localhost to connect to glusterd. So, UFO code
will use the remote-host option everytime to connect to the glusterd.

Change-Id: I5a684d3c43fe9bdc9cc0b7c472a9d8145f9e1fd4
BUG: 878663
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4690
Reviewed-by: Peter Portante 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

object-storage: Fixing the errors and warnings in unittest.

2013-03-07T11:00:10+00:00

Change-Id: Id22c968aefd82c4b62445b3ecc93cbabc2b35ffc
BUG: 887301
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4394
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Peter Portante 
Tested-by: Gluster Build System

object-storage: Restoring multi volume support in UFO.

2013-03-07T08:55:34+00:00

* Currently, the users of UFO are restricted to use just one volume
  at any given point of time. This patch removes this limitation.

* The usage of gluster-swift-gen-builders has also changed. With this
  commit the users should mention the list of volumes that they want
  to expose through UFO. So, only the volumes mentioned during the
  ring file generation can be accessed.

  Usage: gluster-swift-gen-builders  []...

This is an intermediate fix until we remove the account, container and
object server processes. Once we have this frame work running, it will
completely eliminate the ring files.

Change-Id: I9ad3808519fec9c7c60ad846c4f8b653117a8337
BUG: 909053
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4485
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Peter Portante

object-storage: Use the wrapper functions provided by fs_utils.py to make

2013-02-12T02:00:56+00:00

  system calls.

 The set of changes:
 * Unit test cases for fs_utils.py

 * Replaced os.path with os_path

 * Implemented wrapper functions do_write, do_chmod, etc in fs_utils.py

 * Replaced os. with the wrapper functions.

Change-Id: I770da878e83eda6b98e49d70193990406a2642a7
BUG: 887301
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4360
Reviewed-by: Peter Portante 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

object-storage: use temp file optimization

2013-01-23T03:14:30+00:00

A file name '..' will hash to the same GlusterFS node
as a file named '', thus avoiding creation/deletion of linkfiles on a rename.

This is part of the work needed to address BZ 876660
(https://bugzilla.redhat.com/show_bug.cgi?id=876660).

Change-Id: I6f18c14b8eaa7a35c96f7e455ef3a19bee7dbde5
BUG: 876660
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4325
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Peter Portante