glusterfs.git/ufo, branch v3.4.6

object-storage: Store the lock file in /var/run/swift.

2013-02-07T22:02:19+00:00

* Openstack swift uses the /var/run/swift directory to store the pid files
  for all the servers.

* Also, added a script that would unmount the gluster client on a volume stop.

Change-Id: Ib5b9a2964987ca7696d9a2570f1f7af8490b2168
BUG: 861497
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4417
Reviewed-by: Peter Portante 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

object-storage: yield during directory tree walks

2013-02-04T16:44:19+00:00

Have the co-routine (greenlet) yield during long running directory
tree walks to avoid starving out other greenlets that might be
available to run.

A directory walk involves statting every file in the tree to get it
size. For large numbers of files, this can get expensive. So we yield
after every stat call, and after processing every directory.

Change-Id: I07f1dfeef6a09b5817e0c237ecc748c491d52a31
BUG: 894674
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4380
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Jeff Darcy 
Reviewed-by: Mohammed Junaid 
Tested-by: Mohammed Junaid

object-storage: use temp file optimization

2013-01-23T03:14:30+00:00

A file name '..' will hash to the same GlusterFS node
as a file named '', thus avoiding creation/deletion of linkfiles on a rename.

This is part of the work needed to address BZ 876660
(https://bugzilla.redhat.com/show_bug.cgi?id=876660).

Change-Id: I6f18c14b8eaa7a35c96f7e455ef3a19bee7dbde5
BUG: 876660
Signed-off-by: Mohammed Junaid 
Reviewed-on: http://review.gluster.org/4325
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Peter Portante

object-storage: add UFO unit tests

2013-01-17T08:08:44+00:00

Run the UFO unit tests following the main GlusterFS unit tests. The
UFO unit tests run under the Nose unit test running harness, requiring
the python-nose package to be installed. These unit tests run fast
enough that we could even consider running them on a commit hook run
by rfc.sh.

Note that they are not functional tests, as they don't require a real
Gluster file system to run.

Change-Id: I55972890c24be16fc588a6950cb6bfd539287bbe
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4333
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Anand Avati 
Tested-by: Anand Avati

object-storage: only open file when requested

2012-12-17T14:17:04+00:00

Only open a file when the caller of the constructor indicates that it
expects to have a file pointer to access in the "fp" field.

Found by Junaid during review http://review.gluster.org/4284, Patch
set 6.

We also bump the version number to mark a line in the set of changes
we want to perform a mini release with to double check for performance
regressions.

Change-Id: I13cf336bb519088890192ee111f4ece85a5982c4
BUG: 886730
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4303
Reviewed-by: Mohammed Junaid 
Tested-by: Gluster Build System

object-storage: don't sort object list, not req'd

2012-12-17T14:16:44+00:00

The REST API for listing objects in a container does not require that the list
of objects be sorted (the API for listing containers in an account does
require it). Since we can have thousands and thousands of objects in a
container, don't sort them when it is not required.

Change-Id: I6939ef3fec3ea3814a49e3a3046273304889831c
BUG: 887301
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4312
Reviewed-by: Mohammed Junaid 
Tested-by: Gluster Build System

object-storage: do not list directory for delete

2012-12-17T14:12:23+00:00

Do not list the entire directory contents of the directory containing the file
before deleting it. We have already verified the files existence constructing
the Gluster_DiskFile object.

This fixes a customer issue. See 885281
(https://bugzilla.redhat.com/show_bug.cgi?id=885281).

Change-Id: I4425c42af3e03624370c779d74b7f073e6eef024
BUG: 885281
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4302
Reviewed-by: Mohammed Junaid 
Tested-by: Gluster Build System

object-storage: apply upstream DiskFile refactor

2012-12-17T14:11:46+00:00

Apply the upstream DiskFile refactoring in ahead of its use to easiliy apply
the temp file optimization.

Change-Id: I2708733eed3d87759c70eb3d9e6cd74ef91d0c7b
BUG: 876660
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4288
Tested-by: Gluster Build System 
Reviewed-by: Mohammed Junaid 
Reviewed-by: Vijay Bellur

object-storage: a set of cleanups to DiskFile

2012-12-17T14:11:18+00:00

Here are a set of cleanups for things noticed while working on the
unit test coverage:

  * Remove unused constants, adding a constant for default disk chunk
    size

  * Document missing logger parameter from constructor

  * Add iter_hook paramater to constructor in anticipation of 1.7.4
    support

  * Add back meta_file field even though it is not used in our version
    of the constructor for paranoid compatibility

  * Remove is_valid field as it is not referenced anywhere, and it is
    not a field of the super class

  * Rename fields only used internally by DiskFile with leading
    underscores

  * Convert to using os.path.join() instead of hard coded '/'
    references

  * Use data_file field where possible

  * Assert that put_metadata() will only work when the file exists

  * Remove update_object() method since it is identical to
    put_metadata() and a direct write will suffice

  * Create the directory object only if it does not exist when a
    marker directory is requested

Change-Id: If207fed4c0f423e6bd3232e4507f87de877057c4
BUG: 876660
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4287
Tested-by: Gluster Build System 
Reviewed-by: Mohammed Junaid 
Reviewed-by: Vijay Bellur

object-storage: Initial unittest of DiskFile class

2012-12-17T14:10:54+00:00

If we had this ahead of time, we could have avoided the errors that
were encountered leading to the fix-account-mapping fix (see
http://review.gluster.org/4222).

This represents 100% coverage of the DiskFile module, but the coverage
report says otherwise, unfortunately. That is because the put() method
invokes eventlets during the test run, and coverage is not able to
accurately track the coverage lines properly. If one comments out the
"tpool.execute()" line in DiskFile.put() the coverage report then
reports 100% for the DiskFile module.

Additionally, we changed DiskFile.put() in four ways that should not
change its behavior:

  1. Comments were added to explain various code paths and mark
     potential issues / fixes

  2. It no longer returns a boolean value, matching the behavior of
     swift.obj.server.DiskFile.put()

  3. It no longer logs a message when it detects a directory that
     already exists, instead is raises an exception

     We believe this is okay because we cannot find a code path that
     would lead to his condition. As a result, it makes it easier to
     test all the code paths in that routine.

  4. It no longer logs a message when create_dir_object() fails, since
     create_dir_object() raises an exception on failure only

     We also modified create_dir_object() to not return a boolean as a
     result of the above behavior.

Note that by implementing these tests up to this point we found three
code paths that would have failed if encountered due to missing
imports. We also made changes to the DiskFile module to make it a bit
easier to test, also eliminating an extra stat system call when
deleting directory objects.

Change-Id: I3286de83c1ec7c5e8d6cab9354e1c4397cee7497
BUG: 870589
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4284
Tested-by: Gluster Build System 
Reviewed-by: Mohammed Junaid