| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
|
|
|
|
|
|
|
|
| |
Change-Id: Id4dbf684b1e57ec928b30bc2d9cb2e2b7e0f2389
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/4999
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
We also enable XML output for coverage reports.
Change-Id: Ide5057f089f7b6271910be042b510d3b4e149a31
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4984
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix failing unit test by properly setting the errno field value in the
mock object.
Change-Id: I86a60c74d7d88736612f84e0d00ab4d5a94b533c
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4967
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Additionally, we drop the "ufo" references from setup.py, spec file
and README, and add the HISTORY file describing how the repo was
initially created. We also update the RPM spec file to use the name
"gluster-for-swift" to avoid colliding with existing RPM names from
RHS 2.0 (the spec file's description was also updated, along with
the version number).
Change-Id: If804224a94208d57896e4189c63736ffc9e01d5e
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4966
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
|
|\
| |
| |
| | |
Signed-off-by: Peter Portante <peter.portante@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Provide more common configuration defaults, as seen and recommended across
many large customer installations, as well as provide some guidance on how to
set the parameters.
See BZ 904629 (https://bugzilla.redhat.com/show_bug.cgi?id=904629).
Change-Id: Id9f20aafd75f2a0b589c67654dce87534bf80c33
BUG: 904629
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4789
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I4fa1a7b9214c6897459a99bedde80e84eb2b9cbc
BUG: 904370
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4788
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Turn of stat() system calls used to fetch the file size during a
container listing operation since these system calls can swamp Gluster
and the result is most often not used.
When a GET or HEAD request is made on a container, stat() system calls
are made during the Python standard library method, os.walk, to
determine if a given directory entry is another directory to recurse
into, and then utils._update_list() will stat() each file to get it
size, and finally utils.get_container_details_from_fs() will stat()
each directory encountered.
For most installations we have seen so far, we don't need the
container listing to accurately return the size of all the objects in
the container, so we can reduce the number of stat() system calls by
not fetching the size of the object.
For now, turn it off by default, and provide an /etc/swift/fs.conf
configuration parameter to turn it back on:
accurate_size_in_listing = yes
The default for the above is "no".
Change-Id: I7dde11e14bb32ecafa3eabb08852f1ffc4366b35
BUG: 903396
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4787
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add the ability to use tox for unit tests, since it helps us solve the
problem of supporting multiple branches that require different
versions of dependencies, and allows us to possibly support multiple
versions of python in the future.
Also fix the code to work with pre-grizzly environments, by not
requiring the constraints backport.
Also fixed the xattr support to work with both pyxattr and xattr
modules.
And fixed the ring tests to also work without a live /etc/swift
directory.
BUG: 948657 (https://bugzilla.redhat.com/show_bug.cgi?id=948657)
Change-Id: I2be79c8ef8916bb6552ef957094f9186a963a068
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4781
Reviewed-by: Alex Wheeler <wheelear@gmail.com>
Tested-by: Alex Wheeler <wheelear@gmail.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Import the missing modules and implemented unit test case for Glusterfs module.
Thanks to Paul Smith for pointing it out.
Change-Id: Ib04202aa0ae05c4da2ebbf11f87d6accc778f827
BUG: 905946
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Situation:
The function get_part_nodes is being called by Openstack-Swift's proxy/controllers/base.py:
https://github.com/openstack/swift/blob/1.7.4/swift/proxy/controllers/base.py#L410
https://github.com/openstack/swift/blob/1.7.6/swift/proxy/controllers/base.py#L447
As this was not implemented in the current GlusterFS version of ring.py, it was calling swift's
original get_part_nodes, which would often return the incorrect node, resulting in the incorrect
drive being associated with a request.
There is another function that the original ring.py implements -- get_other_nodes, which has to
do with replication. Since GlusterFS is handling replication, this function should never be
called. However, in the interest of completeness, that function is also being replaced.
Code changes:
The two functions, get_part_nodes, and get_other_nodes have been implemented to override the
default functions, and get_nodes has been updated to store information in self vars, about the
account being operated on, as the two new functions are not called with that info, and get_nodes
appears to always be called first.
The code has be refactored to all call _get_part_nodes, much like swift has refactored their code.
Reason for implementation this way:
I didn't see a better way to do it, but am open to suggestions.
Test cases:
A mock ring is created with two different devices, test and iops
test_first_device: Ensure that the first device, test, is returned for both get_nodes, and
get_part_node, and get_more_nodes returns volume_not_in_ring.
test_invalid_device: Ensure that a request for a non-existant device returns volume_not_in_ring.
test_second_device: Same as test_first_device, but for the second device, iops instead of test.
test_second_device_with_reseller_prefix: Test that calling with the reseller prefix, AUTH_ will
still return the correct data.
Change-Id: I2f3d526934a47b01e9c065d0edf0fbf06f300369
BUG: 924792
Signed-off-by: Alex Wheeler <wheelear@gmail.com>
Reviewed-on: http://review.gluster.org/4748
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Gluster cli uses the remote-host option to connect to the glusterd
and by default it uses localhost to connect to glusterd. So, UFO code
will use the remote-host option everytime to connect to the glusterd.
Change-Id: I5a684d3c43fe9bdc9cc0b7c472a9d8145f9e1fd4
BUG: 878663
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4690
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Id22c968aefd82c4b62445b3ecc93cbabc2b35ffc
BUG: 887301
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4394
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Currently, the users of UFO are restricted to use just one volume
at any given point of time. This patch removes this limitation.
* The usage of gluster-swift-gen-builders has also changed. With this
commit the users should mention the list of volumes that they want
to expose through UFO. So, only the volumes mentioned during the
ring file generation can be accessed.
Usage: gluster-swift-gen-builders <vol-name1> [<vol-name2>]...
This is an intermediate fix until we remove the account, container and
object server processes. Once we have this frame work running, it will
completely eliminate the ring files.
Change-Id: I9ad3808519fec9c7c60ad846c4f8b653117a8337
BUG: 909053
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4485
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
system calls.
The set of changes:
* Unit test cases for fs_utils.py
* Replaced os.path with os_path
* Implemented wrapper functions do_write, do_chmod, etc in fs_utils.py
* Replaced os.<sys-call> with the wrapper functions.
Change-Id: I770da878e83eda6b98e49d70193990406a2642a7
BUG: 887301
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4360
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Openstack swift uses the /var/run/swift directory to store the pid files
for all the servers.
* Also, added a script that would unmount the gluster client on a volume stop.
Change-Id: Ib5b9a2964987ca7696d9a2570f1f7af8490b2168
BUG: 861497
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4417
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Have the co-routine (greenlet) yield during long running directory
tree walks to avoid starving out other greenlets that might be
available to run.
A directory walk involves statting every file in the tree to get it
size. For large numbers of files, this can get expensive. So we yield
after every stat call, and after processing every directory.
Change-Id: I07f1dfeef6a09b5817e0c237ecc748c491d52a31
BUG: 894674
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4380
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A file name '.<FILENAME>.<RANDOM>' will hash to the same GlusterFS node
as a file named '<FILENAME>', thus avoiding creation/deletion of linkfiles on a rename.
This is part of the work needed to address BZ 876660
(https://bugzilla.redhat.com/show_bug.cgi?id=876660).
Change-Id: I6f18c14b8eaa7a35c96f7e455ef3a19bee7dbde5
BUG: 876660
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/4325
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Run the UFO unit tests following the main GlusterFS unit tests. The
UFO unit tests run under the Nose unit test running harness, requiring
the python-nose package to be installed. These unit tests run fast
enough that we could even consider running them on a commit hook run
by rfc.sh.
Note that they are not functional tests, as they don't require a real
Gluster file system to run.
Change-Id: I55972890c24be16fc588a6950cb6bfd539287bbe
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4333
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only open a file when the caller of the constructor indicates that it
expects to have a file pointer to access in the "fp" field.
Found by Junaid during review http://review.gluster.org/4284, Patch
set 6.
We also bump the version number to mark a line in the set of changes
we want to perform a mini release with to double check for performance
regressions.
Change-Id: I13cf336bb519088890192ee111f4ece85a5982c4
BUG: 886730
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4303
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The REST API for listing objects in a container does not require that the list
of objects be sorted (the API for listing containers in an account does
require it). Since we can have thousands and thousands of objects in a
container, don't sort them when it is not required.
Change-Id: I6939ef3fec3ea3814a49e3a3046273304889831c
BUG: 887301
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4312
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do not list the entire directory contents of the directory containing the file
before deleting it. We have already verified the files existence constructing
the Gluster_DiskFile object.
This fixes a customer issue. See 885281
(https://bugzilla.redhat.com/show_bug.cgi?id=885281).
Change-Id: I4425c42af3e03624370c779d74b7f073e6eef024
BUG: 885281
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4302
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Apply the upstream DiskFile refactoring in ahead of its use to easiliy apply
the temp file optimization.
Change-Id: I2708733eed3d87759c70eb3d9e6cd74ef91d0c7b
BUG: 876660
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4288
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Here are a set of cleanups for things noticed while working on the
unit test coverage:
* Remove unused constants, adding a constant for default disk chunk
size
* Document missing logger parameter from constructor
* Add iter_hook paramater to constructor in anticipation of 1.7.4
support
* Add back meta_file field even though it is not used in our version
of the constructor for paranoid compatibility
* Remove is_valid field as it is not referenced anywhere, and it is
not a field of the super class
* Rename fields only used internally by DiskFile with leading
underscores
* Convert to using os.path.join() instead of hard coded '/'
references
* Use data_file field where possible
* Assert that put_metadata() will only work when the file exists
* Remove update_object() method since it is identical to
put_metadata() and a direct write will suffice
* Create the directory object only if it does not exist when a
marker directory is requested
Change-Id: If207fed4c0f423e6bd3232e4507f87de877057c4
BUG: 876660
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4287
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we had this ahead of time, we could have avoided the errors that
were encountered leading to the fix-account-mapping fix (see
http://review.gluster.org/4222).
This represents 100% coverage of the DiskFile module, but the coverage
report says otherwise, unfortunately. That is because the put() method
invokes eventlets during the test run, and coverage is not able to
accurately track the coverage lines properly. If one comments out the
"tpool.execute()" line in DiskFile.put() the coverage report then
reports 100% for the DiskFile module.
Additionally, we changed DiskFile.put() in four ways that should not
change its behavior:
1. Comments were added to explain various code paths and mark
potential issues / fixes
2. It no longer returns a boolean value, matching the behavior of
swift.obj.server.DiskFile.put()
3. It no longer logs a message when it detects a directory that
already exists, instead is raises an exception
We believe this is okay because we cannot find a code path that
would lead to his condition. As a result, it makes it easier to
test all the code paths in that routine.
4. It no longer logs a message when create_dir_object() fails, since
create_dir_object() raises an exception on failure only
We also modified create_dir_object() to not return a boolean as a
result of the above behavior.
Note that by implementing these tests up to this point we found three
code paths that would have failed if encountered due to missing
imports. We also made changes to the DiskFile module to make it a bit
easier to test, also eliminating an extra stat system call when
deleting directory objects.
Change-Id: I3286de83c1ec7c5e8d6cab9354e1c4397cee7497
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4284
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I2d540e16cc3d420b9bb88e6857e9795d56e34529
BUG: 884327
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4225
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Override OpenStack Swift's swift.common.utils.write_pickle with our own
implementation that uses the GlusterFS temporary file operation. A file name
'.<FILENAME>.<RANDOM>' will hash to the same GlusterFS node as a file named
'<FILENAME>', those avoiding a move of the file on a rename.
This is part of the work needed to address BZ 876660
(https://bugzilla.redhat.com/show_bug.cgi?id=876660).
Change-Id: I1cb9f97f289ab2ca76ec9221366df74de08268bb
BUG: 876660
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4224
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, we have not solved the account mapping story entirely (we
don't have a way to support more than one account to gluster volume in
the mapping).
As such, do not declare this as the final version.
Also update the default RHS 2.0 spec file to "-3" for the release
number to refer to the fact that we now have a working account mapping
(see http://review.gluster.org/4222).
Change-Id: Ia1cd0d18c1d7a7b20f732cc0ff867fecda30a7af
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4283
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We now have access to openstack-swift-* packages that work with these sources
so we can enforce the dependencies and what gets obsoleted.
Change-Id: I83f442f3eb2424d176a2257f3498ed39eed32251
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4223
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix the account mapping so that we always use what is provided by the request.
With this fix, "/mnt/gluster-object/AUTH_ufo0" directory hierachies won't be
created. But this fix does not restore the one-to-one account to gluster
volume name mapping.
When the user runs the /usr/bin/gluster-swift-gen-builders script, it still
only allows one cluster volume, and that is the only volume that is used. So
the account names are effectively ignored. A future set of changes will
address that problem.
Change-Id: I2df608de2f00fd356a09c500d49fe12cc1a0a574
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4222
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the current lack of authentication, there is no security
improvement with passwordless ssh replaced. However, it is more
convenient to install UFO if you do not have to generate ssh
keys every time.
There is a downside the gluster-server RPM becomes required.
BUG: 880050
Change-Id: I04f4ad666ac96713055723bd432131a4db325b4f
Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-on: http://review.gluster.org/4231
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After the commit b0cb7aaf for bz#870589, which adds 2 tarballs
with layout patterns, Swift tests sometimes fail (depending on the
kind of filesystem they are running at and the phase of the moon).
As pattern tarball is unpacked, the underlying filesystem is free
to return directory listings in any order, and straightforward use
of '==' operator fails. As it turns out, one of the two comparisons
had set() applied to the list already. So, add it to the other one.
BUG: 874390
Change-Id: I02de99593b9567a13076113d58e242b079fde002
Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-on: http://review.gluster.org/4165
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove the "swift/1.4.8" directory from the hierarchy, replacing it with "ufo".
Change-Id: I60ba171182d7402a2193616c1c95c90cd2ead5bc
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4200
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Final set of changes to remove the diffs carried to make UFO work with
OpenStack Swift. The code is now a complete layering on top of OpenStack Swift
where we either "monkey patch" or subclass as necessary.
See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).
There are a lot of changes here due for the most part to rearranging the
directory hierarchy to have create a proper python module hierarchy under the
"gluster" namespace. Plugin references have been removed. The differences that
used to be in the swift.diff file are now replaced with server implementations
for account, container, object, and proxy that subclass the swift versions.
Additionally, the plugins/conf directory has been moved to the "etc"
directory, and the plugins/bin directory promoted a level.
Unit tests pass.
A new setup.py file is provided so that the install process can use it for
creating all the necessary python install infrastructure (eggs and paste
support).
A new RPM spec file is provided which to properly install the new code, and
the sample configuration files have been modified to reference the new python
egg.
Change-Id: I4316c1b66dca80f847fe9b0d583174689c175599
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4180
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor code to use the devices configuration file setting for account,
container, and object servers, dropping mount_path from the fs.conf file, and
constructing new account, container and object server rings.
This removes the next to last set of diffs in the swift diff file. See BZ
870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).
The key to the change is the dropping of the pre-built account, container and
object rings, instead providing a script that will generate them for the user
given the gluster volume name in use. In addition, we override the Swift
check_mount() method and replace it with Gluster's which attempts to
"auto-mount" if it is not already mounted.
The following is an enumeration of the changes contained in this refactoring:
* The refactoring to override the Swift check_mount() involved condensing
a lot of code, removing redundancies and simplifying methods across a
number of modules
* Drop checking the mount point in the low level DiskDir, DiskAccount and
DiskFile objects now that Swift's normal mount checking is used, and
enable it by default in the template configuration file
* Add missing get_container_timestamp() method for DiskAccount objects
* Fix the plugin RPM spec file to provide the new ring builders, and while
we were at it, finally fix the over-writing of the configuration files
on install
* Bug fixes
* The monkey patched version of check_object_creation was not working
correctly due to a missing import of HTTPBadRequest and a bad
reference to validate_obj_name_component()
* Only have the utils module import the plugins constraints module for
the side effect of monkey patching if gluster is enabled
* Removed the db_file.db file in the tree since it is created on the
fly
* Updated 2011 copyright notices to 2012
Change-Id: I8f4454576b1423021c9bbf3c36176f8db51e62c0
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4179
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Not sure how this worked before this, but mount_ip and mount_path do not
exist, their uppercase versions do.
Change-Id: I53e9b550991d17b5541ed4ada5f1bc9b3cb1b8b1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4178
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Further reduce the diffs we have to carry by moving the Gluster specific
metadata stored when we create an object to the DiskFile class's put() method.
Change-Id: I8cef5412627d3ddfdadbf78ef947e067db5bf620
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4177
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Towards swift-diff removal: hide object_only behavior inside DiskDir class
objects. See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589)
One of the last few reasons for having the swift diff is the need to update
the account and container information from the file system when the
configuration is not for object-only use (meaning, other updates to the file
system can come from clients other than UFO, which means UFO currently has to
generate the necessary metadata for each request instead of caching it).
This change moves that logic into the DiskDir class methods so that the
container and account server code does not have to track that.
Change-Id: If0970c77c72245facd09c2035083e681d6d80303
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4176
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ib3c671e693c2c332af98a593ca14c42c36f5ca76
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4175
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I60b4b1b976ad8359ba072d5d0ed6a8dd2087a738
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4174
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
See BZ 868120 (https://bugzilla.redhat.com/show_bug.cgi?id=868120)
Change-Id: Ia7f5562cff32eee30c5ca18652d5d9a5a2b18bf4
BUG: 868120
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4173
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changes to make DiskAccount subclass DiskDir complete. This change helps
reduce the diffs ahead of further refactoring. Note that DiskDir initializes
all the same fields as DiskAccount, plus a few more, and ends up reading the
metadata properly, while only adding a couple of fields not used for account
purposes.
This change also passes the account controller's logger for use by the
DiskAccount class.
Along the way we:
* Remove the redundant update_account() method in the DiskCommon base
class, since the update_container() method in DiskDir performs the exact
same functionality
* Remove the initialization of variables ahead of assignments in two
cases, and also move the initialization of the list variables closer to
where they are used
* Collapse the imports from swift.plugins.utils into one statement
* Use a temporary directory for the db_file.db when we find ourselves in a
unit test environment
Change-Id: I4dc59b416a66393adfe0ed32c7a62c79731d70c0
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4172
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Tested-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix bad references to check_account_exists: this is a module local method, not
provided by Glusterfs. Not sure how we missed this. More unit tests would
help.
Also added the ability to run the Swift tree unit tests with gluster
artificially enabled to test as many code paths as possible ahead of full
functional tests.
Change-Id: Iaf696d34db8f841c887f03569e65321fca8894d1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4171
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I07cd997ef9a98645a87792632f9dd612928678b7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4170
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Minor refactoring ahead of further diffs to add clarity to the code.
Rename the DiskDir field from "name" to "container", which is what it is.
Along the way, add a note regarding the one-to-one mapping of account name to
gluster volume name, and fix the get_info() container method documentation to
list all the dictionary keys returned.
Change-Id: I659e1a9a070c16671287f17a170f9d77198534e7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4169
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unsed parameter cleanup and handling changes. The device and partition
parameters are not used. The account parameter is used in place of the device
parameter. Rather than just making the device parameter reference the value of
the account parameter, we just don't use the device parameter at all and make
references to account. We also remove the device_path field from the class
since it is unused.
Additionally, we assert that we have a logger since the rest of the relies on
that fact.
Change-Id: Iea556319744ab4f729b1ce23d987650a164749db
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4168
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Tested-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Add missing documentation for constructor parameters.
Change-Id: I3a656cd22f6a2a61e0b4d563c75c2f7fe68d3c54
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4167
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Apply a few minor code cleanups to help reduce future refactoring diffs.
* Remove extra lines between methods so that all methods are consistently
spaced by one line
* Remove unused field meta_file
* Add missing documentation for uid and gid parameters
Change-Id: I0a38eb06680eb0709edfaa1828d09657823f778e
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4166
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=870589
Remove the Glusterfs object, transforming it into a module providing module
data fields (like swift.common.constraints) and module methods for
mounting/unmounting and access the gluster volume information. As a result, we
can then remove the glusterfs filter from the pipeline since we no longer need
to provide the Glusterfs object through all the plugin code paths.
This is one more step closer to removing our dependency on modifying the Swift
code directly with these changes. See It is also the first step to acknowledging
that we are not a plugin, but a layering on top of Swift.
The major piece of work here is based on a recognition that the
plugins/Glusterfs.py module provided a Glusterfs class that instantiated
instances of an object that always contained the same data from the
configuration file. The fields of such an object were not being changed and
were treated as read-only in all cases. Since the object's data was the same
for all instantiations there was no need to pass the data from the glusterfs
filter all the way down into the bowels of the Gluster_DiskFile and DiskDir
objects.
Taking advantage of the nature of that data, we now just have those fields
read into module variables, and change the Glusterfs object methods into
module level functions. Much of the changes result from the consequence of
making that switch from object to module.
Here are a few other changes made along the way:
* Bump the release numbers in the spec files in recognition of these changes
* Create the plugins/fs_utils.py module so that the methods in the
plugins/Glusterfs.py module don't have to include plugins/utils.py, which
would create a circular dependency
* Note that this dependency comes from methods in plugins/utils.py
depending on the module level constructs in plugins/Glusterfs.py so that
we only store those values in one place
* Changed plugins/DiskDir.py:DiskDir class to not check for, and/or
optionally create, the /etc/swift/db_file.db at run time, just create it a
module init time
* Removed the duplicate strip_obj_storage_path() from plugins/DiskDir.py and
utils.py and move it to the Glusterfs module
* Used os.path.join in plugins/DiskDir.py where possible
* Renamed the .conf files to .conf-gluster so that we don't clobber existing
config files
* This is not a complete change, as the spec file also needs to be
modified to avoid the clobbering
* See also https://bugzilla.redhat.com/show_bug.cgi?id=865867
* Removed the redundant DIR_TYPE definition in plugins/utils.py
* Removed MOUNT_PATH from plugins/utils.py replacing references with that from
Glusterfs
* This actually fixes a bug if a user every used a different mount path
from the default in fs.conf
* Added ASYNCDIR definition to plugins/utils.py until such time as another
refactoring can rely on the one from swift.obj.server
* Renamed plugins/utils.py's plugin_enabled() function to Gluster_enabled()
* The diffs we carry for Swift are now a bit smaller in that we no longer
have to add the plugin() method, we don't have to keep a fs_object field
in these objects, and we can reference the Glusterfs module directly
* Unit tests were modified appropriately, but now need to be run in the
context of a Swift tree; this is unfortunate, but further refactoring will
address this
Change-Id: Id5d2510d56364761c03b3979bc71187dbe2f82fe
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4141
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|