summaryrefslogtreecommitdiffstats
path: root/test/unit/obj/test_diskfile.py
Commit message (Collapse)AuthorAgeFilesLines
* Rebase to Swift 2.10.1 (newton)venkata edara2017-05-111-1/+1
| | | | | | | | Change-Id: I53a962c9a301089c8aed0b43c50f944c30225944 Signed-off-by: venkata edara <redara@redhat.com> Reviewed-on: https://review.gluster.org/16653 Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Prashanth Pai <ppai@redhat.com>
* Fix validation of marker dir objectsPrashanth Pai2016-09-121-14/+0
| | | | | | | | | | | | | | | | | | For marker directory objects, validate_object() always returned False. This was because st_size from stat was being compared to Content-Length stored in metadata. Unlike files, for directories st_size is always 4096. Hence the comparison would always be '4096 == 0' which would fail. This patch makes the following changes: * Do size comparison of st_size and Content-Length only for files. * Get rid of _is_dir everywhere. This will simplify things. Change-Id: Ib75e06c4e3bce36bab11ce7d029ff327f33c3146 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/14423 Reviewed-by: Thiago da Silva <thiago@redhat.com> Tested-by: Thiago da Silva <thiago@redhat.com>
* Don't attempt unlink on a non-existing objectPrashanth Pai2016-08-101-0/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: getxattr() and unlink() were being called on an object path which was already determined to be non-existent. This resulted in both these syscalls always failing with ENOENT when client issues DELETE request on an object that does not exist. A request to DELETE an object will incur the following DiskFile API calls in sequence: disk_file.read_metadata() disk_file.delete() The above mentioned problem manifests because Swift code invokes disk_file.delete() even when disk_file.read_metadata() has raised DiskFileNotExist. Fix: During disk_file.read_metadata(), make a note that the file does not exist. When disk_file.delete() is called, do not proceed with object deletion. Change-Id: Iaf6915197a8fced7564db8fab80e696dea080c25 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/14501 Reviewed-by: Thiago da Silva <thiago@redhat.com> Tested-by: Thiago da Silva <thiago@redhat.com>
* Don't (f)chown when it has no effectPrashanth Pai2016-05-041-0/+45
| | | | | | | | | | | | | | | | | | For (f)chown calls which can change both UID and GID at once, -1 is reserved as a sentinel value to indicate "omitted argument" or "do not change". This makes sense when one of the args to (f)chown is -1. When both uid and gid args are -1, it doesn't make sense to call (f)chown as neither is going to be changed. Further, as of today, diskfile doesn't get the information (uid and gid) of the authenticated user from auth middleware. Retained the calls in code for future when such functionality might be added. Change-Id: If8463ae78a32c379d698260879810ed3c207af02 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/13778 Reviewed-by: Thiago da Silva <thiago@redhat.com> Tested-by: Thiago da Silva <thiago@redhat.com>
* Open object only if it's going to be readPrashanth Pai2016-05-031-0/+48
| | | | | | | | | | | | | | | | | | | Open()ing an object is necessarry only in two cases: * Serving a GET request * Recalculating etag when metadata is stale (can be triggered by any type of request) This change ensures that for requests other than GET, a file is not opened if the metadata is valid (size and etag accurate). Note that if metadata is stale, the file is still opened and read to compute etag. This patch does not change the behaviour of triggering metadata validation and regeneration for non-GET requests. Change-Id: Icefa4dec7d715ec9e6dd68ae7fe89a0d90fe71b3 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/13684 Reviewed-by: Thiago da Silva <thiago@redhat.com> Tested-by: Thiago da Silva <thiago@redhat.com>
* Remove redundant syscalls in POST pathPrashanth Pai2016-04-041-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During process of POST requests which updates object metadata (xattrs), the following (ordered) sequence of syscalls were being made twice: open(), fstat(), fgetxattr(), close() Intuitively, one may assume that a getxattr() and setxattr() is enough to fulfil the POST request as it is only supposed to update metadata. But this isn't the case. The above series of syscalls is made first during disk_file.open(). This will trigger an update of all stale metadata (outdated size/etag) and the result is retained in a diskfile class attribute named 'self._metadata' Instead of using this pre-fetched metadata, the POST path was internally invoking disk_file.open() again in disk_file.write_metadata(). This is redundant and serves no purpose. self._metadata was being erased during the context manager cleanup of disk_file.open() This change is simple and does the following: * Don't erase fetched metadata during context manager exit of open() * Use a different internal variable to detect and raise DiskFileNotOpen * Re-use self._metadata if available in disk_file.write_metadata() Here's comparing syscalls made (POST path) with and without this fix: https://bugzilla.redhat.com/show_bug.cgi?id=1314171#c4 Change-Id: Ib64c103e5904428df20ec6e8f10140f4f68e7f79 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/13668 Reviewed-by: Thiago da Silva <thiago@redhat.com> Tested-by: Thiago da Silva <thiago@redhat.com>
* Rebase to stable/kiloPrashanth Pai2016-03-071-37/+101
| | | | | | | | | | | | | | | | This change ports most of swiftonfile object server fixes and changes into gluster-swift. Storage policy as a feature is not usable here (it doesn't make sense). The hacky way of creating zero byte tracker objects for object expiration has not been ported to this release due to scalability issues and the need to have a separate volume. Change-Id: I17ba27dacea9ac000bdb8934700996e4d17f4251 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/13269 Reviewed-by: Thiago da Silva <thiago@redhat.com> Tested-by: Thiago da Silva <thiago@redhat.com>
* Update repoThiago da Silva2016-01-061-4/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a squashed commit imported from this repo: https://github.com/openstack/swiftonfile/tree/icehouse Contains the follwing commits from above mentioned repo: eb50236 Merge "Backport: Fix metadata overall limits bug" into icehouse 79ea52a Backport: Fix metadata overall limits bug bc43f0b Fix inconsistent data being returned on GET ad0bb79 Import HTTPBadRequest from swift's module 74d02e6 Exclude .trashcan dir from container listing b2dbc15 Catch ESTALE in addition to ENOENT 8d60b48 Properly handle read_metadata() exceptions 6762fc6 Fix object server leaking file descriptors 2842e82 Fix API incompatibility in update_metadata() 2beeef6 Merge "Remove swiftkerbauth code" into icehouse 93dbcb5 Update object-expirer.conf with explanations c9d2f09 Merge "Check if /etc/swift exists in ring builder" into icehouse d66c14c Remove swiftkerbauth code 3142ed2 Add object expiration functests 97153d1 Merge "Cleanup functest and undo old patch" into icehouse bc234d0 Remove old travis config file and fix typo 260c8ef Check if /etc/swift exists in ring builder 637dac9 Cleanup functest and undo old patch 051e068 Merge pull request #35 from prashanthpai/backport-1 be104a3 Merge pull request #36 from prashanthpai/backport-2 ff76f42 fix issue with GET on large object (icehouse-backport) 04d0a99 Fix unlink call after successful rename 4c6ca1d updating README file with project name change 10b2680 Merge pull request #18 from thiagol11/icehouse 5bcab8f Updating version on __init__ file 5c2cba2 Merge pull request #15 from thiagol11/update_spec 52b00a8 updating spec file to add dependency on swift icehouse ae7c93b Merge pull request #6 from prashanthpai/rebase 191e55b Revert: allow non-root user to run functests cb7e968 Modify unit tests and func tests d23fd1b Sync with OpenStack Swift v1.13.1 b6d1671 Merge pull request #12 from pushpesh/functionalnosetestremove 962622b Merge pull request #8 from thiagol11/update_readme 4560857 Merge pull request #9 from prashanthpai/spec-expirer be0ae7e Minor update 65000f1 Removing functionalnosetests 8ab1069 Fix object-expirer.conf-gluster RPM build error afee30f added new support filesystem section 527b01f updated README.md to Swift-On-File 9a240c7 Merge pull request #3 from thiagol11/add_jenkins_to_travis 34b5a8b removing blank lines 3568b64 fixing missing fi d8f5b0f adding support to run jenkins triggered by travis 6f4a88c Removing functionalnosetests 8041944 Update README.md c015148 Merge pull request #2 from thiagol11/master 3ddd952 fixing travis file to run correct unit test c582669 adding travis status badge to README 8093096 adding py26 unit testing to travis 37835fd trigger travis build cb6332a adding travis ci testing All tests have been run sucessfully against this. tox -e p2p8,py27,functest Change-Id: I096b611da852d3eb3913844034b443b8272c2ac4 Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on: http://review.gluster.org/13188
* fix __exit__ function not closing file descriptorThiago da Silva2014-03-261-0/+10
| | | | | | | | | | | The file descriptor is not being closed because it is self._fd is None Change-Id: I7edc8a78b09bdd76d59ac8f3dbc809af652f9b0e Signed-off-by: Thiago da Silva <thiago@redhat.com> Reviewed-on: http://review.gluster.org/7315 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* PUT of a Directory failedChetan Risbud2014-03-121-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PUT of a directory fails with gluster-swift when there exists object of a same name. This is because the objects and directories are placed on the glusterfs. And hence the filesystem semantics are applicable there. Exceptions raised in such situation are needed to be handled and reported correctly back to proxy-server with HTTPConflict as a error code. Although swift still continues to choose 503 as the best respose in such cases. No tracebacks reported. Fix covers the case when there exists a directory and object of the same name is PUT. Code changes fixes both the failure cases mentioned in the bug. Examples of failing PUT requests: 1) curl -v -X PUT http://127.0.0.1:8080/v1/AUTH_test/c1/dir1/obj2/anotherobject -d'asdasdsadA' -- obj2 was already an object. 2)curl -v -H 'Content-Length: 0' -H 'Content-Type: application/directory' -X PUT http://127.0.0.1:8080/v1/AUTH_test/c1/obj1 -- obj1 was already and object Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1071021 Change-Id: Id3042d920e3f99e740d4042ef5907ac8c59e04db Signed-off-by: Chetan Risbud <crisbud@redhat.com> Reviewed-on: http://review.gluster.org/7181 Reviewed-by: Prashanth Pai <ppai@redhat.com> Tested-by: Prashanth Pai <ppai@redhat.com>
* Unit test should use python 2.6 calls onlyLuis Pabon2014-01-071-1/+1
| | | | | | | Change-Id: I994b0e45d4fdb898378de7cd103f7e429a7fd9cc Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/6662 Reviewed-by: Thiago Da Silva <thiago@redhat.com>
* Fix double close issue in diskfileLuis Pabon2013-12-131-3/+21
| | | | | | | | | | | | | | | The DiskWriter was closing the file descriptor when it finished writing but initializing it to None. Therefore, the DiskFile context manager would then try to close it also. Change-Id: I188ec814d025e28c7b89532f0502ebf1d4a20a09 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/6317 Reviewed-by: Peter Portante <pportant@redhat.com> Reviewed-by: pushpesh sharma <psharma@redhat.com> Tested-by: pushpesh sharma <psharma@redhat.com> Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/6487
* fix metadata overwrite during a post requestThiago da Silva2013-11-201-5/+95
| | | | | | | | | | | | | | | During a post request, the object-server is ovewriting the existing object metadata. This fix prevents the overwrite of the system metadata while it allows for the user to add/remove user metadata Change-Id: Ic62cd064589b625ee425a9934be8766650622c13 Signed-off-by: Thiago da Silva <thiago@redhat.com> Reviewed-on: http://review.gluster.org/6254 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com> Signed-off-by: Thiago da Silva <thiago@redhat.com> Reviewed-on: http://review.gluster.org/6315
* Rebase to lastest OpenStack Swift DiskFile APIPeter Portante2013-10-291-371/+248
| | | | | | | | | | | | | | | | | | | | As of October 28th, 2013, we rebase to OpenStack Swift master (commit 4bfe674) to pick up the lastest officially supported DiskFile API changes. We use a snapshot of OpenStack Swift stored in the gluster-swift launchpad downloads area so that we can deliberately rebase at our own pace. With this refactoring, all the code for handling I/O is wholly contained in the swift tree for object operations. This will allow us to use a different fs_utils implementation in the future (for example, one based on a yet-to-be-implemented python bindings over libgfapi). This also means the "Fake_file" class has been removed entirely. Change-Id: I767983f88c59786e30b6c64da16d1cb6ab3c3e7f Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5993 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Rebase to OpenStack Swift Havana (1.10.0)Peter Portante2013-10-281-205/+226
| | | | | | | | Change-Id: I90821230a1a7100c74d97cccc9c445251d0f65e7 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/6157 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Unit test coverage for Fake_file interfacesChetan Risbud2013-10-211-7/+20
| | | | | | | | | | | | | | | | The modified test case provides the complete coverage over Fake_file class. These interfaces are to be used for directory type of the objects. BUG:987841 https://bugzilla.redhat.com/show_bug.cgi?id=987841 Change-Id: I2401423d2013ce7d90c454c72fbb60f4bc05ceb1 Signed-off-by: Chetan Risbud <crisbud@redhat.com> Reviewed-on: http://review.gluster.org/6037 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Wrap disk file construction in a method; pep8Peter Portante2013-09-201-103/+93
| | | | | | | | Change-Id: I5e7af267bcbb510108f62a9c66b9baa6771cc8fe Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5985 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Remove reliance on /tmp/foo ...Peter Portante2013-09-201-30/+19
| | | | | | | | Change-Id: I1add2e7e74b59bf24ef46d9053723f0243ee08c5 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5984 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Move temp dir setup/cleanup where it belongsPeter Portante2013-09-201-698/+564
| | | | | | | | Change-Id: I18cdb6c9e8aa56014f49d2fcf4458c3499710ddf Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5983 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Handle GET on zero content length failure case.Chetan Risbud2013-08-281-17/+47
| | | | | | | | | | | | | | | Added a fake_file class that implements minimal set of functions that are invoked by the code in GET. BUG: 987841 https://bugzilla.redhat.com/show_bug.cgi?id=987841 Change-Id: I5bdf5be1c0c4c8231f34c9be529e6edc83774f2e Signed-off-by: Chetan Risbud <crisbud@redhat.com> Reviewed-on: http://review.gluster.org/5511 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Fix infinite loop for temp file renames on ENOENTPeter Portante2013-08-281-4/+52
| | | | | | | | | | | | | | | | | | | | | For whatever reason, it appears that GlusterFS, or perhaps FUSE can continuously return ENOENT on a rename system call even when we have double checked that there is no reason to do so. That is a bug for that sub system. However, our response to that bug can result in an infinite loop, which is bad. This code reduces that to 10 attempts. In addition, we restructed the open retry loop to match, providing module constants for the upper bounds of both retry loops. BUG: XXXXXX (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX) Change-Id: Ia2d6dd427daba3ea0461863c5ffe3aef27c88f9b Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5670 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Updates to support Havana interim version 1.9.1.Peter Portante2013-08-211-0/+975
The code changes are basically: * Apply refactoring in the DiskFile class to use the new DiskWriter abstraction * Move and rename our diskfile module to match upstream * ThreadPools allow us to remove the tpool usage around fsync * Update the Ring subclass to support the get_part() method * Update to use the 1.9.1 proxy server unit tests * Move the DebugLogger class to test.unit * Rebuild the Rings to use the new layout * Remove backup ring builder files * Update spec files to 1.9.1, and tox to use swift 1.9.1 * Updated version to 1.9.0-0 Change-Id: Ica12cac8b351627d67500723f1dbd8a54d45f7c8 Signed-off-by: Peter Portante <peter.portante@redhat.com> Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5331
generated by cgit (git 2.34.1) at 2024-04-23 09:48:48 +0000