summaryrefslogtreecommitdiffstats
path: root/gluster
Commit message (Collapse)AuthorAgeFilesLines
* Fix infinite loop for temp file renames on ENOENTPeter Portante2013-08-281-41/+45
| | | | | | | | | | | | | | | | | | | | | For whatever reason, it appears that GlusterFS, or perhaps FUSE can continuously return ENOENT on a rename system call even when we have double checked that there is no reason to do so. That is a bug for that sub system. However, our response to that bug can result in an infinite loop, which is bad. This code reduces that to 10 attempts. In addition, we restructed the open retry loop to match, providing module constants for the upper bounds of both retry loops. BUG: XXXXXX (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX) Change-Id: Ia2d6dd427daba3ea0461863c5ffe3aef27c88f9b Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5670 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Bump to 1.9.1 to match Swift versionPeter Portante2013-08-281-1/+1
| | | | | | | | Change-Id: I125db97e772b86cd51a45627c2fa23809e80878b Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5702 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Updates to support Havana interim version 1.9.1.Peter Portante2013-08-219-324/+346
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code changes are basically: * Apply refactoring in the DiskFile class to use the new DiskWriter abstraction * Move and rename our diskfile module to match upstream * ThreadPools allow us to remove the tpool usage around fsync * Update the Ring subclass to support the get_part() method * Update to use the 1.9.1 proxy server unit tests * Move the DebugLogger class to test.unit * Rebuild the Rings to use the new layout * Remove backup ring builder files * Update spec files to 1.9.1, and tox to use swift 1.9.1 * Updated version to 1.9.0-0 Change-Id: Ica12cac8b351627d67500723f1dbd8a54d45f7c8 Signed-off-by: Peter Portante <peter.portante@redhat.com> Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5331
* Fix spec file to support source rpmsLuis Pabon2013-08-211-4/+6
| | | | | | | | | | | | | | | | Our initial implementation only required Jenkins to export binary RPMs, but as we move foward, we really need to also export SRPMs. To support SRPMs, the spec file in the RPM has to have the correct NAME, VERSION, and RELEASE information. Change-Id: Icd7132b4aafdbe7a1f02a35d0be7ad63b2e7c056 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5669 Reviewed-by: Peter Portante <pportant@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Peter Portante <pportant@redhat.com> Reviewed-on: http://review.gluster.org/5679
* Don't reference Havana (1.9.1) until we switchPeter Portante2013-08-161-1/+16
| | | | | | | | Change-Id: Ic332b2c49a3b253207831a0c411de1a726fc84d1 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5639 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* perf: Container and account performance incLuis Pabon2013-08-063-25/+45
| | | | | | | | | | | | | | | | | | | | | * Container and accounts performance increase by removing the need to update either the object count or the container count, respectively. New hidden configuratoins added to re-enable the functionality. * object_only configuratoin removed from fs.conf and replaced with a hidden configuration. The new hidden configuration reports gratuituosly crated directories as objects, to support a compatibility behavior with previous version 1.4.8 (even though it was incorrect). BUG 988969: https://bugzilla.redhat.com/show_bug.cgi?id=988969 Change-Id: Idca20b8629ec38606ff5692fe62bc2cadabffc86 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5403 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com> Reviewed-on: http://review.gluster.org/5504
* Gluster to handle ENOSPC (Error 28) correctlyChetan Risbud2013-08-063-4/+26
| | | | | | | | | | | | | | | | | A gluster volume could yield an ENOSPC condition seeing that a volume is full. This needed to handled correctly. Added error handling. BUG: 985253 https://bugzilla.redhat.com/show_bug.cgi?id=985253 Change-Id: I85472c0a81a354a2796327fead606da3a938d4bf Signed-off-by: Chetan Risbud <crisbud@redhat.com> Reviewed-on: http://review.gluster.org/5362 Reviewed-by: Peter Portante <pportant@redhat.com> Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Handle the ring_name variable being none in Ring.__init__Mohammed Junaid2013-08-011-7/+11
| | | | | | | | Change-Id: Ifbcf217e525e50f62b98028ed5292be58db64bad Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/5459 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Updating copyrights with current yearChetan Risbud2013-07-2411-11/+11
| | | | | | | | | | | Copyrights are updated across the code to reflect 2013. Copyrights message was added to couple of scripts. Change-Id: I2cfc7bdc6adaa5c9ab0075aab49021a5f0dea7a1 Signed-off-by: Chetan Risbud <crisbud@redhat.com> Reviewed-on: http://review.gluster.org/5371 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Gracefully handle the ring file not found error.Mohammed Junaid2013-07-241-2/+12
| | | | | | | | Change-Id: I35c33f3938709d5d9a9a55640a503f186df9a965 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/5368 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Bump version to 1.9.1 to match HavanaLuis Pabon2013-07-111-1/+1
| | | | | | | | Still need to change unit tests to use Havana Change-Id: I7847e7916043933a47b3553115c79c5d6b28a836 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5312
* Final forward port of PDQ performance patchesPeter Portante2013-07-107-316/+928
| | | | | | | | Change-Id: I4ef131b3cc7648d4571a4d854029efb1aff8b901 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5305 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Remove extraneous system callsMohammed Junaid2013-07-092-18/+31
| | | | | | | | | | | | | Remove a number of stat() system calls that were redundant. BUG 903396 (https://bugzilla.redhat.com/show_bug.cgi?id=903396) Change-Id: I2e89ce8a023650cd745f136f5c697902cf2462ea Signed-off-by: Mohammed Junaid <junaid@redhat.com> Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5293 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Remove do_mkdir and simplify mkdirsLuis Pabon2013-07-091-22/+8
| | | | | | | | Change-Id: I2e629533bceafb341ced5b1b3f2436448293e03f Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5304 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Removed unused metadata key and do_rmdir funcLuis Pabon2013-07-092-14/+0
| | | | | | | | | Change-Id: Ibcef5d817390a11d85cf125bb9addfbfded1b019 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5301 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* OpenStack Swift Functional Tests for G4SLuis Pabon2013-06-284-52/+145
| | | | | | | | | | | | | | | | | | | | This commit has the following changes: * G4S no longer accepts URLs that end in /. A HTTP code of 400 is returned when a / at the end of the object is detected. * Directories can be created as objects setting the content-type to application/directory and content-length to 0. * Functional tests have been adjusted to work with G4S constraints Change-Id: I31038a59699a8e3eeaba902db322218c6400093e Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5246 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* object-storage: remove stat of directoriesMohammed Junaid2013-06-271-55/+19
| | | | | | | | | | | | | | | | | It was painfully slow to gather a list of all the files in a directory tree when there were as many directories as files, since we also stat'd all of the directories as well. We only did that to cache using memcache the result, to know when it changed. However, there was no way for a memcache object to be passed down to this code. So we have removed the memcache support entirely, and removed the stat of the directories as a result. See BZ 911448 (https://bugzilla.redhat.com/show_bug.cgi?id=911448). Change-Id: I2625f82eca36c31eeffa84dc76ef7f3b48aafec5 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/5252 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* object-storage: Use fchown instead of chown.Mohammed Junaid2013-06-273-4/+14
| | | | | | | | | | | This is a step towards making fd based system calls where ever possible to avoid path lookups. Signed-off-by: Mohammed Junaid <junaid@redhat.com> Change-Id: I482ea29ebe0859d0a5307ff25ecb5945d54bc7ca Reviewed-on: http://review.gluster.org/5251 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Return correct status when deleting non-existing containerLuis Pabon2013-06-111-11/+25
| | | | | | | | | | | | | | | The code was raising an exception when the container (which happens to be a directory) did not exist. To be compatible with OpenStack Swift, we need to handle an object which its container/directory does not exist. BUG: 960944 (https://bugzilla.redhat.com/show_bug.cgi?id=960944) Change-Id: Ibb2db354a655e040fb70ebbe6a7d8f815d33dc0f Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5201 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Refactor DiskDir and DiskAccount to use DiskCommonPeter Portante2013-06-071-96/+134
| | | | | | | | | | | | | | | | We have refactored DiskDir and DiskAccount so that truly common, shared functions live in the base class DiskCommon, and each of these then sub-class that. This allows us to isolate unique behaviors. We have also enabled all the skipped unit tests in test_diskdir.py that we could, removing a few that we don't need right now. Change-Id: I48d9c915108df8cc92c3e9a764563e1d10c50050 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5148 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Change filters to use a generator patternPeter Portante2013-06-061-104/+171
| | | | | | | | | | | | | | | | | By using a generator pattern, we avoid creating whole new lists each time, instead we iterate through the original list once (after it is sorted), constructing the final list only once. We also address the behavioral differences between the swift filtering results and our code so that ported unit tests work the same (non-slash objects, that is). Change-Id: If32c1987f24781ff81ab4c28c9ddfff17c2e7787 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5145 Tested-by: Luis Pabon <lpabon@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Luis Pabon <lpabon@redhat.com>
* Refactor filter_* methods as module functionsPeter Portante2013-06-041-67/+72
| | | | | | | | | | | | | The various filter_* methods of the DiskCommon class don't operate on the DiskCommon object instance itself, but always on the list of objects passed in to it. This change does not modify those functions' behaviors but simply refactors them to live at the module level. Change-Id: Ic0f578d94aab65a524470278f10d1fdc8b1d6392 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5138 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Forward port PDQ 3489: turn off cont & act updatesPeter Portante2013-06-042-2/+54
| | | | | | | | | | | This is a simple sub-classing of the proper update methods to just no-op their behavior. Change-Id: Ib1ae5234d372cbce572da34cfe702235b78f2310 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5088 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Copy proxy unit test from OpenStack SwiftLuis Pabon2013-06-033-4/+13
| | | | | | | | | | | | This is a copy of the proxy unit test from OpenStack Swift. The file has minor changes to make it compatible with Gluster For Swift. Change-Id: I64a0f8d274461eb2a2c38524c6282e0d3d3d1457 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5113 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Remove account name from being saved in the objectLuis Pabon2013-06-031-11/+32
| | | | | | | | | | | Instead we save the account in the a list, where the index to the account is the partition number. Change-Id: Ie4abefee48a3b237306a1e301ffa798e24e3f1db Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5120 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Bring initial DiskDir module coverage to 97%.Peter Portante2013-06-033-126/+223
| | | | | | | | | | | | | | | | | | | | This is quite ugly. Sorry. We ported a set of test from OpenStack Swift's test/unit/commont/test_db.py, the testAccountBroker and testContainerBroker, but because of the divergent nature of the current attempt UFO (Unified File and Object) implementation, it was not possible to use the interface calls directly. Instead, we are using these tests to make sure most of the code paths are exercised, and to define much of the expected behavior. Further refactoring, unit tests and functional test work will help us bring the code base and these tests closer to the originals in upstream OpenStack Swift (as of Havana). Change-Id: I095bb03619de6e7e1378b5252913e39b1ea8bf27 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5135 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* pep8 changes to constraint.pyLuis Pabon2013-05-291-1/+3
| | | | | | | | | | | We need the code to pass pep8 tests so that we can enable pep8 in Jenkins Change-Id: I31e4bd10fad65735fabceaedcb1a3bd10e120153 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5114 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* Unittest case for constraints.pyMohammed Junaid2013-05-291-5/+18
| | | | | | | | Change-Id: Ibb37855c675d5def2ce2e1ed387429ce87a4926b Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/5103 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Remove unused is_marker method from utils.Peter Portante2013-05-281-16/+0
| | | | | | | | | Change-Id: I67f695b57405465c7bec22c437c2169403fa59dd Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5089 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Luis Pabon <lpabon@redhat.com>
* Remove middleware directory missed in prev commitPeter Portante2013-05-241-0/+0
| | | | | | | | Change-Id: Ic0e9e78acc06e180f7acfec3e2a5901b7f80598b Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5079 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Bring DiskFile module coverge to 100%Peter Portante2013-05-241-5/+1
| | | | | | | | | | | | | | | | | | | | | | | | To bring the DiskFile module coverage to 100%, we first recognize that do_fsync() will invoke fsync() in a separate thread, which gives coverage fits (see the commit history at https://github.com/portante/coverage/commits/master). To avoid that, we mock out do_fsync to make it a no-op and avoid the problem. The second thing we recognize is that mkstemp() relies on do_unlink from the fs_utils module, which already consumes ENOENT errors, so we don't need that code path in mkstemp() itself. The unused mock routine for do_unlink was removed as well, and we renamed the other do_unlink mock routine to os_unlink since it was mocking out os.unlink directly. Lastly, we rejigger the error on close test for mkstemp() to prematurely close the fd to cause an error which should just be squelched, completing the full coverage. Change-Id: I98283c17cf139f92282f8afd7083d567d3dd9a79 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5082 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Add coverage for gluster.swift module (PkgInfo).Peter Portante2013-05-241-2/+4
| | | | | | | | Change-Id: I614efb853989ab455721bbf28309cec746343f50 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5081 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Add DiskDir unit test skeleton and pep8 filterPeter Portante2013-05-2413-135/+203
| | | | | | | | | | | | | | | | | The new DiskDir unit test skeleton is quite incomplete, but gets the DiskDir module on the board for modules covered, explicitly exposing the fact that we need to get test coverage. This is a first step. At the same time, we also update all the modules we have applying the fix for pep8 errors now run under tox. We can then add a Jenkins pre-commit job to fail on pep8 errors. This brings our code to parity with what they are doing in OpenStack Swift. Change-Id: Ia0565606512efda6e73f67bd00269177b89db858 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5080 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* Remove tmpauth refs and unused gluster middlewarePeter Portante2013-05-231-40/+0
| | | | | | | | Change-Id: Ia3d5813659d49d743f91816c4c7cab57e36ed9fa Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/5076 Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Luis Pabon <lpabon@redhat.com>
* object-store: Added busy_wait unit test to Glusterfs.pyLuis Pabon2013-05-171-1/+1
| | | | | | | | Change-Id: Ifee5dccd47a3e301812533851c45b9fe853f9b71 Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/4983 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* rpm: Enable creation of RPMsLuis Pabon2013-05-171-6/+17
| | | | | | | | | | | | | | | | | | | | | This is the first step. I have confirmed that I can install, update, and remove. I have also setup the ability for Jenkins to pass the BuildNumber and use that value as the release number for the RPM. The RPM depends on Grizzly(1.8.0) Swift from OpenStack. To verify you may need to add the appropiate repo file to your Fedora/RHEL system: http://repos.fedorapeople.org/repos/openstack/openstack-grizzly I have not had the opportunity to test that G4S itself works once installed, but I plan on doing that as the next phase. Change-Id: Ib90f335f5e1e4fc552c32e00ff29b6e8a680c42a Signed-off-by: Luis Pabon <lpabon@redhat.com> Reviewed-on: http://review.gluster.org/5006 Reviewed-by: Peter Portante <pportant@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* Remove "ufo" directory, promoting contents to top-levelPeter Portante2013-05-1021-0/+2095
Additionally, we drop the "ufo" references from setup.py, spec file and README, and add the HISTORY file describing how the repo was initially created. We also update the RPM spec file to use the name "gluster-for-swift" to avoid colliding with existing RPM names from RHS 2.0 (the spec file's description was also updated, along with the version number). Change-Id: If804224a94208d57896e4189c63736ffc9e01d5e Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4966 Reviewed-by: Luis Pabon <lpabon@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>