diff options
author | Thiago da Silva <thiago@redhat.com> | 2014-01-21 14:21:40 -0500 |
---|---|---|
committer | Thiago da Silva <thiago@redhat.com> | 2014-01-22 11:15:11 -0500 |
commit | 5be3fc07eb54bedf441f358ccb2ac62152de2a78 (patch) | |
tree | 29757f73b51bf20cd0318229e7e7824d12c06c95 | |
parent | acef148b02c5ab0204e1820457cb058f08be5db2 (diff) |
first libgfapi-python functional tests
adding a few functional tests and removing old
tests from source code
Change-Id: Iefcb091d614f2825592943cfb42847b5865322c6
Signed-off-by: Thiago da Silva <thiago@redhat.com>
-rw-r--r-- | README.md | 3 | ||||
-rw-r--r-- | doc/markdown/dev_guide.md | 130 | ||||
-rw-r--r-- | gluster/gfapi.py | 221 | ||||
-rw-r--r-- | test/functional/__init__.py | 0 | ||||
-rw-r--r-- | test/functional/libgfapi-python-tests.py | 127 | ||||
-rwxr-xr-x | tools/functional_tests.sh | 65 | ||||
-rw-r--r-- | tools/test-requires | 1 | ||||
-rw-r--r-- | tox.ini | 4 |
8 files changed, 330 insertions, 221 deletions
@@ -1,2 +1,5 @@ # Overview Python bindings for GlusterFS libfapi + +# Table of Contents +1. [Developer Guide](doc/markdown/dev_guide.md) diff --git a/doc/markdown/dev_guide.md b/doc/markdown/dev_guide.md new file mode 100644 index 0000000..1966d2e --- /dev/null +++ b/doc/markdown/dev_guide.md @@ -0,0 +1,130 @@ +#Developer Guide + +##Development Environment Setup + +The workflow for libgfapi-python is largely based upon the [Gluster-Swift Developer Guide][] and [OpenStack Gerrit Workflow][]. Refer to those documents for setting up a Gerrit account and a complete development environment. + +This document focuses on setting up a quick environment for running tox tests (especially the functional tests). + +## Required Package Installation +Install and start the required packages on your system to create a GlusterFS volume. +``` +yum install gcc python-devel python-setuptools libffi-devel glusterfs \ + glusterfs-server git rpm-build xfsprogs +``` + +Install the python pip tool by executing the following command: + +``` +easy_install pip +``` + +#### Tox and Nose + +libgfapi-python uses tox python virtual environment for its unit and functional tests. To install tox type: + +``` +pip install --upgrade tox nose +``` + +### Start services + +Type the following to start the glusterfs service: + +``` +service glusterd start +``` + +Type the following to start the service automatically on system startup: + +``` +chkconfig glusterd on +``` + +## Gluster Volume Setup + +### Loopback Storage Setup + +If you do not have a separate partition, please execute the following instructions to create a disk image as a file: + +``` +truncate -s 5GB /srv/xfsdisk +mkfs.xfs -i size=512 /srv/xfsdisk +mkdir -p /export/brick +``` + +Add the following line to `/etc/fstab` to mount the storage automatically on system startup: + +``` +/srv/xfsdisk /export/brick xfs loop,inode64,noatime,nodiratime 0 0 +``` + +Now type the following to mount the storage: + +``` +mount -a +``` + +## Create a GlusterFS Volume + +You now need to create a GlusterFS volume + +``` +mkdir /export/brick/b1 +gluster volume create test <hostname>:/export/brick/b1 +gluster volume start test +``` + +## Download the Source + +The source for libgfapi-python is available in Github. To download type: + +``` +git clone https://github.com/gluster/libgfapi-python.git +cd libgfapi-python +``` + +## Running tests + +### PEP8 + +To test that the code adheres to the Python PEP8 specification, please type: + +``` +tox -e pep8 +``` + +### Unit Tests + +Once you have made your changes, you can test the quality of the code by executing the automated unit tests as follows: +``` +tox -e ENV +``` + +where ENV is either py27 for systems with Python 2.7+, or py26 for systems with Python 2.6+. + +If new functionality has been added, it is highly recommended that one or more tests be added to the automated unit test suite. Unit tests are available under the test/unit directory. + +### Functional tests +The functional tests expects a `test` volume to be created and accessible. + +To run the functional tests, please type: + +``` +tox -e functest +``` +####Important Notes: +##### Definining a hostname +GlusterFS does not allow for specifiyng `localhost` as a valid hostname when creating a volume, so `gfshost` was used in the functional tests. If you use a different hostname when creating the gluster volume, be sure to update the functional tests. + +##### Stopping services +For the purpose of running this test, stop the `firewalld` service and disable `selinux`. + +``` +service firewalld stop +``` + + +[OpenStack Gerrit Workflow]: https://wiki.openstack.org/wiki/Gerrit_Workflow +[Gerrit]: https://code.google.com/p/gerrit/ +[Gluster-Swift Developer Guide]: https://github.com/gluster/gluster-swift/blob/master/doc/markdown/dev_guide.md diff --git a/gluster/gfapi.py b/gluster/gfapi.py index 516a1f0..c2c267c 100644 --- a/gluster/gfapi.py +++ b/gluster/gfapi.py @@ -1,10 +1,6 @@ -#!/usr/bin/python - import ctypes from ctypes.util import find_library import os -import sys -import types from contextlib import contextmanager @@ -277,220 +273,3 @@ class Volume(object): err = ctypes.get_errno() raise OSError(err, os.strerror(err)) return ret - -if __name__ == "__main__": - - def test_create_write(vol, path, data): - mypath = path + ".io" - with vol.creat(mypath, os.O_WRONLY | os.O_EXCL, 0644) as fd: - if not fd: - return False, "creat error" - rc = fd.write(data) - if rc != len(data): - return False, "wrote %d/%d bytes" % (rc, len(data)) - return True, "wrote %d bytes" % rc - - # TBD: this test fails if we do create, open, write, read - def test_open_read(vol, path, data): - mypath = path + ".io" - with vol.open(mypath, os.O_RDONLY) as fd: - if not fd: - return False, "open error" - dlen = len(data) * 2 - buf = fd.read(dlen) - if isinstance(buf, types.IntType): - return False, "read error %d" % buf - if len(buf) != len(data): - return False, "read %d/%d bytes" % (len(buf), len(data)) - return True, "read '%s'" % buf - - def test_lstat(vol, path, data): - mypath = path + ".io" - sb = vol.lstat(mypath) - if isinstance(sb, types.IntType): - return False, "lstat error %d" % sb - if sb.st_size != len(data): - return False, "lstat size is %d, expected %d" % ( - sb.st_size, len(data)) - return True, "lstat got correct size %d" % sb.st_size - - def test_rename(vol, path, data): - opath = path + ".io" - npath = path + ".tmp" - rc = vol.rename(opath, npath) - if rc < 0: - return False, "rename error %d" % rc - try: - with vol.open(opath, os.O_RDWR) as fd: - return False, "old path working (%s) after rename" % fd - except OSError: - pass - else: - return False, "old path working after rename" - - with vol.open(npath, os.O_RDWR) as nfd: - if not isinstance(nfd, File): - return False, "new path not working after rename" - return True, "rename worked" - - def test_unlink(vol, path, data): - mypath = path + ".tmp" - rc = vol.unlink(mypath) - if rc < 0: - return False, "unlink error %d" % rc - - try: - with vol.open(mypath, os.O_RDWR) as fd: - return False, "old path working (%s) after unlink" % fd - except OSError: - pass - else: - return False, "path still usable after unlink" - - return True, "unlink worked" - - def test_mkdir(vol, path, data): - mypath = path + ".dir" - rc = vol.mkdir(mypath) - if rc < 0: - return False, "mkdir error %d" % rc - - return True, "mkdir worked" - - def test_create_in_dir(vol, path, data): - mypath = path + ".dir/probe" - with vol.creat(mypath, os.O_RDWR, 0644) as fd: - if not isinstance(fd, File): - return False, "create (in dir) error" - return True, "create (in dir) worked" - - def test_dir_listing(vol, path, data): - mypath = path + ".dir" - fd = vol.opendir(mypath) - if not isinstance(fd, Dir): - return False, "opendir error %d" % fd - files = [] - while True: - ent = fd.next() - if not isinstance(ent, Dirent): - break - name = ent.d_name[:ent.d_reclen] - files.append(name) - if files != [".", "..", "probe"]: - return False, "wrong directory contents" - return True, "directory listing worked" - - def test_unlink_in_dir(vol, path, data): - mypath = path + ".dir/probe" - rc = vol.unlink(mypath) - if rc < 0: - return False, "unlink (in dir) error %d" % rc - return True, "unlink (in dir) worked" - - def test_rmdir(vol, path, data): - mypath = path + ".dir" - rc = vol.rmdir(mypath) - if rc < 0: - return False, "rmdir error %d" % rc - try: - vol.lstat(mypath) - except OSError: - pass - else: - return False, "dir still there after rmdir" - return True, "rmdir worked" - - def test_setxattr(vol, path, data): - mypath = path + ".xa" - with vol.creat(mypath, os.O_RDWR | os.O_EXCL, 0644) as fd: - if not fd: - return False, "creat (xattr test) error" - - key1, key2 = "hello", "goodbye" - if vol.setxattr(mypath, "trusted.key1", key1, len(key1)) < 0: - return False, "setxattr (key1) error" - if vol.setxattr(mypath, "trusted.key2", key2, len(key2)) < 0: - return False, "setxattr (key2) error" - return True, "setxattr worked" - - def test_getxattr(vol, path, data): - mypath = path + ".xa" - buf = vol.getxattr(mypath, "trusted.key1", 32) - if isinstance(buf, types.IntType): - return False, "getxattr error" - if buf != "hello": - return False, "wrong getxattr value %s" % buf - return True, "getxattr worked" - - def test_listxattr(vol, path, data): - mypath = path + ".xa" - xattrs = vol.listxattr(mypath) - if isinstance(xattrs, types.IntType): - return False, "listxattr error" - if xattrs != ["trusted.key1", "trusted.key2"]: - return False, "wrong listxattr value %s" % repr(xattrs) - return True, "listxattr worked" - - def test_fallocate(vol, path, data): - mypath = path + ".io" - with vol.creat(mypath, os.O_WRONLY | os.O_EXCL, 0644) as fd: - if not fd: - return False, "creat error" - rc = fd.fallocate(0, 0, 1024 * 1024) - if rc != 0: - return False, "fallocate error" - rc = fd.discard(4096, 4096) - if rc != 0: - return False, "discard error" - return True, "fallocate/discard worked" - - test_list = ( - test_create_write, - #test_open_read, - test_lstat, - test_rename, - test_unlink, - test_mkdir, - test_create_in_dir, - test_dir_listing, - test_unlink_in_dir, - test_rmdir, - test_setxattr, - test_getxattr, - test_listxattr, - test_fallocate, - ) - - ok_to_fail = ( - # TBD: this fails opening the new file, even though the file - # did get renamed. Looks like a gfapi bug, not ours. - (test_rename, "new path not working after rename"), - # TBD: similar, call returns error even though it worked - (test_rmdir, "dir still there after rmdir"), - ) - - volid, path = sys.argv[1:3] - data = "fubar" - vol = Volume("localhost", volid) - vol.set_logging("/dev/null", 7) - #vol.set_logging("/dev/stderr",7) - vol.mount() - - failures = 0 - expected = 0 - for t in test_list: - rc, msg = t(vol, path, data) - if rc: - print "PASS: %s" % msg - else: - print "FAIL: %s" % msg - failures += 1 - for otf in ok_to_fail: - if (t == otf[0]) and (msg == otf[1]): - print " (skipping known failure)" - expected += 1 - break # from the *inner* for loop - else: - break # from the *outer* for loop - - print "%d failures (%d expected)" % (failures, expected) diff --git a/test/functional/__init__.py b/test/functional/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/test/functional/__init__.py diff --git a/test/functional/libgfapi-python-tests.py b/test/functional/libgfapi-python-tests.py new file mode 100644 index 0000000..30a1732 --- /dev/null +++ b/test/functional/libgfapi-python-tests.py @@ -0,0 +1,127 @@ +import unittest +import os +import types +import loremipsum + +from nose import SkipTest +from gluster import gfapi + + +class FileOpsTest(unittest.TestCase): + + vol = None + path = None + data = None + + @classmethod + def setUpClass(cls): + cls.vol = gfapi.Volume("gfshost", "test") + cls.vol.set_logging("/dev/null", 7) + cls.vol.mount() + + @classmethod + def tearDownClass(cls): + cls.vol = None + + def setUp(self): + self.data = loremipsum.get_sentence() + self.path = self._testMethodName + ".io" + with self.vol.creat(self.path, os.O_WRONLY | os.O_EXCL, 0644) as fd: + rc = fd.write(self.data) + + def tearDown(self): + self.path = None + self.data = None + + def test_open_and_read(self): + with self.vol.open(self.path, os.O_RDONLY) as fd: + self.assertTrue(isinstance(fd, gfapi.File)) + buf = fd.read(len(self.data)) + self.assertFalse(isinstance(buf, types.IntType)) + self.assertEqual(buf, self.data) + + def test_lstat(self): + sb = self.vol.lstat(self.path) + self.assertFalse(isinstance(sb, types.IntType)) + self.assertEqual(sb.st_size, len(self.data)) + + def test_rename(self): + newpath = self.path + ".rename" + ret = self.vol.rename(self.path, newpath) + self.assertEqual(ret, 0) + self.assertRaises(OSError, self.vol.lstat, self.path) + + def test_unlink(self): + ret = self.vol.unlink(self.path) + self.assertEqual(ret, 0) + self.assertRaises(OSError, self.vol.lstat, self.path) + + def test_xattr(self): + key1, key2 = "hello", "world" + ret1 = self.vol.setxattr(self.path, "trusted.key1", key1, len(key1)) + self.assertEqual(0, ret1) + ret2 = self.vol.setxattr(self.path, "trusted.key2", key2, len(key2)) + self.assertEqual(0, ret2) + + xattrs = self.vol.listxattr(self.path) + self.assertFalse(isinstance(xattrs, types.IntType)) + self.assertEqual(xattrs, ["trusted.key1", "trusted.key2"]) + + buf = self.vol.getxattr(self.path, "trusted.key1", 32) + self.assertFalse(isinstance(buf, types.IntType)) + self.assertEqual(buf, "hello") + + +class DirOpsTest(unittest.TestCase): + + data = None + dir_path = None + file_path = None + testfile = None + + @classmethod + def setUpClass(cls): + cls.vol = gfapi.Volume("gfshost", "test") + cls.vol.set_logging("/dev/null", 7) + cls.vol.mount() + cls.testfile = "testfile.io" + + @classmethod + def tearDownClass(cls): + cls.vol = None + cls.testfile = None + + def setUp(self): + self.data = loremipsum.get_sentence() + self.dir_path = self._testMethodName + "_dir" + self.vol.mkdir(self.dir_path) + self.file_path = self.dir_path + "/" + self.testfile + with self.vol.creat( + self.file_path, os.O_WRONLY | os.O_EXCL, 0644) as fd: + rc = fd.write(self.data) + + def tearDown(self): + self.dir_path = None + self.file_path = None + self.data = None + + def test_dir_listing(self): + fd = self.vol.opendir(self.dir_path) + self.assertTrue(isinstance(fd, gfapi.Dir)) + files = [] + while True: + ent = fd.next() + if not isinstance(ent, gfapi.Dirent): + break + name = ent.d_name[:ent.d_reclen] + files.append(name) + self.assertEqual(files, [".", "..", self.testfile]) + + def test_delete_file_and_dir(self): + ret = self.vol.unlink(self.file_path) + self.assertEqual(ret, 0) + self.assertRaises(OSError, self.vol.lstat, self.file_path) + + ret = self.vol.rmdir(self.dir_path) + self.assertEqual(ret, 0) + self.assertRaises(OSError, self.vol.lstat, self.dir_path) diff --git a/tools/functional_tests.sh b/tools/functional_tests.sh new file mode 100755 index 0000000..a347e1e --- /dev/null +++ b/tools/functional_tests.sh @@ -0,0 +1,65 @@ +#!/bin/bash + +# Copyright (c) 2014 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +# implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# This program expects to be run by tox in a virtual python environment +# so that it does not pollute the host development system + +sudo_env() +{ + sudo bash -c "PATH=$PATH $*" +} + +cleanup() +{ + sudo rm -rf /export/brick/b1/* > /dev/null 2>&1 +} + +quit() +{ + echo "$1" + exit 1 +} + + +fail() +{ + cleanup + quit "$1" +} + +### MAIN ### + + +# Check the directories exist +DIRS="/export/brick/b1" +for d in $DIRS ; do + if [ ! -x $d ] ; then + quit "$d must exist as GlusterFS volume" + fi +done + + +mkdir functional_tests > /dev/null 2>&1 +nosetests -v --exe \ + --with-xunit \ + --xunit-file functional_tests/libgfapi-python.xml \ + --with-html-output \ + --html-out-file functional_tests/libgfapi-python-result.html \ + test/functional || fail "Functional tests failed" + +cleanup +exit 0 diff --git a/tools/test-requires b/tools/test-requires index fdebbb6..2e47f19 100644 --- a/tools/test-requires +++ b/tools/test-requires @@ -10,3 +10,4 @@ openstack.nose_plugin nosehtmloutput sphinx>=1.1.2 mock>=0.8.0 +loremipsum @@ -19,6 +19,10 @@ commands = nosetests -v --exe --with-xunit --with-coverage --cover-package glust [tox:jenkins] downloadcache = ~/cache/pip +[testenv:functest] +changedir = {toxinidir} +commands = bash tools/functional_tests.sh + [testenv:pep8] deps = --download-cache={homedir}/.pipcache |