summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--README.md168
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/brick_ops.py72
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/devices.py170
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py127
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py361
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py319
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/mount_ops.py302
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py319
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/snap_ops.py307
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py519
10 files changed, 2662 insertions, 2 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..422cc82
--- /dev/null
+++ b/README.md
@@ -0,0 +1,168 @@
+# Glusto Libs
+
+`glusto-libs` repo contains the Libraries/Modules necessary for automating the gluster tests.
+It mainly provides python bindings for the GlusterD-2.0 APIs.
+Latest Code for this repo is managed on review.gluster.org
+
+
+Refer the [glusto-doc](http://glusto.readthedocs.io/en/latest/) for info
+on `glusto` framework.
+Issues need to be filled against the
+[Github](https://github.com/gluster/glusto-libs/issues) repo.
+
+
+To automate/run glusto-tests on GD2 environment we need to do following steps:
+-----------------------------------------------------------------------------
+- install `glusto`
+- clone `glusto-libs` repo
+- clone `glusto-tests` repo
+- install all packages (i.e, glustolibs-gluster-gd2, glustolibs-io and glustolibs-misc libraries)
+
+
+How to install glusto:
+----------------------
+One can use either of the three methods.
+
+- using pip
+
+ # pip install --upgrade git+git://github.com/loadtheaccumulator/glusto.git
+
+- using git
+
+ # git clone https://github.com/loadtheaccumulator/glusto.git
+ # cd glusto
+ # python setup.py install
+
+- using ansible: install glusto, glusto-tests
+
+ # ansible-playbook -i host.ini glusto-tests/ansible/deploy-glusto.yaml
+
+
+For more info refer the [docs](http://glusto.readthedocs.io/en/latest/userguide/install.html).
+
+
+How to clone glusto-tests and glusto-libs repo:
+------------------------------------------------
+- using git
+
+ # git clone ssh://user-name@review.gluster.org/glusto-libs
+
+- using git
+
+ # git clone ssh://user-name@review.gluster.org/glusto-tests
+
+
+How to install the glustolibs-gluster-gd2, glustolibs-io and glustolibs-misc libraries:
+---------------------------------------------------------------------------------------
+ # git clone ssh://user-name@review.gluster.org/glusto-libs
+ # cd glusto-libs/glustolibs-gluster-gd2
+ # python setup.py install
+ # cd ../../glusto-tests/glustolibs-io
+ # python setup.py install
+ # cd ../../glusto-tests/glustolibs-misc
+ # python setup.py install
+
+
+To install glusto-tests dependencies:
+-------------------------------------
+`python-docx` needs to be installed when we run IO's and validates on client node.
+
+- To install run:
+
+ *# easy_install pip
+ # pip install --pre python-docx*
+
+How to run the test case:
+-------------------------
+- Update the information about the servers, clients,
+ servers_info, client_info, running_on_volumes, running_on_mounts
+ and volume_create_force etc, which is necessary on the config
+ file[config](https://github.com/gluster/glusto-tests/blob/master/tests/gluster_tests_config.yml).
+ Refer the following for more info [link](http://glusto.readthedocs.io/en/latest/userguide/configurable.html).
+
+- glusto-tests are run using the `glusto` command available after installing
+ the glusto framework. The various options to run tests as provided by
+ glusto framework:
+
+ - To run PyTest tests:
+
+ - To run all tests that are marked with tag 'bvt':
+ *# glusto -c config.yml --pytest='-v -x tests -m bvt'*
+
+ - To run all tests that are under bvt folder:
+ *# glusto -c config.yml --pytest='-v -s bvt/'*
+
+ - To run a single test case:
+ *# glusto -c config.yml --pytest='-v -s -k test_demo1'*
+
+ For more info about running these tests, refer the [docs](http://glusto.readthedocs.io/en/latest/userguide/glusto.html#options-for-running-unit-tests).
+
+
+Writing tests/libraries for GD2:
+--------------------------------
+ - `tests` directory in glusto-tests contain testcases. Testcases are written as component wise.
+Testcases name and file name should should start with test_.
+
+ - `glustolibs-gluster-gd2` directory in glusto-libs contains libraries for GD2 api's.
+Libraries for io's and miscellaneous are written on `glustolibs-io` and `glustolibs-misc`
+respectively. These functions or libraries can be used while writing testcases.
+
+ - While writing testcases or libraries follow:
+
+ - Please follow the [PEP008 style-guide](https://www.python.org/dev/peps/pep-0008/).
+ - Makes sure all the pylint and pyflakes error are fixed
+ For example:
+
+ - C0326: Exactly one space required around assignment
+ - C0111: Missing module doc-string (missing-doc string)
+ - W: 50: Too long line
+
+ For more information on [pylint](https://docs.pylint.org/en/1.6.0/tutorial.html) and on [pyflakes](http://flake8.pycqa.org/en/latest/user/error-codes.html)
+
+
+
+ - Optimize the code as much as possible. Eliminate the repetitive steps, write it has separate function.
+ - Use proper python standards on returning values.
+ This style guide is a list of do's and don’ts for [Python programs](http://google.github.io/styleguide/pyguide.html).
+
+ - Add docstring to every function you write
+
+ For example: This is an example of a module level function
+
+ def module(param1, param2):
+ """
+ Explain what the module function does in breif
+
+ Args:
+ param1: The first parameter.
+ param2: The second parameter.
+
+ Returns:
+ The return value of the function.
+ """
+
+ - Make sure the log messages are grammatically correct and have no spelling mistakes.
+ - Comment every step of the test case/libraries, log the test step, test result, failure and success.
+ For example:
+
+ # peer status from mnode
+ g.log.info("Get peer status from node %s", self.mnode)
+ ret, out, err = peer_status(self.mnode)
+ self.assertEqual(ret, 0, "Failed to get peer status from node %s: %s" % (self.mnode, err))
+ g.log.info("Successfully got peer status from node %s:\n%s", self.mnode, out)
+ - Don't not use `print` statements in test-cases/libraries because prints statements are not captured in log files. Use logger functions to dump messages into log file.
+
+Logging:
+--------
+Log file name and Log level can be passed as argument to glusto command while
+running the glusto-tests. For example:
+
+ # glusto -c 'config.yml' -l /tmp/glustotests_bvt.log --log-level DEBUG --pytest='-v -x tests -m bvt'
+
+One can configure log files, log levels in the testcases as well. For details
+on how to use glusto framework for configuring logs in tests Refer the following [docs](http://glusto.readthedocs.io/en/latest/userguide/loggable.html).
+
+Default log location is: `/tmp/glustomain.log`
+
+Note: When using `glusto` via the Python Interactive Interpreter,
+the default log location is `/tmp/glusto.log`.
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/brick_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/brick_ops.py
new file mode 100644
index 0000000..bc18b5a
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/brick_ops.py
@@ -0,0 +1,72 @@
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import httplib
+from glustolibs.gluster.rest import RestClient
+from glustolibs.gluster.exceptions import (GlusterApiInvalidInputs)
+from glustolibs.gluster.volume_ops import validate_brick
+
+
+"""This module contains the python glusterd2 brick related api's implementation."""
+
+
+def add_brick(
+ mnode, volname, bricks_list, force=False,
+ replica_count=0, arbiter_count=0):
+ """Add Bricks specified in the bricks_list to the volume.
+ Args:
+ mnode (str): None on which the commands are executed.
+ volname (str): Name of the volume
+ bricks_list (list): List of bricks to be added
+ Kwargs:
+ force (bool): If this option is set to True, then add brick command
+ will get executed with force option. If it is set to False,
+ then add brick command will get executed without force option
+ **kwargs
+ The keys, values in kwargs are:
+ - replica_count : (int)
+ - arbiter_count : (int)
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err' is of type 'str' and is the status
+ error msg of operation else returns None.
+ Example:
+ add_brick(mnode, volname, bricks_list)
+ """
+ if len(bricks_list) <= 0:
+ raise GlusterApiInvalidInputs("Bricks cannot be empty")
+
+ req_bricks = validate_brick(bricks_list)
+ if req_bricks is None:
+ raise GlusterApiInvalidInputs("Invalid Brick details, bricks "
+ "should be in form of "
+ "<peerid>:<path>")
+ # To create a brick dir
+ create_brick_dir = {"create-brick-dir": True}
+
+ data = {
+ "ReplicaCount" : replica_count,
+ "Bricks" : req_bricks,
+ "Force" : force,
+ "Flags": create_brick_dir
+ }
+
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/expand" % volname, httplib.OK, data)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/devices.py b/glustolibs-gluster-gd2/glustolibs/gluster/devices.py
new file mode 100644
index 0000000..35efd49
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/devices.py
@@ -0,0 +1,170 @@
+# Copyright (C) 2018-2019 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""
+ Description: Library for gluster device operation.
+"""
+
+
+import httplib
+from glusto.core import Glusto as g
+from glustolibs.gluster.exceptions import GlusterApiInvalidInputs
+from glustolibs.gluster.lib_utils import validate_peer_id
+from glustolibs.gluster.rest import RestClient
+
+
+def rest_call(ops, mnode, method, path, code, data):
+ """
+ To handle the get methods of devices
+ Args:
+ ops (str): operation performing on devices
+ mnode (str): Node on which commands as to run
+ method (str): rest methods, i.e POST, GET, DELETE
+ path(str): path of the operation
+ e.g: /v1/devices
+ code(str): status code of the operation
+ data(dict): json input
+ Returns:
+ tuple: Tuple containing two elements (ret, out|err)
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation on success
+ The third element 'err' is of type 'dict' and is the
+ error message and code of operation on failure
+ """
+ output = ""
+ ret, out, err = RestClient(mnode).handle_request(method, path, code, data)
+ if ret:
+ g.log.error("Failed to perform device %s operation", ops)
+ output = err
+ else:
+ output = out
+ return ret, output
+
+
+def device_add(mnode, peerid, device):
+ """
+ Gluster device add.
+ Args:
+ mnode (string) : Node on which command as to run
+ peerid (string) : Peer UUID
+ device (string) : device name
+ Returns:
+ tuple: Tuple containing two elements (ret, out|err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation on success
+ The third element 'err' is of type 'str' and is the
+ error message and code of operation on failure
+ """
+ validate_peer_id(peerid)
+ if not device:
+ raise GlusterApiInvalidInputs("Invalid device specified %s" % device)
+ data = {
+ "Device": device
+ }
+ return rest_call("add", mnode, "POST",
+ "/v1/devices/%s" % peerid,
+ httplib.CREATED, data)
+
+
+def device_info(mnode, peerid, device):
+ """
+ Gluster get devices in peer.
+ Args:
+ mnode (string) : Node on which command as to run
+ peerid (string): peerid returned from peer_add
+ devices (dict): device which info needed
+ Returns:
+ tuple: Tuple containing two elements (ret, out|err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation on success
+ The third element 'err' is of type 'str' and is the
+ error message and code of operation on failure
+ """
+ validate_peer_id(peerid)
+ if not device:
+ raise GlusterApiInvalidInputs("Invalid device specified %s" % device)
+ device = {"device": device}
+ return rest_call("info", mnode, "GET",
+ "/v1/devices/%s/%s" % (peerid, device),
+ httplib.OK, None)
+
+
+def devices_in_peer(mnode, peerid):
+ """
+ Gluster get devices in peer.
+ Args:
+ mnode (string) : Node on which command as to run
+ peerid (string): peerid returned from peer_add
+ Returns:
+ tuple: Tuple containing two elements (ret, out|err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation on success
+ The third element 'err' is of type 'str' and is the
+ error message and code of operation on failure
+ """
+ validate_peer_id(peerid)
+ return rest_call("list", mnode, "GET",
+ "/v1/devices/%s" % peerid,
+ httplib.OK, None)
+
+
+def devices(mnode):
+ """
+ Gluster list all devices.
+ Args:
+ mnode (string) : Node on which command as to run
+ Returns:
+ tuple: Tuple containing three elements (ret, out|err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation on success
+ The third element 'err' is of type 'str' and is the
+ error message and code of operation on failure.
+ """
+ return rest_call("list", mnode, "GET",
+ "/v1/devices", httplib.OK, None)
+
+
+def device_edit(mnode, peerid, device, state):
+ """
+ Gluster edit the device
+ Args:
+ mnode (string) : Node on which command as to run
+ device (string) : device name
+ state (string) : state of the device.
+ Either enabled or disabled
+ Returns:
+ tuple: Tuple containing two elements (ret, out|err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation on success
+ The third element 'err' is of type 'str' and is the
+ error message and code of operation on failure
+ """
+ validate_peer_id(peerid)
+ if not device:
+ raise GlusterApiInvalidInputs("Invalid device specified %s" % device)
+ device = {"device": device}
+ data = {
+ "state": state
+ }
+ return rest_call("edit", mnode, "POST",
+ "/v1/devices/%s/%s" % (peerid, device),
+ httplib.CREATED, data)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py b/glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py
new file mode 100644
index 0000000..49db0c2
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py
@@ -0,0 +1,127 @@
+#!/usr/bin/env python
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+"""Exceptions for Gluster libraries and tests"""
+
+
+class GlusterError(Exception):
+ """Base Gluster exception class."""
+ def __init__(self, arg):
+ Exception.__init__(self, arg)
+ self.msg = arg
+
+
+class TransportEndpointNotConnectedError(GlusterError):
+ """Exception for transport endpoint not connected error."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class NoSuchFileOrDirectoryError(GlusterError):
+ """Exception for no such file or directory error."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class AttributesDoNotMatchError(GlusterError):
+ """Attributes do not match exception."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutIsNotCompleteError(GlusterError):
+ """Exception raised when the layout of a file is not complete."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutIsNotBalancedError(GlusterError):
+ """Exception raised when the layout of a file is not balanced."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutHasHolesError(GlusterError):
+ """Exception raised when the layout of a file has holes."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutHasOverlapsError(GlusterError):
+ """Exception raised when the layout of a file has overlaps."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class FileDoesNotExistOnHashedBricksError(GlusterError):
+ """Exception raised when a file/dir does not exist where it is hashed."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class ConfigError(Exception):
+ '''
+ Custom exception thrown when there is an unrecoverable configuration error.
+ For example, a required configuration key is not found.
+ '''
+ pass
+
+
+class ExecutionError(Exception):
+ '''
+ Custom exception thrown when a command executed by Glusto results in an
+ unrecoverable error.
+ For example, all hosts are not in peer state or a volume canot be setup.
+ '''
+ pass
+
+
+class ExecutionParseError(Exception):
+ '''
+ Custom exception thrown when parsing a command executed by Glusto
+ results in an unexpected error.
+ For example, the output of a command when has to be parsed, can have three
+ states. First, the output was as expected. Second, didn't get the expected
+ output after the parsing result and Third, didn't get the expected result
+ as the command itself failed.
+ '''
+ pass
+
+
+class GlusterApiInvalidInputs(Exception):
+ """
+ Custom exception thrown when parsing invalid json inputs for the
+ particular operation
+ For example, invalid peer id specified
+ """
+ pass
+
+
+class GlusterApiError(Exception):
+ """
+ Custom exception thrown when executing rest-api's results in an
+ unrecoverable error.
+ """
+ pass
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py b/glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py
new file mode 100644
index 0000000..138d5e1
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py
@@ -0,0 +1,361 @@
+# Copyright (C) 2019 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+"""
+ Description: Module containing GlusterBaseClass which defines all the
+ variables necessary for tests.
+"""
+
+import unittest
+from glusto.core import Glusto as g
+from glustolibs.gluster.exceptions import ConfigError, ExecutionError
+from glustolibs.gluster.peer_ops import is_peer_connected, peer_status
+from glustolibs.io.utils import log_mounts_info
+from glustolibs.gluster.lib_utils import (
+ get_ip_from_hostname, configure_volumes,
+ configure_mounts, inject_msg_in_gluster_logs,
+ configure_logs, set_conf_entity)
+from glustolibs.gluster.volume_ops import set_volume_options
+from glustolibs.gluster.volume_libs import (setup_volume,
+ cleanup_volume,
+ log_volume_info_and_status)
+
+
+class runs_on(g.CarteTestClass):
+ """Decorator providing runs_on capability for standard unittest script"""
+
+ def __init__(self, value):
+ # the names of the class attributes set by the runs_on decorator
+ self.axis_names = ['volume_type', 'mount_type']
+
+ # the options to replace 'ALL' in selections
+ self.available_options = [['distributed', 'replicated',
+ 'distributed-replicated'],
+ ['glusterfs']]
+
+ # these are the volume and mount options to run and set in config
+ # what do runs_on_volumes and runs_on_mounts need to be named????
+ run_on_volumes = self.available_options[0]
+ run_on_mounts = self.available_options[1]
+ if g.config.get('gluster', "")['running_on_volumes']:
+ run_on_volumes = g.config['gluster']['running_on_volumes']
+ if g.config.get('gluster', "")['running_on_mounts']:
+ run_on_mounts = g.config['gluster']['running_on_mounts']
+
+ # selections is the above info from the run that is intersected with
+ # the limits from the test script
+ self.selections = [run_on_volumes, run_on_mounts]
+
+ # value is the limits that are passed in by the decorator
+ self.limits = value
+
+
+class GlusterBaseClass(unittest.TestCase):
+ """GlusterBaseClass to be subclassed by Gluster Tests.
+ This class reads the config for variable values that will be used in
+ gluster tests. If variable values are not specified in the config file,
+ the variable are defaulted to specific values.
+ """
+
+ volume_type = None
+ mount_type = None
+
+ @classmethod
+ def validate_peers_are_connected(cls):
+ """Validate whether each server in the cluster is connected to
+ all other servers in cluster.
+ Returns (bool): True if all peers are in connected with other peers.
+ False otherwise.
+ """
+ # Validate if peer is connected from all the servers
+ g.log.info("Validating if servers %s are connected from other servers "
+ "in the cluster", cls.servers)
+ for server in cls.servers:
+ g.log.info("Validate servers %s are in connected from node %s",
+ cls.servers, server)
+ ret = is_peer_connected(server, cls.servers)
+ if not ret:
+ g.log.error("Some or all servers %s are not in connected "
+ "state from node %s", cls.servers, server)
+ return False
+ g.log.info("Successfully validated servers %s are all in "
+ "connected state from node %s",
+ cls.servers, server)
+ g.log.info("Successfully validated all servers %s are in connected "
+ "state from other servers in the cluster", cls.servers)
+
+ # Peer Status from mnode
+ peer_status(cls.mnode)
+
+ return True
+
+ @classmethod
+ def setup_volume(cls, volume_create_force=False):
+ """Setup the volume:
+ - Create the volume, Start volume, Set volume
+ options, enable snapshot/quota/tier if specified in the config
+ file.
+ - Wait for volume processes to be online
+ - Export volume as NFS/SMB share if mount_type is NFS or SMB
+ - Log volume info and status
+ Args:
+ volume_create_force(bool): True if create_volume should be
+ executed with 'force' option.
+ Returns (bool): True if all the steps mentioned in the descriptions
+ passes. False otherwise.
+ """
+ force_volume_create = False
+ if cls.volume_create_force:
+ force_volume_create = True
+
+ # Validate peers before setting up volume
+ g.log.info("Validate peers before setting up volume ")
+ ret = cls.validate_peers_are_connected()
+ if not ret:
+ g.log.error("Failed to validate peers are in connected state "
+ "before setting up volume")
+ return False
+ g.log.info("Successfully validated peers are in connected state "
+ "before setting up volume")
+
+ # Setup Volume
+ g.log.info("Setting up volume %s", cls.volname)
+ ret = setup_volume(mnode=cls.mnode,
+ all_servers_info=cls.all_servers_info,
+ volume_config=cls.volume, force=force_volume_create)
+ if not ret:
+ g.log.error("Failed to Setup volume %s", cls.volname)
+ return False
+ g.log.info("Successful in setting up volume %s", cls.volname)
+
+ # ToDo : Wait for volume processes to be online
+
+ # Log Volume Info and Status
+ g.log.info("Log Volume %s Info and Status", cls.volname)
+ ret = log_volume_info_and_status(cls.mnode, cls.volname)
+ if not ret:
+ g.log.error("Logging volume %s info and status failed",
+ cls.volname)
+ return False
+ g.log.info("Successful in logging volume %s info and status",
+ cls.volname)
+
+ return True
+
+ @classmethod
+ def mount_volume(cls, mounts):
+ """Mount volume
+ Args:
+ mounts(list): List of mount_objs
+ Returns (bool): True if mounting the volume for a mount obj is
+ successful. False otherwise
+ """
+ g.log.info("Starting to mount volume %s", cls.volname)
+ for mount_obj in mounts:
+ g.log.info("Mounting volume '%s:%s' on '%s:%s'",
+ mount_obj.server_system, mount_obj.volname,
+ mount_obj.client_system, mount_obj.mountpoint)
+ ret = mount_obj.mount()
+ if not ret:
+ g.log.error("Failed to mount volume '%s:%s' on '%s:%s'",
+ mount_obj.server_system, mount_obj.volname,
+ mount_obj.client_system, mount_obj.mountpoint)
+ return False
+ else:
+ g.log.info("Successful in mounting volume '%s:%s' on "
+ "'%s:%s'", mount_obj.server_system,
+ mount_obj.volname, mount_obj.client_system,
+ mount_obj.mountpoint)
+ g.log.info("Successful in mounting all mount objs for the volume %s",
+ cls.volname)
+
+ # Get mounts info
+ g.log.info("Get mounts Info:")
+ log_mounts_info(mounts)
+
+ return True
+
+ @classmethod
+ def setup_volume_and_mount_volume(cls, mounts, volume_create_force=False):
+ """Setup the volume and mount the volume
+ Args:
+ mounts(list): List of mount_objs
+ volume_create_force(bool): True if create_volume should be
+ executed with 'force' option.
+ Returns (bool): True if setting up volume and mounting the volume
+ for a mount obj is successful. False otherwise
+ """
+ # Setup Volume
+ _rc = cls.setup_volume(volume_create_force)
+ if not _rc:
+ return _rc
+
+ # Mount Volume
+ _rc = cls.mount_volume(mounts)
+ if not _rc:
+ return _rc
+
+ return True
+
+ @classmethod
+ def unmount_volume(cls, mounts):
+ """Unmount all mounts for the volume
+ Args:
+ mounts(list): List of mount_objs
+ Returns (bool): True if unmounting the volume for a mount obj is
+ successful. False otherwise
+ """
+ # Unmount volume
+ g.log.info("Starting to UnMount Volume %s", cls.volname)
+ for mount_obj in mounts:
+ g.log.info("UnMounting volume '%s:%s' on '%s:%s'",
+ mount_obj.server_system, mount_obj.volname,
+ mount_obj.client_system, mount_obj.mountpoint)
+ ret = mount_obj.unmount()
+ if not ret:
+ g.log.error("Failed to unmount volume '%s:%s' on '%s:%s'",
+ mount_obj.server_system, mount_obj.volname,
+ mount_obj.client_system, mount_obj.mountpoint)
+
+ # Get mounts info
+ g.log.info("Get mounts Info:")
+ log_mounts_info(cls.mounts)
+
+ return False
+ else:
+ g.log.info("Successful in unmounting volume '%s:%s' on "
+ "'%s:%s'", mount_obj.server_system,
+ mount_obj.volname, mount_obj.client_system,
+ mount_obj.mountpoint)
+ g.log.info("Successful in unmounting all mount objs for the volume %s",
+ cls.volname)
+
+ # Get mounts info
+ g.log.info("Get mounts Info:")
+ log_mounts_info(mounts)
+
+ return True
+
+ @classmethod
+ def cleanup_volume(cls):
+ """Cleanup the volume
+ Returns (bool): True if cleanup volume is successful. False otherwise.
+ """
+ g.log.info("Cleanup Volume %s", cls.volname)
+ ret = cleanup_volume(mnode=cls.mnode, volname=cls.volname)
+ if not ret:
+ g.log.error("cleanup of volume %s failed", cls.volname)
+ else:
+ g.log.info("Successfully cleaned-up volume %s", cls.volname)
+
+ # Log Volume Info and Status
+ g.log.info("Log Volume %s Info and Status", cls.volname)
+ log_volume_info_and_status(cls.mnode, cls.volname)
+
+ return ret
+
+ @classmethod
+ def unmount_volume_and_cleanup_volume(cls, mounts):
+ """Unmount the volume and cleanup volume
+ Args:
+ mounts(list): List of mount_objs
+ Returns (bool): True if unmounting the volume for the mounts and
+ cleaning up volume is successful. False otherwise
+ """
+ # UnMount Volume
+ _rc = cls.unmount_volume(mounts)
+ if not _rc:
+ return _rc
+
+ # Setup Volume
+ _rc = cls.cleanup_volume()
+ if not _rc:
+ return _rc
+ return True
+
+ @classmethod
+ def setUpClass(cls):
+ """Initialize all the variables necessary for testing Gluster
+ """
+ # Set the values of servers, clients, servers_info and clients_info
+ cls.servers = set_conf_entity('servers')
+ cls.clients = set_conf_entity('clients')
+ cls.all_servers_info = set_conf_entity('servers_info')
+ cls.all_clients_info = set_conf_entity('clients_info')
+
+ # Set mnode : Node on which gluster commands are executed
+ cls.mnode = cls.servers[0]
+
+ # Server IP's
+ cls.servers_ips = []
+ cls.servers_ips = get_ip_from_hostname(cls.servers)
+
+ # Get the volume configuration
+ (cls.default_volume_type_config, cls.volume_create_force,
+ cls.volume, cls.voltype, cls.volname, cls.mnode) = configure_volumes(
+ cls.servers, cls.volume_type)
+
+ # Get the mount configuration.
+ cls.clients, cls.mounts_dict_list, cls.mounts = configure_mounts(
+ cls.mnode, cls.volname, cls.mount_type, cls.all_clients_info)
+
+ # Get gluster Logs info
+ (cls.server_gluster_logs_dirs, cls.server_gluster_logs_files,
+ cls.client_gluster_logs_dirs, cls.client_gluster_logs_files,
+ cls.glustotest_run_id) = configure_logs()
+
+ msg = "Setupclass: %s : %s" % (cls.__name__, cls.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, cls.servers, cls.clients,
+ cls.mount_type, cls.server_gluster_logs_dirs,
+ cls.server_gluster_logs_files,
+ cls.client_gluster_logs_dirs,
+ cls.client_gluster_logs_dirs)
+
+ # Log the baseclass variables for debugging purposes
+ g.log.debug("GlusterBaseClass Variables:\n %s", cls.__dict__)
+
+ def setUp(self):
+ msg = "Starting Test : %s : %s" % (self.id(), self.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, self.servers, self.clients,
+ self.mount_type, self.server_gluster_logs_dirs,
+ self.server_gluster_logs_files,
+ self.client_gluster_logs_dirs,
+ self.client_gluster_logs_dirs)
+
+ def tearDown(self):
+ msg = "Ending Test: %s : %s" % (self.id(), self.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, self.servers, self.clients,
+ self.mount_type, self.server_gluster_logs_dirs,
+ self.server_gluster_logs_files,
+ self.client_gluster_logs_dirs,
+ self.client_gluster_logs_dirs)
+
+ @classmethod
+ def tearDownClass(cls):
+ msg = "Teardownclass: %s : %s" % (cls.__name__, cls.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, cls.servers, cls.clients,
+ cls.mount_type, cls.server_gluster_logs_dirs,
+ cls.server_gluster_logs_files,
+ cls.client_gluster_logs_dirs,
+ cls.client_gluster_logs_dirs)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py b/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py
index 3eb16e2..ebcdd9d 100644
--- a/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+# Copyright (C) 2019 Red Hat, Inc. <http://www.redhat.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -19,7 +18,49 @@
Description: Helper library for gluster modules.
"""
+import os
+import random
+import copy
+import datetime
+import socket
+from uuid import UUID
from glusto.core import Glusto as g
+from glustolibs.gluster.mount_ops import create_mount_objs
+from glustolibs.gluster.expections import (
+ ConfigError, GlusterApiInvalidInputs)
+
+
+def validate_uuid(brick_id, version=4):
+ """
+ Validates the uuid
+ Args:
+ brick_id (str) : Brick_id to be validated
+ version (int)
+ Returns:
+ True (bool) on if the uuid is valid hex code,
+ else false
+ """
+ try:
+ UUID(brick_id, version=version)
+ except ValueError:
+ # If it's a value error, then the string
+ # is not a valid hex code for a UUID.
+ g.log.error("Invalid brick_id %s", brick_id)
+ return False
+ return True
+
+
+def validate_peer_id(peerid):
+ """
+ Validates the peer id
+ Args:
+ peer id (str) : peer id to be validated
+ Returns:
+ Exceptions on failure
+ """
+ if not validate_uuid(peerid):
+ g.log.error("Invalid peer id %s speceified", peerid)
+ raise GlusterApiInvalidInputs("Invalid peer id specified")
def inject_msg_in_logs(nodes, log_msg, list_of_dirs=None, list_of_files=None):
@@ -75,3 +116,277 @@ def inject_msg_in_logs(nodes, log_msg, list_of_dirs=None, list_of_files=None):
log_msg, list_of_dirs, list_of_files, host)
_rc = False
return _rc
+
+
+def inject_msg_in_gluster_logs(msg, servers, clients,
+ mount_type,
+ server_gluster_logs_dirs,
+ server_gluster_logs_files,
+ client_gluster_logs_dirs,
+ client_gluster_logs_files):
+
+ """Inject all the gluster logs on servers, clients with msg
+ Args:
+ msg (str): Message string to be injected
+ Returns:
+ bool: True if injecting msg on the log files/dirs is successful.
+ False Otherwise.
+ """
+ _rc = True
+ # Inject msg on server gluster logs
+ ret = inject_msg_in_logs(servers, log_msg=msg,
+ list_of_dirs=server_gluster_logs_dirs)
+ if not ret:
+ _rc = False
+
+ if mount_type is not None and "glusterfs" in mount_type:
+ ret = inject_msg_in_logs(clients, log_msg=msg,
+ list_of_dirs=client_gluster_logs_dirs,
+ list_of_files=client_gluster_logs_files)
+ if not ret:
+ _rc = False
+ return _rc
+
+
+def get_ip_from_hostname(nodes):
+ """Returns list of IP's for the list of nodes in order.
+ Args:
+ nodes(list|str): List of nodes hostnames
+ Returns:
+ list: List of IP's corresponding to the hostnames of nodes.
+ """
+ nodes_ips = []
+ nodes = to_list(nodes)
+ for node in nodes:
+ try:
+ ip = socket.gethostbyname(node)
+ except socket.gaierror as e:
+ g.log.error("Failed to get the IP of Host: %s : %s", node,
+ e.strerror)
+ ip = None
+ nodes_ips.append(ip)
+ return nodes_ips
+
+
+def set_conf_entity(entity_name):
+ """Set the value of the entity
+ Args:
+ entity_name (str) : Value of entity to be set
+ Returns:
+ Value of the entity
+ """
+ entity = g.config.get(entity_name)
+ if not entity:
+ raise ConfigError("'%s' not defined in the global config" % entity_name)
+ return entity
+
+
+def configure_volumes(servers, volume_type):
+ """Defines the volume configurations.
+ Args:
+ servers(list) : List of servers
+ volume_type(str) : Type of volume which will be created
+ Returns:
+ default_volume_type_config(dict) : Volume type configuration
+ volume_create_force(bool) : Volume with force option
+ volume(dict): Volume configuration
+ volname(str): Volume name
+ voltype(str): Volume type
+ """
+ # Defining default volume_types configuration.
+ default_volume_type_config = {
+ 'distributed': {
+ 'type': 'distributed',
+ 'dist_count': 4,
+ 'transport': 'tcp'
+ },
+ 'replicated': {
+ 'type': 'replicated',
+ 'replica_count': 2,
+ 'arbiter_count': 1,
+ 'transport': 'tcp'
+ },
+ 'distributed-replicated': {
+ 'type': 'distributed-replicated',
+ 'dist_count': 2,
+ 'replica_count': 3,
+ 'transport': 'tcp'
+ }
+ }
+
+ # Check if default volume_type configuration is provided in
+ # config yml
+ if g.config.get('gluster')['volume_types']:
+ default_volume_type_from_config = (g.config['gluster']['volume_types'])
+
+ for vol_type in default_volume_type_from_config.keys():
+ if default_volume_type_from_config[vol_type]:
+ if vol_type in default_volume_type_config:
+ default_volume_type_config[volume_type] = (
+ default_volume_type_from_config[vol_type])
+
+ # Create Volume with force option
+ volume_create_force = False
+ if g.config.get('gluster')['volume_create_force']:
+ volume_create_force = (g.config['gluster']['volume_create_force'])
+
+ # Get the volume configuration.
+ volume = {}
+ if volume_type:
+ found_volume = False
+ if g.config.get('gluster')['volumes']:
+ for volume in g.config['gluster']['volumes']:
+ if volume['voltype']['type'] == volume_type:
+ volume = copy.deepcopy(volume)
+ found_volume = True
+ break
+
+ if found_volume:
+ if 'name' not in volume:
+ volume['name'] = 'testvol_%s' % volume_type
+
+ if 'servers' not in volume:
+ volume['servers'] = servers
+
+ if not found_volume:
+ try:
+ if g.config['gluster']['volume_types'][volume_type]:
+ volume['voltype'] = (g.config['gluster']['volume_types'][volume_type])
+ except KeyError:
+ try:
+ volume['voltype'] = (default_volume_type_config
+ [volume_type])
+ except KeyError:
+ raise ConfigError("Unable to get configs of volume "
+ "type: %s", volume_type)
+ volume['name'] = 'testvol_%s' % volume_type
+ volume['servers'] = servers
+
+ # Define Volume Useful Variables.
+ volname = volume['name']
+ voltype = volume['voltype']['type']
+ servers = volume['servers']
+ mnode = servers[0]
+ return (default_volume_type_config, volume_create_force,
+ volume, voltype, volname, mnode)
+
+
+def configure_mounts(mnode, volname, mount_type, all_clients_info):
+ """Defines the mount configurations.
+ Args:
+ mnode(str): Node on which volume should be mounted
+ volname(str): Name of the volume
+ mount_type(list): Defines the mount type
+ all_clients_info(dict): Dict of clients information
+ Returns:
+ mounts_dict_list(list): List of the mount informations
+ mounts(str) : GlusterMount instance
+ """
+ # Get the mount configuration
+ mounts = []
+ if mount_type:
+ mounts_dict_list = []
+ found_mount = False
+ if g.config.get('gluster')['mounts']:
+ for mount in g.config['gluster']['mounts']:
+ if mount['protocol'] == mount_type:
+ temp_mount = {}
+ temp_mount['protocol'] = mount_type
+ if 'volname' in mount and mount['volname']:
+ if mount['volname'] == volname:
+ temp_mount = copy.deepcopy(mount)
+ else:
+ continue
+ else:
+ temp_mount['volname'] = volname
+ if ('server' not in mount or
+ (not mount['server'])):
+ temp_mount['server'] = mnode
+ else:
+ temp_mount['server'] = mount['server']
+ if ('mountpoint' not in mount or
+ (not mount['mountpoint'])):
+ temp_mount['mountpoint'] = (os.path.join(
+ "/mnt", '_'.join([volname,
+ mount_type])))
+ else:
+ temp_mount['mountpoint'] = mount['mountpoint']
+ if ('client' not in mount or
+ (not mount['client'])):
+ temp_mount['client'] = (
+ all_clients_info[
+ random.choice(
+ all_clients_info.keys())]
+ )
+ else:
+ temp_mount['client'] = mount['client']
+ if 'options' in mount and mount['options']:
+ temp_mount['options'] = mount['options']
+ else:
+ temp_mount['options'] = ''
+ mounts_dict_list.append(temp_mount)
+ found_mount = True
+
+ if not found_mount:
+ for client in all_clients_info.keys():
+ mount = {
+ 'protocol': mount_type,
+ 'server': mnode,
+ 'volname': volname,
+ 'client': all_clients_info[client],
+ 'mountpoint': (os.path.join(
+ "/mnt", '_'.join([volname, mount_type]))),
+ 'options': ''
+ }
+ mounts_dict_list.append(mount)
+ mounts = create_mount_objs(mounts_dict_list)
+
+ # Defining clients from mounts.
+ clients = []
+ for mount in mounts_dict_list:
+ clients.append(mount['client']['host'])
+ clients = list(set(clients))
+
+ return clients, mounts_dict_list, mounts
+
+
+def configure_logs():
+ """Defines the gluster log information.
+ Returns:
+ server_gluster_logs_dirs(list) : List of server logs dirs
+ server_gluster_logs_files(list) : List of server logs files
+ client_gluster_logs_dirs(list) : List of client logs dirs
+ client_gluster_logs_files(list) : List of client logs files
+ glustotest_run_id(str) : Time the test run
+ """
+ # Gluster Logs info
+ server_gluster_logs_dirs = ["/var/log/glusterd2/glusterd2.log"]
+ server_gluster_logs_files = []
+ if g.config.get("gluster")['server_gluster_logs_info']['dirs']:
+ server_gluster_logs_dirs = (
+ g.config['gluster']['server_gluster_logs_info']['dirs'])
+
+ if g.config.get("gluster")['server_gluster_logs_info']['files']:
+ server_gluster_logs_files = (
+ g.config['gluster']['server_gluster_logs_info']['files'])
+
+ client_gluster_logs_dirs = ["/var/log/glusterd2/glusterd2.log"]
+ client_gluster_logs_files = ["/var/log/glusterd2/glusterd2.log"]
+ if g.config.get("gluster")['client_gluster_logs_info']['dirs']:
+ client_gluster_logs_dirs = (
+ g.config['gluster']['client_gluster_logs_info']['dirs'])
+
+ if g.config.get("gluster")['client_gluster_logs_info']['files']:
+ client_gluster_logs_files = (
+ g.config['gluster']['client_gluster_logs_info']['files'])
+
+ # Have a unique string to recognize the test run for logging in
+ # gluster logs
+ if 'glustotest_run_id' not in g.config:
+ g.config['glustotest_run_id'] = (
+ datetime.datetime.now().strftime('%H_%M_%d_%m_%Y'))
+ glustotest_run_id = g.config['glustotest_run_id']
+ g.log.info("Glusto test run id %s", glustotest_run_id)
+ return (server_gluster_logs_dirs, server_gluster_logs_files,
+ client_gluster_logs_dirs, client_gluster_logs_files,
+ glustotest_run_id)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/mount_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/mount_ops.py
new file mode 100644
index 0000000..da0a993
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/mount_ops.py
@@ -0,0 +1,302 @@
+#!/usr/bin/env python
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""
+ Description: Module for Mount operations.
+"""
+
+
+from glusto.core import Glusto as g
+from glustolibs.gluster.exceptions import ConfigError
+import copy
+
+class GlusterMount():
+ """Gluster Mount class
+ Args:
+ mount (dict): Mount dict with mount_protocol, mountpoint,
+ server, client, volname, options
+ Example:
+ mount =
+ {'protocol': 'glusterfs',
+ 'mountpoint': '/mnt/g1',
+ 'server': 'abc.lab.eng.xyz.com',
+ 'client': 'def.lab.eng.xyz.com',
+ 'volname': 'testvol',
+ 'options': ''
+ }
+ Returns:
+ Instance of GlusterMount class
+ """
+ def __init__(self, mount):
+ # Check for missing parameters
+ for param in ['protocol', 'mountpoint', 'server',
+ 'client', 'volname', 'options']:
+ if param not in mount:
+ raise ConfigError("Missing key %s" % param)
+
+ # Get Protocol
+ self.mounttype = mount.get('protocol', 'glusterfs')
+
+ # Get mountpoint
+ if bool(mount.get('mountpoint', False)):
+ self.mountpoint = mount['mountpoint']
+ else:
+ self.mountpoint = "/mnt/%s" % self.mounttype
+
+ # Get server
+ self.server_system = mount.get('server', None)
+
+ # Get client
+ self.client_system = mount.get('client', None)
+
+ # Get Volume name
+ self.volname = mount['volname']
+
+ # Get options
+ self.options = mount.get('options', None)
+
+ def mount(self):
+ """Mounts the volume
+ Args:
+ uses instance args passed at init
+ Returns:
+ bool: True on success and False on failure.
+ """
+ ret, out, err = mount_volume(self.volname, mtype=self.mounttype,
+ mpoint=self.mountpoint,
+ mserver=self.server_system,
+ mclient=self.client_system,
+ options=self.options)
+ if ret:
+ g.log.error("Failed to mount the volume")
+ return False
+ return True
+
+ def is_mounted(self):
+ """Tests for mount on client
+ Args:
+ uses instance args passed at init
+ Returns:
+ bool: True on success and False on failure.
+ """
+ ret = is_volume_mounted(self.volname,
+ mpoint=self.mountpoint,
+ mserver=self.server_system,
+ mclient=self.client_system,
+ mtype=self.mounttype)
+
+ if not ret:
+ g.log.error("Volume is not mounted")
+ return False
+ return True
+
+ def unmount(self):
+ """Unmounts the volume
+ Args:
+ uses instance args passed at init
+ Returns:
+ bool: True on success and False on failure.
+ """
+ (ret, out, err) = umount_volume(mclient=self.client_system,
+ mpoint=self.mountpoint,
+ mtype=self.mounttype)
+ if ret:
+ g.log.error("Failed to unmount the volume")
+ return False
+ return True
+
+
+def is_volume_mounted(volname, mpoint, mserver, mclient, mtype):
+ """Check if mount exist.
+ Args:
+ volname (str): Name of the volume
+ mpoint (str): Mountpoint dir
+ mserver (str): Server to which it is mounted to
+ mclient (str): Client from which it is mounted.
+ mtype (str): Mount type (glusterfs)
+ Returns:
+ bool: True if mounted and False otherwise.
+ """
+ # python will error on missing arg, so just checking for empty args here
+ for param in [volname, mpoint, mserver, mclient, mtype]:
+ if not param:
+ g.log.error("Missing arguments for mount.")
+ return False
+
+ ret, _, _ = g.run(mclient, "mount | grep %s | grep %s | grep \"%s\""
+ % (volname, mpoint, mserver))
+ if ret:
+ g.log.debug("Volume %s is mounted at %s:%s" % (volname, mclient,
+ mpoint))
+ return True
+ g.log.error("Volume %s is not mounted at %s:%s" % (volname,
+ mclient,
+ mpoint))
+ return False
+
+
+def mount_volume(volname, mtype, mpoint, mserver, mclient, options=''):
+ """Mount the gluster volume with specified options.
+ Args:
+ volname (str): Name of the volume to mount.
+ mtype (str): Protocol to be used to mount.
+ mpoint (str): Mountpoint dir.
+ mserver (str): Server to mount.
+ mclient (str): Client from which it has to be mounted.
+ Kwargs:
+ option (str): Options for the mount command.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ (0, '', '') if already mounted.
+ (ret, out, err) of mount commnd execution otherwise.
+ """
+ if is_mounted(volname, mpoint, mserver, mclient, mtype):
+ g.log.debug("Volume %s is already mounted at %s" %
+ (volname, mpoint))
+ return (0, '', '')
+
+ if not options:
+ options = "-o %s" % options
+
+ # Create mount dir
+ g.run(mclient, "test -d %s || mkdir -p %s" % (mpoint, mpoint))
+
+ # Mount
+ mcmd = ("mount -t %s %s %s:/%s %s" %
+ (mtype, options, mserver, volname, mpoint))
+
+ # Create mount
+ return g.run(mclient, mcmd)
+
+
+def umount_volume(mclient, mpoint, mtype=''):
+ """Unmounts the mountpoint.
+ Args:
+ mclient (str): Client from which it has to be mounted.
+ mpoint (str): Mountpoint dir.
+ Kwargs:
+ mtype (str): Mounttype. Defaults to ''.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err) as returned by
+ umount command execution.
+ """
+ cmd = ("umount %s || umount -f %s || umount -l %s" %
+ (mpoint, mpoint, mpoint))
+
+ return g.run(mclient, cmd)
+
+
+def create_mount_objs(mounts):
+ """Creates GlusterMount class objects for the given list of mounts
+ Args:
+ mounts (list): list of mounts with each element being dict having the
+ specifics of each mount
+ Example:
+ mounts: [
+ {'protocol': 'glusterfs',
+ 'mountpoint': '/mnt/g1',
+ 'server': 'abc.lab.eng.xyz.com',
+ 'client': {'host': 'def.lab.eng.xyz.com'},
+ 'volname': 'testvol',
+ 'options': '',
+ 'num_of_mounts': 2}
+ ]
+ Returns:
+ list : List of GlusterMount class objects.
+ Example:
+ mount_objs = create_mount_objs(mounts)
+ """
+ mount_obj_list = []
+ for mount in mounts:
+ temp_mount = copy.deepcopy(mount)
+ if (mount['protocol'] == "glusterfs"):
+ if 'mountpoint' in mount and mount['mountpoint']:
+ temp_mount['mountpoint'] = mount['mountpoint']
+ else:
+ temp_mount['mountpoint'] = ("/mnt/%s_%s" %
+ (mount['volname'],
+ mount['protocol']))
+
+ num_of_mounts = 1
+ if 'num_of_mounts' in mount:
+ if mount['num_of_mounts']:
+ num_of_mounts = mount['num_of_mounts']
+ if num_of_mounts > 1:
+ mount_dir = temp_mount['mountpoint']
+ for count in range(1, num_of_mounts + 1):
+ if mount_dir != "*":
+ temp_mount['mountpoint'] = '_'.join(
+ [mount_dir, str(count)])
+
+ mount_obj_list.append(GlusterMount(temp_mount))
+ else:
+ mount_obj_list.append(GlusterMount(temp_mount))
+
+ return mount_obj_list
+
+
+def operate_mounts(mount_objs, operation):
+ """Mounts/Unmounts using the details as specified
+ in the each mount obj
+ Args:
+ mount_objs (list): list of mounts objects with each element being
+ the GlusterMount class object
+ operation (str): Mount/unmount
+ Returns:
+ bool : True if creating the mount for all mount_objs is successful.
+ False otherwise.
+ Example:
+ ret = operate_mounts(create_mount_objs(mounts), operation='mount')
+ """
+ _rc = True
+ for mount_obj in mount_objs:
+ if operation == 'mount':
+ ret = mount_obj.mount()
+ elif operation == 'unmount':
+ ret = mount_obj.unmount()
+ else:
+ g.log.error("Operation not found")
+ _rc = False
+ return _rc
+
+
+def create_mounts(mount_objs):
+ """Creates Mounts using the details as specified in the each mount obj
+ Args:
+ mount_objs (list): list of mounts objects with each element being
+ the GlusterMount class object
+ Returns:
+ bool : True if creating the mount for all mount_objs is successful.
+ False otherwise.
+ Example:
+ ret = create_mounts(create_mount_objs(mounts))
+ """
+ return operate_mounts(mount_objs, operation='mount')
+
+
+def unmount_mounts(mount_objs):
+ """Unmounts using the details as specified in the each mount obj
+ Args:
+ mount_objs (list): list of mounts objects with each element being
+ the GlusterMount class object
+ Returns:
+ bool : True if unmounting the mount for all mount_objs is successful.
+ False otherwise.
+ Example:
+ ret = unmount_mounts(create_mount_objs(mounts))
+ """
+ return operate_mounts(mount_objs, operation='unmount')
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py
new file mode 100644
index 0000000..f24251b
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py
@@ -0,0 +1,319 @@
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""
+ Description: Library for gluster peer operations.
+"""
+
+import json
+import httplib
+from glustolibs.gluster.rest import RestClient
+from glusto.core import Glusto as g
+
+def peer_probe(mnode, server):
+ """Probe the specified server.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ server (str): Server to be peer probed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the error message
+ and error code of the the command execution.
+ """
+
+ data = {"addresses": [server]}
+ return RestClient(mnode).handle_request('POST', "/v1/peers", httplib.CREATED, data)
+
+
+def pool_list(mnode):
+ """Runs 'gluster pool list' command on the specified node.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+ return RestClient(mnode).handle_request('GET', "/v1/peers", httplib.OK, None)
+
+
+def peer_detach(mnode, server):
+ """ Detach the specified server.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ server (str): Server to be peer detached.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+
+ server_id = get_peer_id(mnode, server)
+ ret, out, err = RestClient(mnode).handle_request('DELETE', "/v1/peers/%s"
+ % server_id, httplib.NO_CONTENT, None)
+ if ret != httplib.NO_CONTENT:
+ returncode = 1
+ g.log.error("Failed to peer detach the node '%s'.", server)
+ else:
+ returncode = 0
+
+ return (returncode, out, err)
+
+
+def peer_status(mnode, peer=None):
+ """ Fetches the peer status
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+
+ path = "/v1/peers"
+ if peer:
+ peerid = get_peer_id(mnode, peer)
+ path = "%s/%s" % (path, peerid)
+ return RestClient(mnode).handle_request('GET', path, httplib.OK, None)
+
+
+def peer_edit(mnode, peerid, zone):
+ """ Edits the peer zone
+_
+ Args:
+ mnode (str): Node on which command has to be executed.
+ peerid (str): The peerid of the peer.
+ Zone (str): The zone details that has to be edited.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+
+ data = {"metadata": {"zone": zone}}
+ return RestClient(mnode).handle_request("POST", "/v1/peers/%s" % peerid,
+ httplib.CREATED, data)
+
+
+def get_peer_id(mnode, server):
+ """
+ Returns the peer_id of the given server
+
+ Args:
+ server (str) : A server to fetch peer-ids
+
+ Returns:
+ server_id (str) : Peer-id of the given server/peer
+ """
+ from glustolibs.gluster.lib_utils import get_ip_from_hostname
+
+ _ip = node = ids = []
+ _ip = get_ip_from_hostname([server])
+ server = ''.join(_ip)
+ _, out, _ = pool_list(mnode)
+ output = json.loads(out)
+ for elem in output:
+ item = elem['client-addresses'][1].split(":")
+ node.append(item[0])
+ item = elem['id']
+ ids.append(item)
+ if server in node:
+ return ids[-1]
+
+def is_peer_connected(mnode, servers):
+ """Checks whether specified peer is in cluster and 'Connected' state.
+
+ Args:
+ mnode (str): Node from which peer probe has to be executed.
+ servers (str): A server| list of servers to be validated.
+
+ Returns
+ bool : True on success (peer in cluster and connected), False on
+ failure.
+ """
+ from glustolibs.gluster.lib_utils import to_list
+
+ servers = to_list(servers)
+
+ for server in servers:
+ _, out, _ = peer_status(mnode, server)
+ out = json.loads(out)
+ if not out['online']:
+ g.log.error("The peer %s is not connected", server)
+ return False
+ return True
+
+
+def nodes_from_pool_list(mnode):
+ """Return list of nodes from the 'gluster pool list'.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+
+ Returns:
+ NoneType: None if command execution fails.
+ list: List of nodes in pool on Success, Empty list on failure.
+ """
+ _, pool_list_data, _ = pool_list(mnode)
+ server_list = json.loads(pool_list_data)
+ if server_list is None:
+ g.log.error("Unable to get Nodes from the pool list command.")
+ return None
+
+ nodes = []
+ for server in server_list:
+ nodes.append(server['name'])
+ return nodes
+
+
+def peer_probe_servers(mnode, servers, validate=True):
+ """Probe specified servers and validate whether probed servers
+ are in cluster and connected state if validate is set to True.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ servers (str|list): A server|List of servers to be peer probed.
+
+ Kwargs:
+ validate (bool): True to validate if probed peer is in cluster and
+ connected state. False otherwise. Defaults to True.
+
+ Returns:
+ bool: True on success and False on failure.
+ """
+ from glustolibs.gluster.lib_utils import to_list
+
+ servers = to_list(servers)
+
+ if mnode in servers:
+ servers.remove(mnode)
+
+ # Get list of nodes from 'gluster pool list'
+ nodes_in_pool_list = nodes_from_pool_list(mnode)
+ if not nodes_in_pool_list:
+ g.log.error("Unable to get nodes from gluster pool list. "
+ "Failing peer probe.")
+ return False
+
+ for server in servers:
+ if server not in nodes_in_pool_list:
+ ret, _, _ = peer_probe(mnode, server)
+ if ret != 0:
+ g.log.error("Failed to peer probe the node '%s'.", server)
+ return False
+ g.log.info("Successfully peer probed the node '%s'.", server)
+
+ # Validating whether peer is in connected state after peer probe
+ if validate:
+ _rc = False
+ i = 0
+ while i < 200:
+ if is_peer_connected(mnode, servers):
+ _rc = True
+ break
+
+ if not _rc:
+ g.log.error("Peers are in not connected state")
+ g.log.info("All peers are in connected state")
+ return _rc
+
+
+def peer_detach_servers(mnode, servers, validate=True):
+ """Detach peers and validate status of peer if validate is set to True.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ servers (str|list): A server|List of servers to be detached.
+
+ Kwargs:
+ validate (bool): True if status of the peer needs to be validated,
+ False otherwise. Defaults to True.
+
+ Returns:
+ bool: True on success and False on failure.
+ """
+
+ from glustolibs.gluster.lib_utils import to_list
+
+ servers = to_list(servers)
+
+ if mnode in servers:
+ servers.remove(mnode)
+
+ for server in servers:
+ ret, _, _ = peer_detach(mnode, server)
+ if ret:
+ g.log.error("Failed to peer detach the node '%s'.", server)
+ return False
+
+ # Validating whether peer detach is successful
+ if validate:
+ i = 0
+ while i < 200:
+ count = 0
+ nodes_in_pool = nodes_from_pool_list(mnode)
+ _rc = True
+ for server in servers:
+ if server in nodes_in_pool:
+ g.log.error("Peer '%s' still in pool", server)
+ _rc = False
+ count += 1
+ if not count:
+ break
+
+ if not _rc:
+ g.log.error("Validation after peer detach failed.")
+ g.log.info("Validation after peer detach is successful")
+ return _rc
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/snap_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/snap_ops.py
new file mode 100644
index 0000000..8e94e1d
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/snap_ops.py
@@ -0,0 +1,307 @@
+# Copyright (C) 2018-2019 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+"""
+ Description: Library for gluster snapshot operations.
+"""
+
+import json
+import httplib
+from glusto.core import Glusto as g
+from glustolibs.gluster.rest import RestClient
+from glustolibs.gluster.volume_ops import volume_start, volume_stop
+
+
+def snap_create(mnode, volname, snapname, timestamp=False, description=None):
+ """Creates snapshot for the given volume.
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ snapname (str): snapshot name
+
+ Kwargs:
+ timestamp (bool): If this option is set to True, then
+ timestamps will get appended to the snapname. If this option
+ is set to False, then timestamps will not be appended to snapname.
+ description (str): description for snapshot creation
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+
+ Example:
+ snap_create("abc.com", testvol, testsnap)
+
+ """
+ data = {"snapname": snapname, "volname": volname,
+ "description": description, "timestamp": timestamp}
+ return RestClient(mnode).handle_request("POST", "/v1/snapshots", httplib.CREATED, data)
+
+
+def snap_activate(mnode, snapname):
+ """Activates the given snapshot
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ snapname (str): snapshot name to be activated
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+
+ Example:
+ snap_activate("abc.com", testsnap)
+
+ """
+ return RestClient(mnode).handle_request('POST', "/v1/snapshots/%s/activate"
+ % snapname, httplib.OK, None)
+
+
+def snap_deactivate(mnode, snapname):
+ """Deactivates the given snapshot
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ snapname (str): snapshot name to be deactivated
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+
+ Example:
+ snap_deactivate("abc.com", testsnap)
+
+ """
+ return RestClient(mnode).handle_request('POST',
+ "/v1/snapshots/%s/deactivate"
+ % snapname, httplib.OK, None)
+
+
+def snap_clone(mnode, snapname, clonename):
+ """Clones the given snapshot
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ snapname (str): snapshot name to be cloned
+ clonename (str): clone name
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+
+ Example:
+ snap_clone("abc.com", testsnap, clone1)
+
+ """
+ data = {"clonename": clonename}
+ return RestClient(mnode).handle_request('POST', "/v1/snapshots/%s/clone"
+ % snapname, httplib.CREATED, data)
+
+
+def snap_restore(mnode, snapname):
+ """Snap restore for the given snapshot
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ snapname (str): snapshot name to be cloned
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+
+ Example:
+ snap_restore(mnode, testsnap)
+
+ """
+ return RestClient(mnode).handle_request('POST', "/v1/snapshots/%s/restore"
+ % snapname, httplib.CREATED, None)
+
+
+def snap_restore_complete(mnode, volname, snapname):
+ """stops the volume, restore the snapshot and starts the volume
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ snapname (str): snapshot name
+
+ Returns:
+ bool: True on success, False on failure
+
+ Example:
+ snap_restore_complete(mnode, testvol, testsnap)
+
+ """
+
+ # Stopping volume before snap restore
+ ret, _, _ = volume_stop(mnode, volname)
+ if not ret:
+ g.log.error("Failed to stop %s volume before restoring snapshot %s
+ in node %s", volname, snapname, mnode)
+ return False
+
+ ret, _, _ = snap_restore(mnode, snapname)
+ if ret:
+ g.log.error("Snapshot %s restore failed on node %s", snapname, mnode)
+ return False
+
+ # Starting volume after snap restore
+ ret, _, _ = volume_start(mnode, volname)
+ if not ret:
+ g.log.error("Failed to start volume %s after restoring snapshot %s
+ in node %s" , volname, snapname, mnode)
+ return False
+ return True
+
+
+def snap_info(mnode, snapname):
+ """Gets the snap info by snapname
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ snapname (str): snapshot name
+
+ Returns:
+ NoneType: None if command execution fails, parse errors.
+ dict: on success.
+ """
+ return RestClient(mnode).handle_request('GET', "/v1/snapshots/%s"
+ % snapname, httplib.OK, None)
+
+
+def snap_list(mnode):
+ """Lists the snapshots
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+ return RestClient(mnode).handle_request('GET', "/v1/snapshots", httplib.OK, None)
+
+
+def get_snap_list(mnode):
+ """ Lists the snapname
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+
+ Returns:
+ list: List containing the snapname if exists else returns None
+
+ """
+ _, out, _ = snap_list(mnode)
+ if out:
+ output = json.loads(out)
+ snap_info = output[0]
+ snaps_list = []
+ for elem in snap_info['snaps']:
+ snaps = elem['snapinfo']['name']
+ snaps_list.append(snaps)
+ return snaps_list
+ return None
+
+
+def snap_status(mnode, snapname):
+ """Get the snap status by snapname
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ snapname (str): snapshot name
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+
+ """
+ return RestClient(mnode).handle_request('GET', "/v1/snapshots/%s/status"
+ % snapname, httplib.OK, None)
+
+
+def snap_delete(mnode, snapname):
+ """Deletes the given snapshot
+
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ snapname (str): snapshot name to be deleted
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+ return RestClient(mnode).handle_request('DELETE', "/v1/snapshots/%s"
+ % snapname, httplib.DELETE, None)
+ # TODO: Few snapshot functions are yet to be automated after it is
+ # implemented in gd2
+
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py
new file mode 100644
index 0000000..e65043b
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py
@@ -0,0 +1,519 @@
+# Copyright (C) 2019 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import json
+import httplib
+from glusto.core import Glusto as g
+from glustolibs.gluster.rest import RestClient
+from glustolibs.gluster.lib_utils import validate_uuid
+from glustolibs.gluster.exceptions import GlusterApiInvalidInputs
+
+
+"""This module contains the python glusterd2 volume api's implementation."""
+
+
+def validate_brick(bricks_list):
+ """Validate brick pattern.
+ Args:
+ bricks(list): in the form of ["nodeid:brickpath"]
+ Returns:
+ brick_req(list): list of bricks
+ """
+ brick_req = []
+ result = True
+ if bricks_list:
+ for brick in bricks_list:
+ brk = brick.split(":")
+ if len(brk) != 2 or not validate_uuid(brk[0]):
+ result = None
+ break
+ req = {}
+ req['peerid'] = brk[0]
+ req['path'] = brk[1]
+ brick_req.append(req)
+ else:
+ result = None
+
+ if result:
+ return brick_req
+ else:
+ return result
+
+
+def volume_create(mnode, volname, bricks_list, force=False, replica_count=0,
+ arbiter_count=0, transport_type="tcp",
+ options=None, metadata=None):
+ """Create the gluster volume with specified configuration
+ Args:
+ mnode(str): server on which command has to be executed
+ volname(str): volume name that has to be created
+ bricks_list (list): List of bricks to use for creating volume.
+ Example:
+ from glustolibs.gluster.lib_utils import form_bricks_list
+ bricks_list = form_bricks_list(mnode, volname, num_of_bricks,
+ servers, servers_info)
+ Kwargs:
+ force (bool): If this option is set to True, then create volume
+ will get executed with force option. If it is set to False,
+ then create volume will get executed without force option
+ replica_count (int): if volume is replicated type
+ arbiter_count (int):if volume is arbiter type
+ transport_type : tcp, rdma
+ options (dict): volume options
+ metadata (dict): volume metadata
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ (-1, '', ''): If not enough bricks are available to create volume.
+ (ret, out, err): As returned by volume create command execution.
+ Example:
+ volume_create(mnode, volname, bricks_list)
+ """
+
+ if len(bricks_list) <= 0:
+ raise GlusterApiInvalidInputs("Bricks cannot be empty")
+
+ req_bricks = validate_brick(bricks_list)
+ if not req_bricks:
+ raise GlusterApiInvalidInputs("Invalid Brick details, bricks "
+ "should be in form of "
+ "<peerid>:<path>")
+
+ if transport_type not in ("tcp", "rdma", "tcp,rdma"):
+ raise GlusterApiInvalidInputs("Transport type %s not "
+ "supported" % transport_type)
+
+ if not options:
+ options = {}
+
+ if not metadata:
+ metadata = {}
+
+ num_bricks = len(bricks_list)
+ sub_volume = []
+
+ if replica_count > 0:
+ replica = arbiter_count + replica_count
+
+ if num_bricks % replica != 0:
+ raise GlusterApiInvalidInputs(
+ "Invalid number of bricks specified")
+
+ num_subvol = num_bricks / replica
+ for i in range(0, num_subvol):
+ idx = i * replica
+ ida = i * replica + 2
+ # If Arbiter is set, set it as Brick Type for 3rd th brick
+ if arbiter_count > 0:
+ req_bricks[ida]['type'] = 'arbiter'
+ subvol_req = {}
+ subvol_req['type'] = 'replicate'
+ subvol_req['bricks'] = req_bricks[idx:idx + replica]
+ subvol_req['replica'] = replica_count
+ subvol_req['arbiter'] = arbiter_count
+ sub_volume.append(subvol_req)
+ else:
+ subvol_req = {}
+ subvol_req['type'] = 'distrubute'
+ subvol_req['bricks'] = req_bricks
+ sub_volume.append(subvol_req)
+
+ # To create a brick dir
+ create_brick_dir = {"create-brick-dir": True}
+
+ data = {
+ "name": volname,
+ "subvols": sub_volume,
+ "transport": transport_type,
+ "options": options,
+ "force": force,
+ "metadata": metadata,
+ "Flags": create_brick_dir
+ }
+
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes", httplib.CREATED, data)
+
+
+def volume_start(mnode, volname, force=False):
+ """Starts the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ force (bool): If this option is set to True, then start volume
+ will get executed with force option. If it is set to False,
+ then start volume will get executed without force option
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_start("w.x.y.z", "testvol")
+ """
+ data = {
+ "force-start-bricks": force
+ }
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/start" % volname,
+ httplib.OK, data)
+
+
+def volume_stop(mnode, volname, force=False):
+ """Stops the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ force (bool): If this option is set to True, then stop volume
+ will get executed with force option. If it is set to False,
+ then stop volume will get executed without force option
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_stop(w.x.y.z, "testvol")
+ """
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/stop" % volname,
+ httplib.OK, None)
+
+
+def volume_delete(mnode, volname, xfail=False):
+ """Deletes the gluster volume if given volume exists in
+ gluster and deletes the directories in the bricks
+ associated with the given volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ xfail (bool): expect to fail (non existent volume, etc.)
+ Returns:
+ bool: True, if volume is deleted
+ False, otherwise
+ Example:
+ volume_delete("w.x.y.z", "testvol")
+ """
+ hosts = []
+ paths = []
+ volinfo = get_volume_info(mnode, volname, xfail)
+ if not volinfo:
+ if xfail:
+ g.log.info(
+ "Volume {} does not exist in {}"
+ .format(volname, mnode)
+ )
+ return True
+ else:
+ g.log.error(
+ "Unexpected: volume {} does not exist in {}"
+ .format(volname, mnode))
+ return False
+
+ _, _, err = RestClient(mnode).handle_request(
+ "DELETE", "/v1/volumes/%s" % volname,
+ httplib.NO_CONTENT, None)
+ if err:
+ if xfail:
+ g.log.info("Volume delete is expected to fail")
+ return True
+
+ g.log.error("Volume delete failed")
+ return False
+
+ # remove all brick directories
+ for j in volinfo['subvols']:
+ for i in j['bricks']:
+ g.run(i['host'], "rm -rf %s" % i['path'])
+
+ return True
+
+
+def volume_reset(mnode, volname, force=False,
+ options=None, all_volumes=False):
+ """Resets the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ force (bool): If this option is set to True, then reset volume
+ will get executed with force option. If it is set to False,
+ then reset volume will get executed without force option.
+ options (dict): volume options
+ all_volumes (bool)
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_reset("w.x.y.z", "testvol")`
+ """
+ if not 'options':
+ options = {}
+ data = {
+ "options": options,
+ "force": force,
+ "all": all_volumes,
+ }
+ return RestClient(mnode).handle_request(
+ "DELETE", "/v1/volumes/%s/options" % volname,
+ httplib.OK, data)
+
+
+def volume_info(mnode, volname):
+ """Get gluster volume info
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ Kwargs:
+ volname (str): volume name.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_info("w.x.y.z")
+ """
+ return RestClient(mnode).handle_request("GET",
+ "/v1/volumes/%s" % volname,
+ httplib.OK, None)
+
+
+def get_volume_info(mnode, volname, xfail=False):
+ """Fetches the volume information as displayed in the volume info.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name.
+ Kwargs:
+ xfail (bool): Expect failure to get volume info
+ Returns:
+ NoneType: If there are errors
+ dict: volume info in dict of dicts
+ Example:
+ get_volume_info("abc.com", volname="testvol")
+ """
+ ret, vol_info, err = volume_info(mnode, volname)
+ if ret:
+ if xfail:
+ g.log.error(
+ "Unexpected: volume info {} returned err ({} : {})"
+ .format(volname, vol_info, err)
+ )
+ return None
+ vol_info = json.loads(vol_info)
+ g.log.info("Volume info: %s", vol_info)
+ return vol_info
+
+
+def volume_status(mnode, volname):
+ """Get gluster volume status
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_status("w.x.y.z", "testvol")
+ """
+ return RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/status" % volname,
+ httplib.OK, None)
+
+
+def get_volume_status(mnode, volname, service=''):
+ """This module gets the status of all or specified volume(s)/brick
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name.
+ Kwargs:
+ service (str): name of the service to get status
+ can be bricks
+ Returns:
+ dict: volume status in dict of dictionary format, on success
+ NoneType: on failure
+ Example:
+ get_volume_status("10.70.47.89", volname="testvol")
+ """
+ if service:
+ _, status, err = volume_brick_status(mnode, volname)
+ else:
+ _, status, err = volume_status(mnode, volname)
+ if not err:
+ status = json.loads(status)
+ return status
+ return None
+
+
+def volume_brick_status(mnode, volname):
+ """Get gluster volume brick status
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_status("w.x.y.z","testvol")
+ """
+ return RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/bricks" % volname,
+ httplib.OK, None)
+
+
+def volume_list(mnode):
+ """List the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_list("w.x.y.z")
+ """
+ return RestClient(mnode).handle_request(
+ "GET", "/v1/volumes", httplib.OK, None)
+
+
+def get_volume_list(mnode, xfail=False):
+ """Fetches the volume names in the gluster.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ Kwargs:
+ xfail (bool): Expect failure to get volume info
+ Returns:
+ NoneType: If there are errors
+ list: List of volume names
+ Example:
+ get_volume_list("w.x.y.z")
+ """
+ vol_list = []
+ ret, volumelist, err = volume_list(mnode)
+ if ret:
+ if xfail:
+ g.log.error(
+ "Unexpected: volume list returned err ({} : {})"
+ .format(volumelist, err)
+ )
+ return None
+ volumelist = json.loads(volumelist)
+ for i in volumelist:
+ vol_list.append(i["name"])
+ g.log.info("Volume list: %s", vol_list)
+ return vol_list
+
+
+def get_volume_options(mnode, volname, option=None):
+ """Gets the option values for the given volume.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ option (str): volume option to get status.
+ If not given, the function returns all the options for
+ the given volume
+ Returns:
+ dict: value for the given volume option in dict format, on success
+ NoneType: on failure
+ Example:
+ get_volume_options(mnode, "testvol")
+ """
+ if not option:
+ _, get_vol_options, err = RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/options" % volname, httplib.OK, None)
+ else:
+ _, get_vol_options, err = RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/options/%s" % (volname, option),
+ httplib.OK, None)
+ if not err:
+ get_vol_options = json.loads(get_vol_options)
+ return get_vol_options
+ return None
+
+
+def set_volume_options(mnode, volname, options,
+ advance=True, experimental=False,
+ deprecated=False):
+ """Sets the option values for the given volume.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ options (dict): volume options in key
+ value format
+ Kwargs:
+ advance (bool): advance flag to set options. Default set True
+ experimental (bool): experimental flag to set options.
+ Default set False.
+ deprecated (bool): deprecated flag to set options.
+ Default set False
+ Returns:
+ bool: True, if the volume option is set
+ False, on failure
+ Example:
+ set_volume_option("w.x.y.z", "testvol", options)
+ """
+ if not options:
+ raise GlusterApiInvalidInputs("cannot set empty options")
+
+ vol_options = {}
+ req = {}
+ for key in options:
+ vol_options[key] = options[key]
+ req['options'] = vol_options
+ req['allow-advanced-options'] = advance
+ req['allow-experimental-options'] = experimental
+ req['allow-deprecated-options'] = deprecated
+ _, _, err = RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/options" % volname,
+ httplib.CREATED, req)
+ if err:
+ return True
+ return False