glusterfs.git/geo-replication/src, branch v4.1.2

core/build/various: python3 compat, prepare for python2 -> python3

2018-04-12T11:04:27+00:00

Note 1) we're not supposed to be using #!/usr/bin/env python, see
https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines

Note 2) we're also not supposed to be using "!/usr/bin/python,
see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out

The previous patch (https://review.gluster.org/19767) tried to do too
much in one patch, so it was abandoned.

This patch does two things:
1) minor cleanup of configure(.ac) to explicitly use python2
2) change all the shebang lines to #!/usr/bin/python2 and add them
where they were missing based on warnings emitted during rpmbuild.

In a follow-up patch python2 will eventually be changed to python3.

Before that python2-isms (e.g. print, string.join(), etc.) need to be
converted to python3. Some of those can be rewritten in version agnostic
python. E.g. print statements become print() with "from __future_ import
print_function". The python 2to3 utility will be used for some of those.
Also Aravinda has given guidance in the comments to the first patch for
changes.

updates: #411
Change-Id: I471730962b2526022115a1fc33629fb078b74338
Signed-off-by: Kaleb S. KEITHLEY

geo-rep: Remove unused working directory check in gsyncd

2018-01-29T07:33:00+00:00

To append the default config file path, gsyncd calls gluster
command to get the workdir path and constructs config file path.
This is not required now since the Config management in Geo-replication
is changed with patch 18257(Issue #73)

BUG: 1539545
Change-Id: Ia7eb39e36ed59ece4de65ea7ec71a0f615e338bb
Signed-off-by: Aravinda VK

geo-rep: Improve geo-rep pre-validation logs

2018-01-22T06:33:48+00:00

Geo-rep runs gverify.sh which does pre-validation.
As part of it, master and slave volume is mounted
to verify the size. If for some reason, the mount
fails, the error message does not point out the
mount log file location. Also both master and
slave mount logs are same.

Patch does following improvements.

1. Master and slave mount logs are separated and
   error message points the log file to be looked for.
2. The log location is changed to /var/log/glusterfs/geo-replication
   instead of /var/log/glusterfs/geo-replication-slaves
3. The log file name is changed to "gverify-mastermnt.log" and
   "gverify-slavemnt.log" for master and slave mount respectively

Fixes: #395
Change-Id: Ia644ec0afebbdaae92e01adf03c635e5f8866a02
Signed-off-by: Kotresh HR

geo-rep: Validate availability of gluster binary on slave

2018-01-19T03:53:24+00:00

1. Adds validation to check if gluster binary is available on slave
2. Add a simple geo-rep setup test case to verify whether setup is fine.
   It's named in such a way that it runs first.

BUG: 1532591
Change-Id: Ie777e55ae13db8fa97d4e32464ad82269ee5fd07
Signed-off-by: Kotresh HR

tests: Enable geo-rep test cases

2018-01-05T07:08:10+00:00

This patch re-enables the geo-rep test cases.
Along with it does following optimizations.

1. Use EXPECT_WITHIN instead of sleep
2. Clean up geo-rep ssh key after test
3. Changes to gverify.sh and S56glusterd-geo-rep-create-post.sh
   to use the given ssh identity file for geo-rep create
4. Make gluster-command-dir configurable and introduce
   slave-gluster-command-dir which points the parent directory
   of gluster binaries in master and slave respectively.

Change-Id: Ia7696278d9dd3ba04224dcd7c3564088ca970b04
BUG: 1480491
Signed-off-by: Kotresh HR

build/packaging: Debian and Ubuntu don't have /usr/libexec

2017-03-13T14:18:07+00:00

GLUSTERFS_LIBEXECDIR is effectively hard-coded to /usr/libexec/glusterfs
in configure(.ac)

Debian-based distributions don't have a /usr/libexec/ directory

This issues is partially mitigated by the use of $libexecdir in
some of the Makefile.am files, but even so the incorrectly defined
GLUSTERFS_LIBEXECDIR results in various things such as gsyncd,
glusterfind, eventsd, etc., trying to invoke other scripts and
programs from a location that doesn't exist.

And once we correctly define GLUSTERFS_LIBEXECDIR, then we might as
well use it appropriatedly.

Change-Id: If5219cadc51ae316f7ba2e2831d739235c77902d
BUG: 1430841
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16880
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Milind Changire 
Reviewed-by: Joe Julian 
Reviewed-by: Shyamsundar Ranganathan

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

geo-rep: Alternate command to generate SSH Keys

2016-08-31T10:09:12+00:00

`gluster system:: execute gsec_create` is used to generate SSH
Keys in all Master nodes and collect public keys in command initiated
node. But this tool will not provide details if a peer node is down and
unable to generate keys.

New command will be introduced to create SSH Keys in all peer nodes.

Usage:

    gluster-georep-sshkey generate
    or
    gluster-georep-sshkey generate --no-prefix

Generates two SSH keys(one for gsyncd access and other for tar) in all
peer nodes and collects the public keys to the local node where it is
initiated. Adds `command=` prefix to common_secret.pem.pub if `--no-prefix`
argument is not set.

Shows status as below,

+-----------+-------------+---------------+
|    NODE   | NODE STATUS | KEYGEN STATUS |
+-----------+-------------+---------------+
|    fvm2   |          UP |            OK |
| localhost |          UP |            OK |
+-----------+-------------+---------------+

BUG: 1356508
Change-Id: Ib202811f41f9986694f07d9eedba31db6ed4d18f
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14732
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: Simplify Non root user(mountbroker) Setup

2016-08-31T05:22:55+00:00

Setting up Non root Geo-replication is troublesome. Lot of steps
involved like user setup, directory creation, setting up proper
permissions, editing glusterd.vol file etc.

Introducing `gluster-mountbroker` command, with this tool non root
user setup steps are(Run the following commands in any one Slave
node),

    gluster-mountbroker setup  

For example,

    gluster-mountbroker setup /var/mountbroker-root geogroup

Add user using,

    gluster-mountbroker add  

For example,

    gluster-mountbroker add slavevol geoaccount

Remove user or Volume using,

    gluster-mountbroker remove [--volume ] [--user ]

Example,

    gluster-mountbroker remove --volume slavevol --user geoaccount
    gluster-mountbroker remove --user geoaccount
    gluster-mountbroker remove --volume slavevol

Check the status of setup using,

    gluster-mountbroker status

Once the changes are complete, restart glusterd in all Slave nodes.
and follow the Geo-rep instructions.

Change-Id: Ied3fa009df6a886b24ddd86d39283fcfcff68c82
BUG: 1343333
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/14544
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
Reviewed-by: Saravanakumar Arumugam 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

geo-rep: address potential leak of memory

2016-07-19T11:25:35+00:00

fix memory leak reported by clang analyzer:
gsyncd.c:411:17: warning: Potential leak of 
                 memory pointed to by 'argv'
                 in main()


Change-Id: I179fb1c095cf4d331b5910df6388dc724cb4ac96
BUG: 1332419
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/14151
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR