summaryrefslogtreecommitdiffstats
path: root/extras
Commit message (Collapse)AuthorAgeFilesLines
* snap_scheduler: python3 compatibility and new test caseSunny Kumar2020-04-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: "snap_scheduler.py init" command failing with the below traceback: [root@dhcp43-104 ~]# snap_scheduler.py init Traceback (most recent call last): File "/usr/sbin/snap_scheduler.py", line 941, in <module> sys.exit(main(sys.argv[1:])) File "/usr/sbin/snap_scheduler.py", line 851, in main initLogger() File "/usr/sbin/snap_scheduler.py", line 153, in initLogger logfile = os.path.join(process.stdout.read()[:-1], SCRIPT_NAME + ".log") File "/usr/lib64/python3.6/posixpath.py", line 94, in join genericpath._check_arg_types('join', a, *p) File "/usr/lib64/python3.6/genericpath.py", line 151, in _check_arg_types raise TypeError("Can't mix strings and bytes in path components") from None TypeError: Can't mix strings and bytes in path components Solution: Added the 'universal_newlines' flag to Popen to support backward compatibility. Added a basic test for snapshot scheduler. Backport of: >Upstream Patch: https://review.gluster.org/#/c/glusterfs/+/24257/ >Change-Id: I78e8fabd866fd96638747ecd21d292f5ca074a4e >Fixes: #1134 >Signed-off-by: Sunny Kumar <sunkumar@redhat.com> >(cherry picked from commit a7d7ec066e56ac03bf252c26beb20fdc2c3b6772) Change-Id: I78e8fabd866fd96638747ecd21d292f5ca074a4e Fixes: #1134 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* extras: enable log rotation for USS logsSunny Kumar2020-02-111-0/+21
| | | | | | | | | | | | | | Added logrotate support for user serviceable snapshot's logs. Backport of: >Change-Id: Ic920eaa8ab5e44daf5937a027c6913d7bb26d517 >Fixes: bz#1786722 >Signed-off-by: Sunny Kumar <sunkumar@redhat.com> Change-Id: Ic920eaa8ab5e44daf5937a027c6913d7bb26d517 Fixes: bz#1786754 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* extras: Cgroup(CPU/Mem) restriction are not working on gluster processMohit Agrawal2019-11-192-2/+2
| | | | | | | | | | | | | | | | | | | | Problem: After Configure the Cgroup(CPU/MEM) limit to a gluster processes resource(CPU/MEM) limits are not applicable to the gluster processes.Cgroup limits are not applicable because all threads are not moved into a newly created cgroup to apply restriction. Solution: To move a gluster thread to newly created cgroup change the condition in script > Change-Id: I8ad81c69200e4ec43a74f6052481551cf835354c > Fixes: bz#1764208 > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit f5811979935ce607391825ac6913a95f588818e3) > (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/23599/) Change-Id: I8ad81c69200e4ec43a74f6052481551cf835354c Fixes: bz#1766425 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* extras/hooks: Install and package newly added post add-brick hook scriptAnoop C S2019-07-161-2/+2
| | | | | | | | | | | | | | | | | | https://review.gluster.org/c/glusterfs/+/22913 added a new SELinux hook script as a post add-brick operation to label new brick paths. But the change failed to install and package new script. Therefore making necessary changes to Makefile and spec file to get it installed and packaged. Backport of: > Change-Id: I67b8f4982c2783c34a4bc749fb4387c19a038225 > fixes: bz#1717953 > Signed-off-by: Anoop C S <anoopcs@redhat.com> Change-Id: I67b8f4982c2783c34a4bc749fb4387c19a038225 fixes: bz#1718227 Signed-off-by: Anoop C S <anoopcs@redhat.com> (cherry picked from commit 647a1b83fa39a635b01fcba38a330485fcd6afdb)
* extras/hooks: Add SELinux label on new bricks during add-brickAnoop C S2019-07-041-0/+100
| | | | | | | | | | | | Backport of: > Change-Id: Ifd8ae5eeb91b968cc1a9a9b5d15844c5233d56db > fixes: bz#1717953 > Signed-off-by: Anoop C S <anoopcs@redhat.com> Change-Id: Ifd8ae5eeb91b968cc1a9a9b5d15844c5233d56db fixes: bz#1718227 Signed-off-by: Anoop C S <anoopcs@redhat.com> (cherry picked from commit 582f91fcad42b6b7da653d2a587d196a7a1e0204)
* extras/hooks: syntactical errors in SELinux hooks, scipt logic improvedMilan Zink2019-05-082-30/+43
| | | | | | | Fixes: bz#1701818 Change-Id: Ia5fa1df81bbaec3a84653d136a331c76b457f42c Signed-off-by: Milan Zink <zeten30@gmail.com> (cherry picked from commit 1ad201a9fd6748d7ef49fb073fcfe8c6858d557d)
* rpm: add thin-arbiter packagev6.0rc1Ashish Pandey2019-03-135-60/+181
| | | | | | | | | | | | | | | | | | | | Discussion on thin arbiter volume - https://github.com/gluster/glusterfs/issues/352#issuecomment-350981148 Main idea of having this rpm package is to deploy thin-arbiter without glusterd and other commands on a node, and all we need on that tie-breaker node is to run a single glusterfs command. Also note that, no other glusterfs installation needs thin-arbiter.so. Make sure RPM contains sample vol file, which can work by default, and a script to configure that volfile, along with translator image. Change-Id: Ibace758373d8a991b6a19b2ecc60c93b2f8fc489 updates: bz#1672818 Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Ashish Pandey <aspandey@redhat.com> (cherry picked from commit ca9bef7f1538beb570fcb190ff94f86f0b8ba38a)
* build: include gluster/__init__.* files for python packagingNiels de Vos2019-02-253-1/+10
| | | | | | | | | | | | | | | | | | | | | The gluster/__init__.py file was originaly part of the glupy installation. This file is required to have the python-gluster package function correctly, it is expected to provide the 'gluster' namespace for other python packages (like from the libgfapi-python project). Because glupy does not exist anymore, this file is now added to in a new extras/python directory. Backport of commit b235a56394c09bb1d7308a620053dbe098d15967: > Change-Id: I14fe959778ee3344d7d54ba342c7928a4d8080a2 > Fixes: c3fcff9ccbfcec1be242fd5cf210c9995586b078 > Fixes: 8293d21280fd6ddfc9bb54068cf87794fc6be207 > Updates: bz#1642810 > Signed-off-by: Niels de Vos <ndevos@redhat.com> Change-Id: I04e1e0dc4022b53c02053d5452f305b78e56c389 Updates: bz#1680585 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* extras/systemd: Remove socket activated syslog.target and add documentation keysVijay Bellur2019-01-112-1/+4
| | | | | | | | | | Adopting patch provided by Patrick Matthäi <pmatthaei@debian.org> at: https://bugzilla.redhat.com/attachment.cgi?id=1470751 fixes: bz#1560561 Change-Id: I363a049c743c65823e6c4f153cbe20b47fcb64fb Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* extras: Add readdir-ahead to samba group commandAnoop C S2019-01-041-0/+1
| | | | | | Change-Id: I310f014a9ab8416d9833f5711ff902da1457f415 updates: bz#1193929 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* glusterd: define max-port to 60999Atin Mukherjee2018-12-181-1/+1
| | | | | | | | | | | | | | | | | | As glusterd scans through all the ports in its defined range, with RHEL 7.3 onwards any port beyond 60999 isn't within the ephemeral port range and following AVC denial message is seen. type=AVC msg=audit(1471946614.154:109): avc: denied { name_bind } for pid=2302 comm="glusterd" src=61000 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:object_r:ephemeral_port_t:s0 tclass=tcp_socket Fix is to define the max port range to 60999 in glusterd.vol file. The port range can be tweaked through a reconfigure of this configuration file though. Fixes: bz#1659857 Change-Id: I60fd4a421d8509b8dca4ca13b73999ae33965f72 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* geo-rep : fix slave volume read-only optionSunny Kumar2018-12-171-1/+1
| | | | | | | | | | | | Problem: When separate ssh key is given for non root user setting slave volume read-only option results in failure. Solution: Check for extra param in case separate key is given for non-root user and take action accordingly. Change-Id: Iafe9a2aa6b86cde1dcd7d63771048a6ae33c2cde fixes: bz#1659971 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* extras: New group volume set command for Samba integrationAnoop C S2018-12-142-2/+14
| | | | | | | | | | | | | | | | | | | # gluster volume set <VOLNAME> group samba List of volume set options from group-samba are aimed at improving the below workloads which consumes time and network hops in SMB world: * Listing of large directories * Negative lookups during creation of files Caching the necessary metadata required for these workloads saves us time and network hops. On the other side we have to ensure correctness(avoiding stale cache) in caching(via md-cache) with the help of cache invalidation in an environment where multiple client access is expected. Change-Id: Icdd2d8e5eb290e12bc509105418c668f432f4eae fixes: bz#1656771 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* geo-rep: Make slave volume read-only (by default)Harpreet Kaur2018-12-071-0/+2
| | | | | | | | | | | | | | | Added a command to set "features.read-only" option to a default value "on" for slave volume. Changes are made in: $SRC//extras/hook-scripts/S56glusterd-geo-rep-create-post.sh for root geo-rep and $SRC/geo-replication/src/set_geo_rep_pem_keys.sh for non-root geo-rep. Fixes: bz#1654187 Change-Id: I15beeae3506f3f6b1dcba0a5c50b6344fd468c7c Signed-off-by: Harpreet Kaur <hlalwani@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-052-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* extras: Add group-distributed-virt for single-brick ovirt-gluster use-caseKrutika Dhananjay2018-12-052-2/+14
| | | | | | Change-Id: I930011327332b7ba30cc76f614efaf5932eb4f3d fixes: bz#1654138 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* Do not blindly add volume share section to smb.confMilan Zink2018-12-042-5/+26
| | | | | | | | | | | | | With this change, by default GlusterFS volume share section will no longer be added to smb.conf for client access unless user.cifs or user.smb volume set options are enabled. This also fixes the hook script to check for presence of all configuration possibilities for those volume set options like 'enable' or 'on'. Change-Id: Ibecf7fffb4507d7255d963c3b1482afb0d0db984 Signed-off-by: Milan Zink <mzink@redhat.com> Signed-off-by: Anoop C S <anoopcs@redhat.com> Fixes: bz#1575836
* georep: python2 to python3 compat - schedulerKotresh HR2018-10-301-1/+1
| | | | | | | | | 1. scheduler - Popen 2. syncdutils - corner case on failure fixes: bz#1643932 Change-Id: I65af97a244a8790e976acedc2728db6ebbf2ae10 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* cliutils: python2 to python3 compatKotresh HR2018-10-301-1/+2
| | | | | | | | Make Popen py2 and py3 compatiable fixes: bz#1643935 Change-Id: Ife34cb38024dcdc0420436e7d76fd208223f9d86 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* cliutils : python2 to python3 compatSunny Kumar2018-10-091-13/+13
| | | | | | | | | | This patch fixes import issue in cliutils. Provided solution is to use relative import. Change-Id: I14c9a0b528ef52e7c91f6b17b569c68c2ced8912 updates: #411 Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* extras: Script to collect system-statsPranith Kumar K2018-10-091-0/+52
| | | | | | | | | | | | | For debugging performance issues we found that this script is very useful. Script collects the stats of vmstat/iostat/top/sar to get metrics about memory/disk/cpu/network respectively at the configured interval. It is very important to collect all these metrics at the same time so that they can be correlated easily, so made this into a script fixes bz#1633926 Change-Id: Iea52e55179b21c505953722c8898293fe92cb8de Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
* packaging: remove RHEL5-isms from glusterfs.spec(.in)Kaleb S. KEITHLEY2018-10-082-10/+3
| | | | | | | | | | | CentOS5 too It's been a couple of years since we stopped doing anything with RHEL/CentOS 5 Change-Id: I83e78fa84b93fa5baddaf3fab92736d692a9bbca updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* python: remove shebangs of non-main scripts and make others executableNiels de Vos2018-10-022-1/+0
| | | | | | | | | | | | | | | Some of the scripts that have a #!/usr/bin/python3 shebang do not have a main() like function. These scripts will not get executed but only imported. They do not need the shebang. A few others are not installed with 'make install', but do have a main() like function. These scripts are expected to be used by developers for different tasks (mostly code generation). Marking these scripts executable to make it easier to identify them. Change-Id: I73541471deb7e0830766b804786244e73dfe4221 Updates: #411 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* rpc: make binding to port 0 as the default if no option is providedAmar Tumballi2018-10-021-0/+2
| | | | | | | | | | | | | | | | | | | | | | | Right now, if no option is provided, the default port is assumed, which is 24007. Ideally, for 'glusterfsd' processes, it is better to not assume there are any ports given, so it can start listening on any port which is available. This helps us to cleanup the dependencies on glusterd from glusterfsd at the moment. No changes would be done to glusterd code, but making the right defaults helps to make glusterfsd more independent process later. NOTE: This patch is a reduced version of below set of patches: * https://review.gluster.org/14613/ & * https://review.gluster.org/14670/ & * https://review.gluster.org/14671/ Credits: Prasanna Kumar Kalever <pkalever@redhat.com> updates: bz#1343926 Change-Id: Ib874e10505e7366dc56ba754458252b67052e653 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Update database profile groupXavi Hernandez2018-09-241-1/+5
| | | | | | | | | | | | | | | Some performance testing has revealed that pgbench performs 3x better when these options are set: performance.client-io-threads=on performance.open-behind=on performance.read-after-open=yes server.event-threads=4 client.event-threads=4 Change-Id: I36ce389f893a8af13aac5f8285104d749b73d098 fixes: bz#1631886 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* python3: Fix python3 compatibilityNigel Babu2018-09-244-46/+49
| | | | | | Updates: #411 Change-Id: I7c1eaa92cd4ac05c3667b760e0db2cddcfbbaed8 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* extras: Add new options to group "virt"Krutika Dhananjay2018-09-211-0/+3
| | | | | | | | | | | | | In some of the recent performance tests on gluster-as-vm-image-store use-case, it has been observed that sometimes the lone fuse thread can hit near-100% CPU utilization and become a performance bottleneck. Enabling client-io-threads (in addition to bumping up epoll threads on server and client side) has shown to be helpful in getting around this bottleneck and pushing more IOPs. Change-Id: I231db309de0e37c79cd44f5666da4cd776fefa04 fixes: bz#1630798 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* extras/devel-tools: script to strace processesMilind Changire2018-09-191-0/+55
| | | | | | | | | | | | | | | | Script to strace processes at high priority. Script should be run at high priority, like nice value of -19, to help get a trace of a run-away process consuming high CPU. eg. $ nice -n -19 ./strace-brick.sh glusterfsd 50 will strace glusterfsd processes for 60 seconds which are consuming a minimum of 50% CPU. Change-Id: I6374054c7342b19122a3588d982388199f955930 BUG: 1427397 Signed-off-by: Milind Changire <mchangir@redhat.com>
* logrotate: utilize the new 'maxsize' optionAmar Tumballi2018-09-192-5/+33
| | | | | | | | | | | Since logrotate 3.8.x version, a new option 'maxsize' is supported, which helps in rotating the logs before the specified time if the size exceeds maxsize limit. This should help in reducing the overflow of gluster logs. Updates: bz#1193929 Change-Id: Id33c34d72f1931e7996893de9338db667e1e900d Signed-off-by: Amar Tumballi <amarts@redhat.com>
* extras/devel-tools: script to resolve backtracesMilind Changire2018-09-181-11/+16
| | | | | | | | | print-backtrace.sh script to resolve backtraces Change-Id: I24688ae5160cc706d9f4794521d17290bef6765b BUG: 1486532 Signed-off-by: Milind Changire <mchangir@redhat.com>
* extras/hooks: General improvements to S30samba-start.shAnoop C S2018-09-141-13/+13
| | | | | | | | | Based on https://review.gluster.org/c/glusterfs/+/19204 from Milan Zink <mzink@redhat.com> Change-Id: I61f62407a62475a74a6cc046c24748a31c66e6cd Updates: bz#1193929 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* Land part 2 of clang-format changesGluster Ant2018-09-129-2689/+2636
| | | | | Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* misc: fix misc. shebangsKaleb S. KEITHLEY2018-09-113-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * One #!/usr/bin/env python and three #!/usr/bin/python were overlooked in all the other python fixups. Ugh. * Two new python files missed the memo about #!/usr/bin/python3. * One #!/usr/bin/env bash. Various distribution packaging policies have strong wording about the use of #!/usr/bin/env ... Note: this patch does not change the use of #!/usr/bin/env bash in the two files extras/{clang-checker.sh,check_goto.pl} as these are not included in any packages. (Although I'm not actually sure why anyone would ever use '/usr/bin/env {sh,bash}' as I'm not aware of any version-specific differences like there are with, e.g., python.) * One #!/usr/bin/bash. On Fedora and CentOS > 6, /bin is a symlink to /usr/bin, so it makes little difference. But Debian & Ubuntu still have separate /bin and /usr/bin; and sh and bash are in /bin, not /usr/bin. (Historically, in BSD and SYSV Unix it was /bin/sh.) Note: Fedora and CentOS package build runs a script that converts all /bin/sh and /bin/bash to /usr/bin/sh and /usr/bin/bash. Change-Id: I9171265829af78dd0cd7622c22b56d22179ff8a3 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* classification: provide infra to start labelling features/componentsAmar Tumballi2018-09-041-1/+3
| | | | | | | | | | | | | `doc/xlator-classification.md` talks about the reasoning and expectations Reviewers are expected to check the 'category' of new option / translator added in the codebase, and make sure the flag is always properly set. It helps to keep the 'expectation' proper on the codebase. updates: #430 Change-Id: I2bfc9934a5f6eed77fcc3e20364046242decc82c Signed-off-by: Amar Tumballi <amarts@redhat.com>
* multiple files: calloc -> mallocYaniv Kaul2018-09-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xlators/cluster/stripe/src/stripe-helpers.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible xlators/cluster/dht/src/tier.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible xlators/cluster/dht/src/dht-layout.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible xlators/cluster/dht/src/dht-helper.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible xlators/cluster/dht/src/dht-common.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible xlators/cluster/afr/src/afr.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible xlators/cluster/afr/src/afr-inode-read.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible tests/bugs/replicate/bug-1250170-fsync.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible tests/basic/gfapi/gfapi-async-calls-test.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible tests/basic/ec/ec-fast-fgetxattr.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible rpc/xdr/src/glusterfs3.h: Move to GF_MALLOC() instead of GF_CALLOC() when possible rpc/rpc-transport/socket/src/socket.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible rpc/rpc-lib/src/rpc-clnt.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible extras/geo-rep/gsync-sync-gfid.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-xml-output.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-rpc-ops.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-cmd-volume.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-cmd-system.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-cmd-snapshot.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-cmd-peer.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible cli/src/cli-cmd-global.c: Move to GF_MALLOC() instead of GF_CALLOC() when possible It doesn't make sense to calloc (allocate and clear) memory when the code right away fills that memory with data. It may be optimized by the compiler, or have a microscopic performance improvement. In some cases, also changed allocation size to be sizeof some struct or type instead of a pointer - easier to read. In some cases, removed redundant strlen() calls by saving the result into a variable. 1. Only done for the straightforward cases. There's room for improvement. 2. Please review carefully, especially for string allocation, with the terminating NULL string. Only compile-tested! updates: bz#1193929 Original-Author: Yaniv Kaul <ykaul@redhat.com> Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Signed-off-by: Amar Tumballi <amarts@redhat.com> Change-Id: I16274dca4078a1d06ae09a0daf027d734b631ac2
* core: python3Kaleb S. KEITHLEY2018-09-0312-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* thin arbiter: set notify-contention option to yesAshish Pandey2018-08-311-0/+1
| | | | | | | | | | | | | | | | | | | | | | If this option is "ON' and if there is a lock contention for a file from 2 clients/self-heal daemon, an up call will be sent to all the clients holding that locks. Clients can handle this notification and may release the locks at the right time asap. If this option is ON, we get "instant" lock contention notification on client side. In next version of thin arbiter, we need to implement lock contention notification handling and two domain locking to synchronize update/access of xattrs of thin arbiter replica id file. This option is an efficient way to find out lock contention. In this case it is also helping to implement two domain locking for replica id file. Change-Id: I69883898d52feb425dfceb31cf12fa2157aaa314 fixes bz#1609207 Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* extras/geo-rep/gsync-sync-gfid.c: move from strlen() to sizeof()Yaniv Kaul2018-08-251-4/+5
| | | | | | | | | | For const strings, just do compile-time size calc instead of runtime. Compile-tested only! Change-Id: I89470016389a4ecb3bed35a1c9e684f5fae79997 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* Bash integration script should namespace variablesMark Mielke2018-08-151-20/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the original submitted script, it looks like there was effort put into namespacing all global variables. However a few mistakes remained. GLUSTER_TOP_SUBOPTIONSx were defined, but TOP_SUBOPTIONSx were referenced. This was likely an unrecognized defect in the original code submission? These are now corrected to refer to GLUSTER_TOP_SUBOPTIONSx. FINAL_LIST, LIST, and TOP were leaked into all Bash shells and used by the command completion functions. The most problematic of these was TOP, which was declared with "-i" making it an integer. This cause other code which used TOP to define a path to fail like this: $ bash $ TOP=/abc bash: /abc: syntax error: operand expected (error token is "/abc") These are now qualified as GLUSTER_FINAL_LIST, GLUSTER_LIST, and GLUSTER_TOP to reduce impact on scripts that might choose to use these extremely common variable names. Change-Id: Ic96eda8efd1f3238bbade6c6ddb69118e8d82158 Fixes: bz#1425325 Signed-off-by: Mark Mielke <mark.mielke@gmail.com>
* geo-rep/hook-script: Fix ssh/scp optionsKotresh HR2018-08-031-11/+12
| | | | | | | | | | | | | | Always use ssh and scp with "-oPasswordAuthentication=no" and "-oStrictHostKeyChecking=no" options. It might hang the post script otherwise leading geo-rep setup failure Also increased geo-rep timeout. Occasionally, it's taking more time to reach Active/Passive status. Especially, the first start after create. fixes: bz#1610405 Change-Id: I9560d64dbe0edf5db73446a9fc97dda19b88d233 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* All: run codespell on the code and fix issues.Yaniv Kaul2018-07-224-11/+11
| | | | | | | | | | | | Please review, it's not always just the comments that were fixed. I've had to revert of course all calls to creat() that were changed to create() ... Only compile-tested! Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* extras: Remove the prot_filter.py codeNigel Babu2018-07-191-144/+0
| | | | | | | | | | This code was meant to work with the protection xlator which was removed in Nov 2016. Unfortunately, this file remained and should be removed as well. Change-Id: Ia6daa06542ebc8c6fb935c352cc19924f3bb3d8a Fixes: bz#1598390 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* build/core: location of xattr.hKaleb S. KEITHLEY2018-07-185-5/+5
| | | | | | | | | | | | | | | | | | | starting with libattr-devel-2.4.48-1 in Fedora 28 <attr/xattr.h> has been removed from the package. On Fedora, RHEL/CentOS, and SUSE, the glibc-headers package has provided a near identical file, <sys/xattr.h>, in all the releases that we care about. On Debian, libc6-dev provides <sys/xattr.h> all the way back to 8/Jessie and presumably all Ubuntu derivatives since then. Note that on Debian the sys headers are installed in /usr/include/$DEB_HOST_GNU_TYPE/sys/... Change-Id: Id07c4b225bdaa6556bd54772604e75b8f346fb60 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* Add the tab space in the list of BROKEN_TESTS environment variabledkhandel2018-07-181-30/+30
| | | | | | | Fixes: bz#1598325 Change-Id: I4c14514d33288fd33cca903853c6d5e4f70bd441 Signed-off-by: Deepshikha Khandelwal <dkhandel@redhat.com>
* testing: Fix the build environmentNigel Babu2018-07-171-2/+1
| | | | | | | | | The build environment for FB is vastly different from us. This change will match the config to what we use for our regression testing Change-Id: Ib3584a02dbffb75892f2cbeebcbf51cc3aff3eb6 Fixes: bz#1601390 Signed-off-by: Nigel Babu <nigelb@redhat.com>
* geo-rep/scheduler: Fix EBUSY trace backKotresh HR2018-07-121-5/+20
| | | | | | | | | | Fix the trace back during temporary mount cleanup. Temporary mount is done to touch the root required for checkpoint to complete. fixes: bz#1598977 Change-Id: I97fea538e92c4ef0747747e981ef98499504e336 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-07-125-17/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, and https://review.gluster.org/#/c/20441/ Fixes some overlooked string.join()s: + AFAICT extras/profiler/glusterfs-profiler, extras/prot_filter.py, extras/rebalance.py, and extras/volfilter.py would only manifest at runtime. + xlators/experimental/fdl/src/gen_recon.py is a build-time error when using python3 during the build, thus was not noticed previously when building with python2. + extras/create_new_xlator/generate_xlator.py seems to be example code and does not affect the build or runtime AFAICT Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: Ia1fe2958d136f4303e30f7e7e86b6fe7d7b52c81 updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* Update mount-shared-storage.sh to automatically include all enabled ↵Josiah2018-07-101-27/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs mounts in fstab Currently the mount-shared-storage.sh script requires manual entry of each volume and mount point. There is a TODO item from Bug 1452527 to automatically pull the list of glusterfs mounts from the /etc/fstab file, validate that they are mounted, or attempt to mount them. This is to complete that TODO item. The extras/mount-shared-storage.sh file has been updated to read only active glusterfs mounts from the /etc/fstab file, verify if they are mounted, and attemt to mount them if they are not. It will exit with a status 0 if successfully mounted or status 1 if any of the mounts fail. This method will allow the standard method of mounts in /etc/fstab to be utilized and will ensure they are monted following system reboots. This can be tested by creating a volume, mounting it using the /etc/fstab method, and rebooting the server. Output will be similar to this: Jun 29 19:09:38 localhost systemd[1]: Starting Mount glusterfs sharedstorage... Jun 29 19:09:43 localhost mount-shared-storage.sh[4096]: /mnt/mount1 has been mounted Jun 29 19:09:48 localhost mount-shared-storage.sh[4096]: /mnt/mount2 has been mounted Jun 29 19:09:48 localhost systemd[1]: Started Mount glusterfs sharedstorage. Multiple fixes based on code review suggestions and spellcheck Removed spaces in grep for glusterfs as tabs are valid as well Added tab as an IFS field separator for the lines read from /etc/fstab Increased sleep time to 10 seconds before mount validation Changed mount check to use “mountpoint -q” Change-Id: Ib90c99ced8f7aefc8dbe87340afc73a28f6195e7 fixes: bz#1596789 Signed-off-by: Josiah <josiahp@gmail.com>
* core/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-07-096-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, and https://review.gluster.org/#/c/20364/ Fixes glupy.py python2isms, iteritems -> items, and some overlooked print() in georep/peer_mountbroker.in Note: Fedora packaging guidelines and SUSE rpmlint require explicit shebangs; popular practices like #!/usr/bin/env python and #!/usr/bin/python are not allowed; they must be #!/usr/bin/python2 or #!/usr/bin/python3 Note: Selected small fixes from 2to3 utility. Specifically apply, basestring, funcattrs, has_key, idioms, map, numliterals, raise, set_literal, types, urllib, and zip have already been applied. Also version agnostic imports for urllib, cpickle, socketserver, _thread, queue, etc., suggested by Aravinda in https://review.gluster.org/#/c/19767/1 Note: these 2to3 fixes report no changes are necessary: asserts, buffer, exec, execfile, exitfunc, filter, getcwdu, imports2, input, intern, itertools, metaclass, methodattrs, ne, next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr, standarderror, sys_exc, throw, tuple_params, xreadlines. Change-Id: Idda031c1ec975417c79323aea33e7b694e752b2a updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* Replace the BROKEN_TESTS environment variable valuedkhandel2018-07-061-125/+30
| | | | | | | | | | | In distributed-testing framework for regression, the value of BROKEN_TESTS environment variable needs to be replaced. This list of broken tests is different from what we have marked as a bad test in our test suites. Fixes: bz#1598325 Change-Id: I6a3ca5d9500170201ecaafcd5a96d9df3e46a851 Signed-off-by: Deepshikha Khandelwal <dkhandel@redhat.com>