summaryrefslogtreecommitdiffstats
path: root/configure.ac
Commit message (Collapse)AuthorAgeFilesLines
* socket/ssl: fix crl handlingMilind Changire2019-03-191-0/+2
| | | | | | | | | | | | | | | | Problem: Just setting the path to the CRL directory in socket_init() wasn't working. Solution: Need to use special API to retrieve and set X509_VERIFY_PARAM and set the CRL checking flags explicitly. Also, setting the CRL checking flags is a big pain, since the connection is declared as failed if any CRL isn't found in the designated file or directory. A comment has been added to the code appropriately. Change-Id: I8a8ed2ddaf4b5eb974387d2f7b1a85c1ca39fe79 fixes: bz#1687326 Signed-off-by: Milind Changire <mchangir@redhat.com>
* rpm: add thin-arbiter packageAmar Tumballi2019-03-111-0/+1
| | | | | | | | | | | | | | | | | | | Discussion on thin arbiter volume - https://github.com/gluster/glusterfs/issues/352#issuecomment-350981148 Main idea of having this rpm package is to deploy thin-arbiter without glusterd and other commands on a node, and all we need on that tie-breaker node is to run a single glusterfs command. Also note that, no other glusterfs installation needs thin-arbiter.so. Make sure RPM contains sample vol file, which can work by default, and a script to configure that volfile, along with translator image. Change-Id: Ibace758373d8a991b6a19b2ecc60c93b2f8fc489 updates: bz#1674389 Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* build: include gluster/__init__.* files for python packagingNiels de Vos2019-02-221-0/+2
| | | | | | | | | | | | | | | | The gluster/__init__.py file was originaly part of the glupy installation. This file is required to have the python-gluster package function correctly, it is expected to provide the 'gluster' namespace for other python packages (like from the libgfapi-python project). Because glupy does not exist anymore, this file is now added to in a new extras/python directory. Change-Id: I14fe959778ee3344d7d54ba342c7928a4d8080a2 Fixes: c3fcff9ccbfcec1be242fd5cf210c9995586b078 Fixes: 8293d21280fd6ddfc9bb54068cf87794fc6be207 Updates: bz#1642810 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* core: implement a global thread poolXavi Hernandez2019-02-181-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements a thread pool that is wait-free for adding jobs to the queue and uses a very small locked region to get jobs. This makes it possible to decrease contention drastically. It's based on wfcqueue structure provided by urcu library. It automatically enables more threads when load demands it, and stops them when not needed. There's a maximum number of threads that can be used. This value can be configured. Depending on the workload, the maximum number of threads plays an important role. So it needs to be configured for optimal performance. Currently the thread pool doesn't self adjust the maximum for the workload, so this configuration needs to be changed manually. For this reason, the global thread pool has been made optional, so that volumes can still use the thread pool provided by io-threads. To enable it for bricks, the following option needs to be set: config.global-threading = on This option has no effect if bricks are already running. A restart is required to activate it. It's recommended to also enable the following option when running bricks with the global thread pool: performance.iot-pass-through = on To enable it for a FUSE mount point, the option '--global-threading' must be added to the mount command. To change it, an umount and remount is needed. It's recommended to disable the following option when using global threading on a mount point: performance.client-io-threads = off To enable it for services managed by glusterd, glusterd needs to be started with option '--global-threading'. In this case all daemons, like self-heal, will be using the global thread pool. Currently it can only be enabled for bricks, FUSE mounts and glusterd services. The maximum number of threads for clients and bricks can be configured using the following options: config.client-threads config.brick-threads These options can be applied online and its effect is immediate most of the times. If one of them is set to 0, the maximum number of threads will be calcutated as #cores * 2. Some distributions use a very old userspace-rcu library (version 0.7) for this reason, some header files from version 0.10 have been copied into contrib/userspace-rcu and are used if the detected version is 0.7 or older. An additional change has been made to io-threads to prevent that threads are started when iot-pass-through is set. Change-Id: I09d19e246b9e6d53c6247b29dfca6af6ee00a24b updates: #532 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* tests: run nfs tests only if --enable-gnfs is providedAmar Tumballi2019-01-241-0/+1
| | | | | | Fixes: bz#1665358 Change-Id: Idbf88ec3ac683733b32c313377eeb72f2819bf0d Signed-off-by: Amar Tumballi <amarts@redhat.com>
* configure: fix the duplicate CFLAGS optionsAmar Tumballi2019-01-111-27/+11
| | | | | | updates: bz#1193929 Change-Id: I403878719a3f81fb2ea951a951f84880fb54f3cc Signed-off-by: Amar Tumballi <amarts@redhat.com>
* configure: add a message about asan optionAmar Tumballi2018-12-291-0/+12
| | | | | | | | | This added hint helps to get ASan logs for the daemon processes, with this one can start using asan for regression tests. updates: bz#1633930 Change-Id: I3b39892d45d29ae514dad8ab10f65703c02003f1 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* symlink-cache: remove from the buildAmar Tumballi2018-12-131-2/+0
| | | | | | | | | | | | | | | symlink-cache was written as an experiment to reduce the load on 'build' systems, which keep doing symlink resolution to get the proper header files. But since last 6+ years, there was no way to add it to the volfile using gluster cli, and hence was not supported anymore. As it is not maintained, and as announced on [1], we are planning to remove it from the build system. [1]- https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html updates: bz#1635688 Change-Id: Iaa25069bceed04cf65f79a4b4a02c05cee848eb5 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* copy_file_range support in GlusterFSRaghavendra Bhat2018-12-121-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * libglusterfs changes to add new fop * Fuse changes: - Changes in fuse bridge xlator to receive and send responses * posix changes to perform the op on the backend filesystem * protocol and rpc changes for sending and receiving the fop * gfapi changes for performing the fop * tools: glfs-copy-file-range tool for testing copy_file_range fop - Although, copy_file_range support has been added to the upstream fuse kernel module, no release has been made yet of a kernel which contains the support. It is expected to come in the upcoming release of linux-4.20 So, as of now, executing copy_file_range fop on a fused based filesystem results in fuse kernel module sending read on the source fd and write on the destination fd. Therefore a small gfapi based tool has been written to be able test the copy_file_range fop. This tool is similar (in functionality) to the example program given in copy_file_range man page. So, running regular copy_file_range on a fuse mount point and running gfapi based glfs-copy-file-range tool gives some idea about how fast, the copy_file_range (or reflink) can be. On the local machine this was the result obtained. mount -t glusterfs workstation:new /mnt/glusterfs [root@workstation ~]# cd /mnt/glusterfs/ [root@workstation glusterfs]# ls file [root@workstation glusterfs]# cd [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new real 0m6.495s user 0m0.000s sys 0m1.439s [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr OPEN_SRC: opening /file is success OPEN_DST: opening /rrr is success FSTAT_SRC: fstat on /rrr is success copy_file_range successful real 0m0.309s user 0m0.039s sys 0m0.017s This tool needs following arguments 1) hostname 2) volume name 3) log file path 4) source file path (relative to the gluster volume root) 5) destination file path (relative to the gluster volume root) "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>" - Added a testcase as well to run glfs-copy-file-range tool * io-stats changes to capture the fop for profiling * NOTE: - Added conditional check to see whether the copy_file_range syscall is available or not. If not, then return ENOSYS. - Added conditional check for kernel minor version in fuse_kernel.h and fuse-bridge while referring to copy_file_range. And the kernel minor version is kept as it is. i.e. 24. Increment it in future when there is a kernel release which contains the support for copy_file_range fop in fuse kernel module. * The document which contains a writeup on this enhancement can be found at https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367 updates: #536 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* encryption: remove crypt xlator from buildAmar Tumballi2018-12-111-27/+4
| | | | | | | | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removing crypt translator from the build. [1] - https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html Crypt xlator helped in on-disk / at-rest encryption of data. But currently as there are no maintainers for this, planning to remove it from master codebase. We are planning to host these experimental/ tech-preview xlators in another repository, so people who want to contribute can still use the bits. updates: bz#1635688 Change-Id: I7f2453907a595c34f635a88c49aab0845369c6e7 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* configure.ac: fix option to stop automake aborting on pointless errorEli Schwartz2018-12-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | glusterfs does not follow the GNU coding standards and therefore must use the "foreign" strictness. Without this, autoreconf -fi would fail to execute successfully because automake returned non-zero. This change ensures that people using autoreconf, the GNU preferred invocation method for the autotools build system, can successfully set up the build. Remove the pointless --foreign argument from the autogen.sh invocation of automake. Not only is configure.ac the preferred way to define such options (rather than handwritten, piecemeal invocations of every tool in the autotools toolchain), it was never needed in the autogen.sh as that script provides no error handling at all and always (incorrectly) returns successfully as long as autotools itself is installed (no matter how broken glusterfs itself is). Change-Id: Ib0246d5368a54594f517a322465cffb9a85c1b49 fixes: bz#1656100 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
* build: add option to compile with ThreadSanitizerNiels de Vos2018-11-271-0/+14
| | | | | | | | | | | | ThreadSanitizer is a debugging tool that can detect threads that race for data modifications. These races can result in data corruption and are difficult to track and fix. Change-Id: Ibbdaf17c811e30e79cd5bdcf9cd9ff2d0cdb2abb URL: https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual Reported-by: Yaniv Kaul <ykaul@redhat.com> Fixes: #543 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* bd: remove from the buildAmar Tumballi2018-11-081-40/+0
| | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removed BD (block device) translator from the build. [1] - https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html Updates: bz#1635688 Change-Id: Ia96db406c58a7aef355dde6bc33523bb2492b1a9 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* glupy: remove from the buildAmar Tumballi2018-11-081-44/+1
| | | | | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removing 'glupy' translator from the build. [1] https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html This patch aims at clearing the translator from build and tests. A followup is needed to remove the code from repository. Updates: bz#1642810 Change-Id: I41d0c1956330c3bbca62c540ccf9ab01bbf3a092 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* xlators/experimental: remove from the buildAmar Tumballi2018-11-081-32/+0
| | | | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removing all experimental translators from the build. [1] https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html Note that as followup patch, there would be a patch to remove the code. Updates: bz#1635688 Change-Id: I8ebc256517feb37fc7580104e8bebe27d047f959 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* tiering: remove the translator from build and glusterdAmar Tumballi2018-11-021-34/+0
| | | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removing tier translator from the build. Also make sure there are no regression tests involving tiering feature are present. [1] https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html Change-Id: I2c177f711f9b54b7b24e1a13525ff3132bd9a9c5 updates: bz#1642807 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* stripe: remove the translator from build and glusterdAmar Tumballi2018-10-311-2/+0
| | | | | | | | | | | | | | | | Based on the proposal to remove few features as they are not actively maintained [1], removing stripe translator from the build. Also make sure there are no regression tests involving stripe translator. [1] https://lists.gluster.org/pipermail/gluster-users/2018-July/034400.html Note that this patch aims at removing the translator from build, and a followup patch is needed to remove the code from repository. Updates: bz#1364707 Change-Id: I235b305338f138e29e9f30cba65bc0dadbebbbd5 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* socket: use accept4/paccept for nonblocking socketKrishnan Parthasarathi2018-10-121-0/+10
| | | | | | | | | | | | | | | | | This reduces the no. of syscalls on Linux systems from 2, accept(2) and fcntl(2) for setting O_NONBLOCK, to a single accept4(2). On NetBSD, we have paccept(2) that does the same, if we leave signal masking aside. Added sys_accept which accepts an extra flags argument than accept(2). This would opportunistically use accept4/paccept as available. It would fallback to accept(2) and fcntl(2) otherwise. While at this, the patch sets FD_CLOEXEC flag on the accepted socket fd. BUG: 1236272 Change-Id: I41e43fd3e36d6dabb07e578a1cea7f45b7b4e37f fixes: bz#1236272 Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
* python3: assume python3 unless building _packages_ on sys without py3Kaleb S. KEITHLEY2018-09-271-13/+0
| | | | | | | | | | | | | | | The jenkins release-new job runs on a CentOS 7 box, which does not have python3. As a result it runs (autogen.sh and) configure before producing the dist tar file, converting all the python3 shebangs to python2 shebangs in the dist tar file. Then when that tar file is "carried" to, e.g. Fedora koji build system to build packages, the shebangs are incorrect, despite having originally been correct in the git repo. Change-Id: I5154baba3f6d29d3c4823bafc2b57abecbf90e5b updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* build: cleanup xlator link, --no-undefined, libuuidv6devKaleb S. KEITHLEY2018-09-121-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While attempting to build a (pre-)5.0 of glusterfs on Ubuntu bionic and cosmic, it became apparent that there are some gremlins hiding in the combination of the xlator export-symbols, the newish addition of -Wl,--no-undefined, and the new switch to libuuid from the old contrib/uuid. Note: even though Fedora 28 (and later) and Ubuntu bionic (and later) have the same nominal version of libtool, the Fedora version appears to do a better job of recursing through dependencies to determine the libraries to link with. Examination of the build logs showed that despite appearing to work on Fedora, not all xlators and shared libs were linked with -Wl, --no-undefined, and -luuid. And in the case of the gnfs xlator, it was not only not linked with -Wl,--no-undefined but alsos not linked with -lgfxdr and -lgfrpc. Added GF_XLATOR_LDFLAGS, similar to GF_XLATOR_DEFAULT_LDFLAGS. GF_XLATOR_DEFAULT_LDFLAGS is for xlators that export/expose the default or common set of symbols. GF_XLATOR_LDFLAGS is for those remaining xlators that export/expose non-default symbols, e.g. dht and glupy. This removes the need in the future to add things like $(UUID_LIBS) to every xlator's Makefile.am. Just add it to GF_XLATOR_LDFLAGS and GF_XLATOR_DEFAULT_LDFLAGS in configure.ac and you're done. This patch was tested on Fedora 28 (build, rpmbuild), Fedora Rawhide/30 (rpmbuild), RHEL8 (rpmbuild), CentOS7 (rpmbuild), Fedora koji --scratch build for f30/rawhide, and a Launchpad build for Ubuntu cosmic/18.10. Change-Id: Ieca104fa5c5d3c094e701c8ca4a73754dd0292b0 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* rpm: add "--with asan" build optionNiels de Vos2018-09-111-1/+1
| | | | | | | | | | | | RHEL-6 does not have libasan, enabling the "--with asan" option is a no-op there. RHEL-7 has an earlier version of libasan, and that does not have the __asan_init symbol. Test for __asan_report_error in confiure.as instead. Change-Id: I6322e832c5cfbd7d750f5c32c84c28771674ced6 Updates: #492 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: Make building of glupy more obviousAnoop C S2018-09-101-12/+8
| | | | | | | | | | | | | | | | | | | | | | Yet again glupy..!! Core changes: * Provide --disable-glupy(instead of --enable-glupy) since we try to build it by default. * AC_SUBST macro is considered/executed even if it is inside an if condition. As a result GLUPY_SUBDIR was always substituted which in turn would make compilation enter glupy/ translator sources in absence of python{2/3}-devel and fails. Miscellaneous changes: * Remove an explicit echo for printing PYTHON version. * Replace hard coded python version in warning message displayed in absence of python{2/3}-devel. * Redirect pushd and popd output to /dev/null Change-Id: If1ba30a12a8bda5763ef528787fccb2f7946b136 Updates: bz#1193929 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* core: python3Kaleb S. KEITHLEY2018-09-031-49/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://review.gluster.org/#/c/19788/, https://review.gluster.org/#/c/19871/, https://review.gluster.org/#/c/19952/, https://review.gluster.org/#/c/20104/, https://review.gluster.org/#/c/20162/, https://review.gluster.org/#/c/20185/, https://review.gluster.org/#/c/20207/, https://review.gluster.org/#/c/20227/, https://review.gluster.org/#/c/20307/, https://review.gluster.org/#/c/20320/, https://review.gluster.org/#/c/20332/, https://review.gluster.org/#/c/20364/, https://review.gluster.org/#/c/20441/, and https://review.gluster.org/#/c/20484 shebangs changed from /usr/bin/python2 to /usr/bin/python3. (Reminder, various distribution packaging guidelines require use of explicit python version and don't allow '#!/usr/bin/env python', regardless of how handy that idiom may be.) glusterfs.spec(.in) package python{2,3}-gluster and python2 or python3 dependencies as appropriate. configure(.ac): + test for and use python2 or python3 as appropriate. If build machine has python2 and python3, use python3. Override by setting PYTHON=/usr/bin/python2 when running configure. + PYTHONDEV_CPPFLAGS from python[23]-config --includes is a better match to the original python sysconfig.get_python_inc(). All those other extraneous flags breaks the build. + Only change the shebangs once. Changing them over and over again, e.g., during a `make glusterrpms` in extras/LinuxRPM just sends make (is it really make that's looping?) into an infinite loop. If you figure out why, let me know. + Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie). Everything else has 2.7 or 3.x + logic from https://review.gluster.org/c/glusterfs/+/21050, which needs to be removed/merged after that patch is merged. Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the mysterious RHEL > 7. Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f updates: #411 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* build: add --enable-asan configure optionsNiels de Vos2018-08-301-6/+23
| | | | | | | | | | | | | | Introduce a `./configure --enable-asan` to build with `-fsanitize=address -fno-omit-frame-pointer` options. This uses the libasan.so shared library, so that needs to be available. While running builds with the ASAN options, several linker issues surfaced and these have been addressed with this change as well. Building with --enable-asan has been tested on Fedora 28. Change-Id: I428a9da70dd8f7d0056cfbe5c398619a571469b2 Updates: #492 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: use standard PKG_CHECK_MODULES for libxml2 availabilityNiels de Vos2018-08-161-2/+1
| | | | | | | | | | | | | In case the development parts of libxml2 are not installed, it was required to re-run ./autogen.sh to cleanup the cached values for the check. This is not nice towards users. By using the standard PKG_CHECK_MODULES for libxml-2.0 the results of the check are not cached and will be probed again when running ./configure. Change-Id: I3c4586e5555a521be5d4fb61bdb873ae0317311a Fixes: bz#1599219 Reported-by: Sachidananda Urs <surs@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: remove bundled arg-standaloneNiels de Vos2018-07-281-22/+15
| | | | | | | | | | | | | | libargp or argp-standalone is available on all commonly used distributions. There is no need to bundle an unmaintained version of argp-standalone in this repository anymore. FreeBSD places the argp.h file in /usr/local/include when argp-standalone is installed. This path is not added to CPPFLAGS by default, so thats done in configure.ac as well. Change-Id: I384a53ab0a008ec9d48fd83afeaf8fbc197e91ee Fixes: bz#1609337 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: remove uuid from contrib/Niels de Vos2018-07-271-4/+17
| | | | | | | | | | | | | | Bundling libuuid is not needed anymore, all current distributions provide it now. Some OS's provide their own uuid_*() functions in libc. These may not be fully compatible with libuuid.so found on Linux systems. In that case, either e2fsprogs-libuuid can be installed, or support for the native uuid_*() functions can be added to libglusterfs/src/compat-uuid.h. Change-Id: Icfa48caea81307a3bca549364969c2038911942b Fixes: bz#1607319 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: rename event.h to gf-event.hNiels de Vos2018-07-271-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Newer FreeBSD versions (noticed with 10.3-RELEASE) provide a event.h file that on occasion gets included instead of the libglusterfs file. When this happens, 'struct event_pool' will not be defined and building will fail with errors like: autoscale-threads.c:18:55: error: incomplete definition of type 'struct event_pool' int thread_count = pool->eventthreadcount; ~~~~^ autoscale-threads.c:17:16: note: forward declaration of 'struct event_pool' struct event_pool *pool = ctx->event_pool; ^ This problem is caused by 'pkg-config --cflags uuid' that adds /usr/local/include to the GF_CPPFLAGS. The use of libuuid is preferred so that the contrib/uuid/ directory can be removed. By renaming event.h to gf-event.h there is no conflict between the different event.h files anymore and compiling on FreeBSD works without issues. Change-Id: Ie69f6b8a4f8f8e9630d39a86693eb74674f0f763 Updates: bz#1607319 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* Fix compile warningsXavi Hernandez2018-07-101-3/+0
| | | | | | | | | | | This patch fixes compile warnings that appear with newer compilers. The solution applied is only to remove the warnings, but it doesn't always solve the problem in the best way. It assumes that the problem will never happen, as the previous code assumed. Change-Id: I6e8470d6c2e2dbd3bd7d324b5fd2f92ffdc3d6ec updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* cloudsync: Adding s3 plugin for cloudsyncSusant Palai2018-05-301-0/+22
| | | | | | | | | | | | | | | | | | This is a plugin which provides an interface to retrive files from amazon-s3 which are archived in to s3. Users need to give the above information for cloudsync to retrieve the file from s3. TODO: 1- A separate commit in to developer-guide will detail about the usage of this plugin in more detail. 2- Need to create target file in aws-bucket with "gfid" names. Helps avoiding name collisions. Change-Id: I2e4a586f4e3f86164de9178e37673a07f317e7d9 Updates: #387 Signed-off-by: Susant Palai <spalai@redhat.com>
* build: Disallow unresolved symbol referencesPrashanth Pai2018-05-181-6/+21
| | | | | | | | | | | | | | | | | | | | | | | | In the past, it was often[1] forgotten for xlators to be linked against the symbols they refer to. This often caused glusterd2 to fail while loading xlator's shared object (.so) file. This change adds "--no-undefined" as a linker flag which causes the linker to treat unresolved symbol references as an error and hence fail linking. [1]: https://review.gluster.org/#/c/19912/ https://review.gluster.org/#/c/19664/ https://review.gluster.org/#/c/19056/ https://review.gluster.org/#/c/17659/ https://bugzilla.redhat.com/show_bug.cgi?id=1532238 Bonus: Added cloudsync and utime xlator's generated source files to .gitignore Updates: bz#1193929 Change-Id: I9604a4a87b7313a5fa43bda5fdb37dfa7ef8facd Signed-off-by: Prashanth Pai <ppai@redhat.com>
* utime: ctime client side xlatorKotresh HR2018-05-061-0/+2
| | | | | | | | | | | | | | The client side utime xlator does two things. 1. Update unix epoch time in frame->root->ctime 2. Update the frame->root->flags based on the fop which indicates time attributes that should be updated for the parent/entry. Credits: Rafi KC <rkavunga@redhat.com> Updates: #208 Change-Id: I9cad297040c70798a0a8468a080eb4aeff73138d Signed-off-by: Kotresh HR <khiremat@redhat.com>
* feature/thin-arbiter: Implement thin-arbiter translatorAshish Pandey2018-04-251-0/+2
| | | | | | | Updates #352 Change-Id: I3d8caa6479dc8e48bec62a09b056971bb061f0cf Signed-off-by: Ashish Pandey <aspandey@redhat.com>
* build: exclude '--with-previous-options' to prevent infinite loopXie Changlong2018-04-161-1/+1
| | | | | | | | | | | | Reproducible Steps: 1. cd glusterfs/; rm -rf *; git reset --hard #clean repo 2. cd extras/LinuxRPM/; ./make_glusterrpms #it's ok here 3. ./make_glusterrpms #infinite loop 4. cd ../../; make distclean #infinite loop Change-Id: I162953d4576cedea7c6f6c631a77163a5cca023e updates: #439 Signed-off-by: Xie Changlong <xiechanglong@cmss.chinamobile.com>
* core/build/various: python3 compat, prepare for python2 -> python3Kaleb S. KEITHLEY2018-04-121-45/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note 1) we're not supposed to be using #!/usr/bin/env python, see https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines Note 2) we're also not supposed to be using "!/usr/bin/python, see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out The previous patch (https://review.gluster.org/19767) tried to do too much in one patch, so it was abandoned. This patch does two things: 1) minor cleanup of configure(.ac) to explicitly use python2 2) change all the shebang lines to #!/usr/bin/python2 and add them where they were missing based on warnings emitted during rpmbuild. In a follow-up patch python2 will eventually be changed to python3. Before that python2-isms (e.g. print, string.join(), etc.) need to be converted to python3. Some of those can be rewritten in version agnostic python. E.g. print statements become print() with "from __future_ import print_function". The python 2to3 utility will be used for some of those. Also Aravinda has given guidance in the comments to the first patch for changes. updates: #411 Change-Id: I471730962b2526022115a1fc33629fb078b74338 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* experimental/cloudsync: Download xlator for archival featureSusant Palai2018-04-101-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | spec-files: https://review.gluster.org/#/c/18854/ Overview: * Cloudsync maintains three file states in it's inode-ctx i.e 1 - LOCAL, 2 - REMOTE, 3 - DOWNLOADING. * A data modifying fop is allowed only if the state is LOCAL. If the state is REMOTE or DOWNLOADING, client will download or wait for the download to finish initiated by other client. * Multiple download and upload from different clients are synchronized by inodelk. * In POSIX a state check is done (part of different commit)before allowing the fop to continue. If the state is remote/downloading the fop is unwound with EREMOTE. The client will then download the file and continue with the fop again. * Basic Algo for fop (let's say write fop): - If LOCAL -> resume fop - If REMOTE -> - INODELK - STAT (this gets state and heal the state if needed) - DOWNLOAD - resume fop Note: * Developers will need to write plugins for download, based on the remote store they choose. In phase-1, support will be added for one remote store per volume. In future, more options for multiple remote stores will be explored. TODOs: - Implement stat/lookup/readdirp to return size info from xattr - Make plugins configurable - Implement unlink fop - Add metrics collection - Add sharding support Design Contributions: Aravinda V K <avishwan@redhat.com> Amar Tumballi <amarts@redhat.com> Ram Ankireddypalle <areddy@commvault.com> Susant Palai <spalai@redhat.com> updates: #387 Change-Id: Iddf711ee7ab4e946ae3e472ff62791a7b85e6d4b Signed-off-by: Susant Palai <spalai@redhat.com>
* build: revert configure --without-ipv6-default behaviourKaleb S. KEITHLEY2018-04-021-14/+31
| | | | | | | | | | | | | | | | | | | | | | | | | patch https://review.gluster.org/19692 breaks gluster on systems with IPv6 enabled but don't have IPv6 reverse DNS. Also it defaulted to enabling ipv6-default regardless of whether --with-ipv6-default or --without-ipv6-default were specified in the options to configure. (Also the patch was merged without review.) Prefer libtirpc over glibc rpc. On newer linux with tirpc and without glibc rpc use tirpc (obviously) on less new linux with both tirpc and glibc rpc default to use tirpc, unless --without-tirpc is specified, in which case use glibc rpc On less new linux without tirpc fall back to glib rpc (obviously) ipv6-default requires libtirpc. It is off by default. It must be explicitly enabled with --with-ipv6-default. If --with-ipv6-default is specified, but tirpc is not available, disable it and issue a warning Change-Id: Ib96a230fafb83ec83a71948fe55af1215a7a6ffa BUG: 1562052 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* build: Fix misleading TIRPC result in configure summaryJames Le Cuirot2018-03-241-5/+4
| | | | | | | | | | | | | | Requesting ipv6-default even if you explicitly disable libtirpc will then implicitly enable libtirpc because that is required. That is fine but the configure summary should not then show TIRPC as disabled when it is not. The result has also been made clearer by stating that TIRPC is "missing" when it has been tried but not found. BUG: 1553938 Change-Id: I945bd6859aaf3defa682b0d05ee34a9827b9c45f Signed-off-by: James Le Cuirot <chewi@gentoo.org>
* build: Fix configure --without-ipv6-default behaviourJames Le Cuirot2018-03-241-1/+1
| | | | | | | | | | | | | | | | The current behaviour disables ipv6-default when no switch is given at all but otherwise checks if libtirpc was requested, regardless of whether you have given --with-ipv6-default or --without-ipv6-default. I believe the intention was to enable when libtirpc is requested by default but otherwise respect the switch given. This is important because ipv6-default breaks Gluster for systems that have IPv6 disabled. BUG: 1553926 Change-Id: I76b91ae2699574b2e5b777453732bb5cbd79bbca Signed-off-by: James Le Cuirot <chewi@gentoo.org>
* build: fix typo, spelling mistakeKaleb S. KEITHLEY2018-03-021-1/+1
| | | | | | | transistional -> transitional Change-Id: I1eb7e063288384458c305afea6d6c46a358701ff Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* xlators/features/namespace: Add namespace xlator and link into brick graphVarsha Rao2018-02-211-0/+2
| | | | | | | | | | | | | | | | | | | | | The following release-3.8-fb branch patch is upstreamed: > features/namespace: Add namespace xlator and link into brick graph > Commit ID: dbd30776f26e > https://review.gluster.org/#/c/18041/ > By Michael Goulet <mgoulet@fb.com> Changes in this patch: Removes extra config.h and namespace.h file in namespace.c Adds default_getspec_cbk to libglusterfs.sym Rename dict_for_each to dict_foreach_inline Remove fd.h header file stack.h Add test case for truncate, open and symlink This patch is required to forward port io-threads namespace patch. Updates: #401 Change-Id: Ib88c95b89eecee9b8957df8a4c8712c899c761d1 Signed-off-by: Varsha Rao <varao@redhat.com>
* build: add --without-server optionNiels de Vos2018-02-191-7/+15
| | | | | | | | | | | | | | | | With Gluster 4.0 we will not provide the server components for EL6 and older. At one point Gluster 4.x will get GlusterD2, which requires Golang tools in the distribution. EL6 does not contain these at the moment. With this change, it is possible to `./configure --without-server` which prevents building glusterd and the xlators for the bricks. Building RPMs can pass `--without server` and the glusterfs-server sub-package will not be created. Change-Id: I97f5ccf9f2c76e60d9af83915fc59fae57ad6d25 BUG: 1074947 Signed-off-by: Niels de Vos <ndevos@redhat.com>
* build: Remove unwanted autoconf messages for libtirpcAnoop C S2018-02-041-2/+0
| | | | | | | | | | | | | AC_MSG_RESULT is normally used when it is preceded by AC_MSG_CHECKING during explicit checking of certain requirements. With PKG_CHECK_MODULES being used for checking the presence of libtirpc itself generates the following message: checking for TIRPC... yes Change-Id: I3f088a45ef2ced6f6fd9e1524e758c812deecb8f BUG: 1536186 Signed-off-by: Anoop C S <anoopcs@redhat.com>
* build: restore ability to control verbosity settingsJan Engelhardt2018-01-261-8/+0
| | | | | | | | | | | | | | | `make V=1` is broken — no commands are printed whatsoever. At the same time, `make V=0` *also* is broken in that no summary lines ("CC  foo.o") are printed, either. Kill the annoying hardcoded --quiet in configure.ac, since it seems to override everything that automake offers. Change-Id: I377c0e0469619a33586afb4a93dde6d241e7bc21 Fixes: #381 BUG: 1539023 Original-author: Jan Engelhardt <jengelh@inai.de> Signed-off-by: Niels de Vos <ndevos@redhat.com> [ndevos: silence rpc/xdr/gen rpgcen]
* build: glibc has removed rpc headers and rpcgen in Fedora28, use libtirpcKaleb S. KEITHLEY2018-01-241-18/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Other Linux distributions are doing the same; some others have already done so. Switch to libtirpc(-devel) and unbundled rpcgen packages. For now rpcgen is still provided by the glibc-rpcgen RPM, but rpcsvc-proto's rpcgen subpackage is available now but will not be used until glibc-rpcgen is retired. (note, rpcsvc-proto's rpcgen is just named rpcgen-...rpm. I.e. not rpcsvc-proto-rpcgen.) Right now either one will satisfy the BuildRequires: rpcgen. Also, when a .spec file has BuildRequires: foo-devel it is not necessary to also have: BuildRequires: foo or even: BuildRequires: foo foo-devel The foo-devel package has a dependency on foo, which will install foo automatically. It's usually also not necessary to have a corresponding Requires: foo as the rpmbuild process will also automatically determine the install-time dependencies. And other minor glusterfs.spec.in cleanup of ipv6-default, including sorting the argument definitions because the comment says "keep them sorted" (Though nobody appears to have ever sorted them in the first place.) Change-Id: I86f847dfda0fef83e22c6e8b761342d652a2d9ba BUG: 1536186 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* dentry fop serializer: added new server side xlator for dentry fop serializationSakshi Bansal2018-01-241-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problems addressed by this xlator : [1]. To prevent race between parallel mkdir,mkdir and lookup etc. Fops like mkdir/create, lookup, rename, unlink, link that happen on a particular dentry must be serialized to ensure atomicity. Another possible case can be a fresh lookup to find existance of a path whose gfid is not set yet. Further, storage/posix employs a ctime based heuristic 'is_fresh_file' (interval time is less than 1 second of current time) to check fresh-ness of file. With serialization of these two fops (lookup & mkdir), we eliminate the race altogether. [2]. Staleness of dentries This causes exponential increase in traversal time for any inode in the subtree of the directory pointed by stale dentry. Cause : Stale dentry is created because of following two operations: a. dentry creation due to inode_link, done during operations like lookup, mkdir, create, mknod, symlink, create and b. dentry unlinking due to various operations like rmdir, rename, unlink. The reason is __inode_link uses __is_dentry_cyclic, which explores all possible path to avoid cyclic link formation during inode linkage. __is_dentry_cyclic explores stale-dentry(ies) and its all ancestors which is increases traversing time exponentially. Implementation : To acheive this all fops on dentry must take entry locks before they proceed, once they have acquired locks, they perform the fop and then release the lock. Some documentation from email conversation: [1] http://www.gluster.org/pipermail/gluster-devel/2015-December/047314.html [2] http://www.gluster.org/pipermail/gluster-devel/2015-August/046428.html With this patch, the feature is optional, enable it by running: `gluster volume set $volname features.sdfs enable` Also the feature is tested for a month without issues in the experiemental branch for all the regression. Change-Id: I6e80ba3cabfa6facd5dda63bd482b9bf18b6b79b Fixes: #397 BUG: 1304962 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
* build: Use pkg-config to get libtirpc flagsJames Le Cuirot2017-11-291-4/+4
| | | | | | | This benefits cross-compiling and other exotic setups. Change-Id: Id9f168728d96264ccab23d2e618fa2b4003455a3 Signed-off-by: James Le Cuirot <chewi@gentoo.org>
* build: Allow libtirpc to be explicitly requestedJames Le Cuirot2017-11-221-2/+10
| | | | | | | | Some distributions like Gentoo no longer include the RPC stuff in their glibc packages. Change-Id: Ic47065e9c2f5a0ccd860df9d7185eff59990ff10 Signed-off-by: James Le Cuirot <chewi@gentoo.org>
* geo-rep: Refactoring Config and Arguments parsingAravinda VK2017-11-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Fixed Python pep8 issues - Removed dead code - Rewritten configuration management - Rewritten Arguments/subcommands handling - Added Args upgrade to accommodate all these changes without changing glusterd code - use of md5 removed, which was used to hash the brick path for workdir Both Master and Slave nodes will have subdir for session in the format "<mastervol>_<primary_slave_host>_<slavevol> $GLUSTER_LOGDIR/geo-replication/<mastervol>_<primary_slave_host>_<slavevol> $GLUSTER_LOGDIR/geo-replication-slaves/<mastervol>_<primary_slave_host>_<slavevol> Log file paths renamed since session info is available with directory name itself. $LOG_DIR_MASTER/ - gsyncd.log - Gsyncd, Worker monitor logs - mnt-<brick-path>.log - Aux mount logs, mounted by each worker - changes-<brick-path>.log - Changelog related logs(One per brick) $LOG_DIR_SLAVE/ - gsyncd.log - Slave Gsyncd logs - mnt-<master-node>-<master-brick-path>.log - Aux mount logs, mounted for each connection from master-node:master-brick - mnt-mbr-<master-node>-<master-brick-path>.log - Same as above, but mountbroker setup Fixes: #73 Change-Id: I2ec2a21e4e2a92fd92899d026e8543725276f021 Signed-off-by: Aravinda VK <avishwan@redhat.com>
* libglusterfs/atomic: Improved atomic supportXavier Hernandez2017-11-141-3/+9
| | | | | | | | | | | | | | | | This patch solves a detection problem in configure.ac that prevented that compilation detects builtin __atomic or __sync functions. It also adds more atomic types and support for other atomic functions. An special case has been added to support 64-bit atomics on 32-bit systems. The solution is to fallback to the mutex solution only for 64-bit atomics, but smaller atomic types will still take advantage of builtins if available. Change-Id: I6b9afc7cd6e66b28a33278715583552872278801 BUG: 1510397 Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-24 15:48:36 +0000