| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
| |
When using inode_ctx_get() or inode_ctx_set(), a 'uint64_t *' is expected.
In many cases, the value to retrieve or store is a pointer, which will be
of smaller size in some architectures (for example 32-bits). In this case,
directly passing the address of the pointer casted to an 'uint64_t *' is
wrong and can cause memory corruption.
Change-Id: Iae616da9dda528df6743fa2f65ae5cff5ad23258
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Fixes: bz#1785611
|
|
|
|
|
|
|
|
|
| |
Convert all gf_msg() to gf_smsg()
Updates: #657
Change-Id: I72215b2518df78174dda8a7bc8de6f21fe1ba10f
Signed-off-by: yatipadia <ypadia@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are many include statements that are not needed.
A previous more ambitious attempt failed because of *BSD plafrom
(see https://review.gluster.org/#/c/glusterfs/+/21929/ )
Now trying a more conservative reduction.
It does not solve all circular deps that we have, but it
does reduce some of them. There is just too much to handle
reasonably (dht-common.h includes dht-lock.h which includes
dht-common.h ...), but it does reduce the overall number of lines
of include we need to look at in the future to understand and fix
the mess later one.
Change-Id: I550cd001bdefb8be0fe67632f783c0ef6bee3f9f
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch cleans some iovec code and creates two additional helper
functions to simplify management of iovec structures.
iov_range_copy(struct iovec *dst, uint32_t dst_count, uint32_t dst_offset,
struct iovec *src, uint32_t src_count, uint32_t src_offset,
uint32_t size);
This function copies up to 'size' bytes from 'src' at offset
'src_offset' to 'dst' at 'dst_offset'. It returns the number of
bytes copied.
iov_skip(struct iovec *iovec, uint32_t count, uint32_t size);
This function removes the initial 'size' bytes from 'iovec' and
returns the updated number of iovec vectors remaining.
The signature of iov_subset() has also been modified to make it safer
and easier to use. The new signature is:
iov_subset(struct iovec *src, int src_count, uint32_t start, uint32_t size,
struct iovec **dst, int32_t dst_count);
This function creates a new iovec array containing the subset of the
'src' vector starting at 'start' with size 'size'. The resulting
array is allocated if '*dst' is NULL, or copied to '*dst' if it fits
(based on 'dst_count'). It returns the number of iovec vectors used.
A new set of functions to iterate through an iovec array have been
created. They can be used to simplify the implementation of other
iovec-based helper functions.
Change-Id: Ia5fe57e388e23392a8d6cdab17670e337cadd587
Updates: bz#1193929
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
| |
fixes: bz#1622665
Change-Id: I777d67b1b62c284c62a02277238ad7538eef001e
Signed-off-by: Iraj Jamali <ijamali@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently io-cache invalidate pages falling in the range of write. But
instead it can update pages with same data so that reads can make use
of the cache.
credits: Xavi Hernandez <xhernandez@redhat.com>
Change-Id: I932bd3da97ddfd464187f3009b1013eb334f00a7
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
updates: bz#1659869
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since glusterd2 don't maintain the xlator option details in code, it
directly reads the xlators options table from `*.so` files. To support
enable and disable of xlator new option added to the option table with
the name same as xlator name itself.
This change will not affect the functionality with glusterd1.
Change-Id: I23d9e537f3f422de72ddb353484466d3519de0c1
updates: #302
Signed-off-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
| |
Fixes: #164
Change-Id: I93ad6f0232a1dc534df099059f69951e1339086f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.
Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs
This change although big, is just moving around the headers and
making it correct when including these headers from other sources.
This helps us correctly include libglusterfs includes without
namespace conflicts.
Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
io-cache xlator has been skipping xdata references when the
date needs to be read into page cache. This patch fixes the same.
Note: similar changes may be needed for other fops as well
which are handled by io-cache.
Change-Id: I28d73d4ba471d13eb55d0fd0b5197d222df77a2a
updates: bz#1648768
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
|
|
|
|
|
|
| |
Change-Id: I666eeb63ebd000711b3f793b948d4e0c04b1a242
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Updates: bz#1644629
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, there are possibilities in few places, where a user-controlled
(like filename, program parameter etc) string can be passed as 'fmt' for
printf(), which can lead to segfault, if the user's string contains '%s',
'%d' in it.
While fixing it, makes sense to make the explicit check for such issues
across the codebase, by making the format call properly.
Fixes: CVE-2018-14661
Fixes: bz#1644763
Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While it may slightly reduce accuracy, I think it's better to
acquire the time outside a lock and then memcpy the value while
under lock.
Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ic8fb94a56c4fa2d3b13e59767e13646fb39342ba
|
|
|
|
|
|
|
|
|
|
| |
When compiling in other architectures there appear many warnings. Some
of them are actual problems that prevent gluster to work correctly on
those architectures.
Change-Id: Icdc7107a2bc2da662903c51910beddb84bdf03c0
fixes: bz#1632717
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
| |
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
|
|
|
|
|
|
|
|
| |
This patch fixes CID 1382439 and 1382412.
Change-Id: I8696623c168ba76ae2ecac7c9582b4e50437bc53
updates: bz#789278
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
xlators/storage/posix/src/posix-inode-fd-ops.c:
xlators/storage/posix/src/posix-helpers.c:
xlators/storage/bd/src/bd.c:
xlators/protocol/client/src/client-lk.c:
xlators/performance/quick-read/src/quick-read.c:
xlators/performance/io-cache/src/page.c
xlators/nfs/server/src/nfs3-helpers.c
xlators/nfs/server/src/nfs-fops.c
xlators/nfs/server/src/mount3udp_svc.c
xlators/nfs/server/src/mount3.c
xlators/mount/fuse/src/fuse-helpers.c
xlators/mount/fuse/src/fuse-bridge.c
xlators/mgmt/glusterd/src/glusterd-utils.c
xlators/mgmt/glusterd/src/glusterd-syncop.h
xlators/mgmt/glusterd/src/glusterd-snapshot.c
xlators/mgmt/glusterd/src/glusterd-rpc-ops.c
xlators/mgmt/glusterd/src/glusterd-replace-brick.c
xlators/mgmt/glusterd/src/glusterd-op-sm.c
xlators/mgmt/glusterd/src/glusterd-mgmt.c
xlators/meta/src/subvolumes-dir.c
xlators/meta/src/graph-dir.c
xlators/features/trash/src/trash.c
xlators/features/shard/src/shard.h
xlators/features/shard/src/shard.c
xlators/features/marker/src/marker-quota.c
xlators/features/locks/src/common.c
xlators/features/leases/src/leases-internal.c
xlators/features/gfid-access/src/gfid-access.c
xlators/features/cloudsync/src/cloudsync-plugins/src/cloudsyncs3/src/libcloudsyncs3.c
xlators/features/bit-rot/src/bitd/bit-rot.c
xlators/features/bit-rot/src/bitd/bit-rot-scrub.c
bxlators/encryption/crypt/src/metadata.c
xlators/encryption/crypt/src/crypt.c
xlators/performance/md-cache/src/md-cache.c:
Move to GF_MALLOC() instead of GF_CALLOC() when possible
It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.
In some cases, also changed allocation size to be sizeof some
struct or type instead of a pointer - easier to read.
In some cases, removed redundant strlen() calls by saving the result
into a variable.
1. Only done for the straightforward cases. There's room for improvement.
2. Please review carefully, especially for string allocation, with the
terminating NULL string.
Only compile-tested!
.. and allocate memory as much as needed.
xlators/nfs/server/src/mount3.c :
Don't blindly allocate PATH_MAX, but strlen() the string and allocate
appropriately.
Also, align error messges.
updates: bz#1193929
Original-Author: Yaniv Kaul <ykaul@redhat.com>
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ibda6f33dd180b7f7694f20a12af1e9576fe197f5
|
|
|
|
|
|
|
|
| |
This patch fixes CID 1382361, 1124714 and 1382432.
Change-Id: I0407f35ee44ec6e4522de46092658223d0c8ee6a
updates: bz#789278
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
| |
Fixes: bz789278
Change-Id: If8ca1fef8a10f1e7270390b61121f8a20a76b1d0
updates: bz#789278
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Add pass-through option in performance traslators. Set the option in
GF_OPTION_INIT() and GF_OPTION_RECONF()
Updates: #304
Change-Id: If1537450147d154905831e36f7162a32866d7ad6
Signed-off-by: Varsha Rao <varao@redhat.com>
|
|
|
|
|
| |
Change-Id: I811225ad20e3bd9f05820212e6a843f05d96b246
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Coverity issue : FORWARD_NULL
fd is assigned within a condition, but the fd is used even outside
the condition.
Change-Id: I6548d605d8a8acc6a25f1657f9fb75586d513042
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Update io-cache options for GD2
Updates gluster/glusterfs#302
Change-Id: I7fabf912d1e3cd024989404526db0a29644768c7
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
|
|
|
| |
This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.
Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity ID: 173
Issue: Event dead_error_line: Execution cannot reach this statement:
"fd_ctx_set(fd, this, 1UL);".
Removed if block as "weight == 0U" cannot be true, so if block will
never execute. Also removed weight variable because it is unused.
Change-Id: I4f028df29bfde91167fb15befa99b1fe1892adc6
BUG: 789278
Signed-off-by: Girjesh Rajoria <grajoria@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to generate statedumps per glusterfs_ctx_t, it is needed to
place all the memory pools in a structure that the context can reach.
The 'struct mem_pool' has been extended with a 'list_head owner' that is
linked with the glusterfs_ctx_t->mempool_list.
All callers of mem_pool_new() have been updated to pass the current
glusterfs_ctx_t along. This context is needed to add the new memory pool
to the list and for grabbing the ctx->lock while updating the
glusterfs_ctx_t->mempool_list.
Updates: #307
Change-Id: Ia9384424d8d1630ef3efc9d5d523bf739c356c6e
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18075
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
io-cache stores read-cache in inode which is currently created only in
lookup. But, with readdirplus and md-cache absorbing lookups, io-cache
need not receive a lookup before a fop like readv.
Change-Id: I6eba995b0a90d4d5055a4aef0489707b852da1b8
BUG: 1474180
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: https://review.gluster.org/5029
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ioc_inode_wakeup does not lock the ioc_inode for the duration
of the operation, leaving a window where ioc_prune could find
a NULL waitq and hence free the page which ioc_inode_wakeup later
tries to access.
Thanks to Mohit for the analysis.
credit: moagrawa@redhat.com
Change-Id: I54b064857e2694826d0c03b23f8014e3984a3330
BUG: 1456385
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/17410
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The original situation was as follows:
The function that validates xlator options indicating a size,
xlator_option_validate_sizet(), handles the case when the name
of the option is "cache-size" in a special way.
- Xlator options (things of type volume_option_t) has a
min and max attribute of type double.
- An xlator option is endowed with a gluster specific type (not
C type). An instance of an xlator option goes through a validation
process by a type specific validator function (which are collected
in option.c).
- Validators of numeric types - size being one of them - make use the
min and max attributes to perform a range check, except in one case:
if an option is defined with min = max = 0, then this option will be
exempt of range checking. (Note: the volume_option_t definition
features the following comments along the min, max fields:
double min; /* 0 means no range */
double max; /* 0 means no range */
which is slightly misleading as it lets one to conclude that
zeroing min or max buys exemption from low or high boundary check,
which is not true -- only *both* being zero buys exemption.)
- Besides this, the validator for options of size type,
xlator_option_validate_sizet() special cases options
named "cache-size" so that only min is enforced. (The only consequence
of a value exceeding max is that glusterd logs a warning about it, but
the cli user who makes such a setting gets no feedback on it.)
- This was introduced because a hard coded limit is not useful for
io-cache and quick-read. They rather use a runtime calculated
upper limit. (See changes
I7dd4d8c53051b89a293696abf1ee8dc237e39a20
I9c744b5ace10604d5a814e6218ca0d83c796db80
about the last two points.)
- As an unintended consequence, the upper limit check of
cache-size of write-behind, for which a conventional hard coded limit
is specified, is defeated.
What we do about it:
- Remove the special casing clause for cache-size in
xlator_option_validate_sizet. Thus the general range
check policy (as described above) will apply to
cache-size too.
- To implement a lower bound only check by the validator
for cache-size of io-cache and quick-read, change the
max attribute of these options to INFINITY.
The only behavioral difference is the omission of the warnings
about cache-size of io-cache and quick-read exceeding the former max
values. (They were rather heuristic anyway.)
BUG: 1445609
Change-Id: I0bd8bd391fa7d926f76e214a2178833fe4673b4a
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: https://review.gluster.org/17125
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Issue:
=====
In certain cases, there was no unwind of read
from read-ahead xlator, thus resulting in hang.
RCA:
====
In certain cases, ioc_readv() issues STACK_WIND_TAIL() instead
of STACK_WIND(). One such case is when inode_ctx for that file
is not present (can happen if readdirp was called, and populates
md-cache and serves all the lookups from cache).
Consider the following graph:
...
io-cache (parent)
|
readdir-ahead
|
read-ahead
...
Below is the code snippet of ioc_readv calling STACK_WIND_TAIL:
ioc_readv()
{
...
if (!inode_ctx)
STACK_WIND_TAIL (frame, FIRST_CHILD (frame->this),
FIRST_CHILD (frame->this)->fops->readv, fd,
size, offset, flags, xdata);
/* Ideally, this stack_wind should wind to readdir-ahead:readv()
but it winds to read-ahead:readv(). See below for
explaination.
*/
...
}
STACK_WIND_TAIL (frame, obj, fn, ...)
{
frame->this = obj;
/* for the above mentioned graph, frame->this will be readdir-ahead
* frame->this = FIRST_CHILD (frame->this) i.e. readdir-ahead, which
* is as expected
*/
...
THIS = obj;
/* THIS will be read-ahead instead of readdir-ahead!, as obj expands
* to "FIRST_CHILD (frame->this)" and frame->this was pointing
* to readdir-ahead in the previous statement.
*/
...
fn (frame, obj, params);
/* fn will call read-ahead:readv() instead of readdir-ahead:readv()!
* as fn expands to "FIRST_CHILD (frame->this)->fops->readv" and
* frame->this was pointing ro readdir-ahead in the first statement
*/
...
}
Thus, the readdir-ahead's readv() implementation will be skipped, and
ra_readv() will be called with frame->this = "readdir-ahead" and
this = "read-ahead". This can lead to corruption / hang / other problems.
But in this perticular case, when 'frame->this' and 'this' passed
to ra_readv() doesn't match, it causes ra_readv() to call ra_readv()
again!. Thus the logic of read-ahead readv() falls apart and leads to
hang.
Solution:
=========
Ideally, STACK_WIND_TAIL() should be modified as:
STACK_WIND_TAIL (frame, obj, fn, ...)
{
next_xl = obj /* resolve obj as the variables passed in obj macro
can be overwritten in the further instrucions */
next_xl_fn = fn /* resolve fn and store in a tmp variable, before
modifying any variables */
frame->this = next_xl;
...
THIS = next_xl;
...
next_xl_fn (frame, next_xl, params);
...
}
But for this solution, knowing the type of variable 'next_xl_fn' is
a challenge and is not easy. Hence just modifying all the existing
callers to pass "FIRST_CHILD (this)" as obj, instead of
"FIRST_CHILD (frame->this)".
Change-Id: I179ffe3d1f154bc5a1935fd2ee44e912eb0fbb61
BUG: 1388292
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/15901
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Writes are already passing the correct flags at the time of open().
Also, make io-cache honor direct-io for anon-fds with
O_DIRECT flag during reads.
Change-Id: I215cb09ef1b607b9f95cabf0ef3065c00edd9e78
BUG: 1377556
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/15537
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
And minor cleanup of a few of the Makefile.am files while we're
at it.
Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.
Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.
Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.
Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.
Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.
It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.
E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:
...
#include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
...
which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:
...
#include "glusterfs3-xdr.h"
...
Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".
Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/... Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits
Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Revisiting http://review.gluster.org/#/c/11814/, which unintentionally
introduced warnings from libtool about the xlator .so names.
According to [1], the -module option must appear in the Makefile.am
file(s); if -module is defined in a macro, e.g. in configure(.ac),
then libtool will not recognize that this is a module and will emit a
warning.
[1]
http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules
Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078
BUG: 1248669
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/13003
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We've been lucky that we haven't had any symbol collisions until now.
Now we have a collision between the snapview-client's svc_lookup() and
libntirpc's svc_lookup() with nfs-ganesha's FSAL_GLUSTER and libgfapi.
As a short term solution all the snapview-client's FOP methods were
changed to static scope. See http://review.gluster.org/11805. This
works in snapview-client because all the FOP methods are defined in
a single source file. This solution doesn't work for other xlators
with FOP methods defined in multiple source files.
To address this we link with libtool's '-export-symbols $symbol-file'
(a wrapper around `ld --version-script ...` --- on linux anyway) and
only export the minimum required symbols from the xlator sharedlib.
N.B. the libtool man page says that the symbol file should be named
foo.sym, thus the rename of *.exports to *.sym. While foo.exports
worked, we will follow the documentation.
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
BUG: 1248669
Change-Id: I1de68b3e3be58ae690d8bfb2168bfc019983627c
Reviewed-on: http://review.gluster.org/11814
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are three kinds of inline functions: plain inline, extern inline,
and static inline. All three have been removed from .c files, except
those in "contrib" which aren't our problem. Inlines in .h files, which
are overwhelmingly "static inline" already, have generally been left
alone. Over time we should be able to "lower" these into .c files, but
that has to be done in a case-by-case fashion requiring more manual
effort. This part was easy to do automatically without (as far as I can
tell) any ill effect.
In the process, several pieces of dead code were flagged by the
compiler, and were removed.
Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155
BUG: 1245331
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/11769
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I5cc2b4669b164fe09637c86da05d2d94589dd7e4
BUG: 1253149
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/11906
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix the build broken because of patch
http://review.gluster.org/#/c/9822/
Change-Id: I0ee502c0fad5be87186c80ab4729036f52f85fa3
BUG: 1194640
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11451
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
logs to new logging framework.
Change-Id: Ie6aaf8d30bd4457bb73c48e23e6b1dea27598644
BUG: 1194640
Signed-off-by: arao <arao@redhat.com>
Reviewed-on: http://review.gluster.org/9822
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.
Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.
A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.
BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Initially even after calling glfs_fini(), all the threads created
during init and many other resources like memory pool, iobuf pool,
event pool and other memory allocs were not being freed.
With this patch these resources are freed in glfs_fini().
The two thumb rules followed in this patch are:
- The threads are not killed, they are made to exit voluntarily,
once the queued tasks are completed. The main thread waits for
the other threads to exit.
- Free the memory pools and destroy the graphs only after all the
other threads are stopped, so that there are less chances of
hitting access after free.
Resources freed and its order:
1. Destroy the inode table of all the graphs - Call forget on all the inodes.
This will not be required when the cleanup during graph switch is
implemented to perform inode table destroy.
2. Deactivate the current graph, call fini of all the xlators.
3. Syncenv destroy - Join the synctask threads and cleanup syncenv resources
Sets the destroy mode, complete the existing synctasks, then join the
synctask threads.
After entering the destroy mode,
-if a new synctask is submitted, it fails.
-if syncenv_new() is called, it will end up creating new threads,
but this is called only during init.
4. Poller thread destroy
Register an event handler which sets the destroy mode for the poller.
Once the poller is done processing all the events, it exits.
5. Tear down the logging framework
The log file is closed and the log level is set to none, after this
point no log messages appear either in log file or in stderr.
6. Destroy the timer thread
Set the destroy bit, once the pending timer events are processed
the timer thread exits.
Note: Log infrastructure should be shutdown before destroying the timer
thread as gf_log uses timers.
7. Destroy the glusterfs_ctx_t
For all the graphs(active and passive), free graph, xlator structs and few other lists.
Free the memory pools - iobuf pool, event pool, dict, logbuf pool,
stub mem pool, stack mem pool, frame mem pool.
Few things not addressed in this patch:
1. rpc_transport object not destroyed, the PARENT_DOWN should have
destroyed this object but has not, needs to be addressed as a part
of different patch
2. Each xlator fini should clean up the local pool allocated by its xlator.
Needs to be addresses as a part of different patch.
3. Each xlator should implement forget to free its inode_ctx.
Needs to be addresses as a part of different patch.
3. Few other leaks reported by valgrind.
4. fd and fd contexts
The numbers:
The resource usage by the test case in this patch:
Without the fix, Memory: ~3GB; Threads: ~81
With this fix, Memory: 300MB; Threads: 1(main thread)
Change-Id: I96b9277541737aa8372b4e6c9eed380cb871e7c2
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/7642
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs
Working functionality on MacOSX
- GlusterD (management daemon)
- GlusterCLI (management cli)
- GlusterFS FUSE (using OSXFUSE)
- GlusterNFS (without NLM - issues with rpc.statd)
Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac
BUG: 1089172
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Signed-off-by: Dennis Schafroth <dennis@schafroth.com>
Tested-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Dennis Schafroth <dennis@schafroth.com>
Reviewed-on: http://review.gluster.org/7503
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Fix for coverity bug CID:1124625
Change-Id: I76df453a17f2af7c48a80b6fc0ccd411ab96e371
BUG: 789278
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/6949
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ic4276c6d76c36f4eb77282dc06d2b8b212b58f08
BUG: 789278
Signed-off-by: Poornima <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/6822
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I24c10d874511a2f24dda2fb84d31f5074da1616f
BUG: 789278
Signed-off-by: Poornima <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/6869
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During a genuine error condition like network outage, the log grows
with redundant information.
Change-Id: I5a4f2f62da10ef656f14200c4c84a6917b1f0ddd
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
BUG: 1048084
Reviewed-on: http://review.gluster.org/6635
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
.. by UNWINDing ENOMEM, rather than leaving pointer in vector pointing
to stale memory.
Change-Id: I7f3917ac056fae144f845c9d123233e91e278187
BUG: 1034398
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6351
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glfs_zerofill() can be potentially called to zero-out entire file and
hence allow for bigger value of length parameter.
Change-Id: I75f1d11af298915049a3f3a7cb3890a2d72fca63
BUG: 1028673
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-on: http://review.gluster.org/6266
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: M. Mohan Kumar <mohan@in.ibm.com>
Tested-by: M. Mohan Kumar <mohan@in.ibm.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add support for a new ZEROFILL fop. Zerofill writes zeroes to a file in
the specified range. This fop will be useful when a whole file needs to
be initialized with zero (could be useful for zero filled VM disk image
provisioning or during scrubbing of VM disk images).
Client/application can issue this FOP for zeroing out. Gluster server
will zero out required range of bytes ie server offloaded zeroing. In
the absence of this fop, client/application has to repetitively issue
write (zero) fop to the server, which is very inefficient method because
of the overheads involved in RPC calls and acknowledgements.
WRITESAME is a SCSI T10 command that takes a block of data as input and
writes the same data to other blocks and this write is handled
completely within the storage and hence is known as offload . Linux ,now
has support for SCSI WRITESAME command which is exposed to the user in
the form of BLKZEROOUT ioctl. BD Xlator can exploit BLKZEROOUT ioctl to
implement this fop. Thus zeroing out operations can be completely
offloaded to the storage device , making it highly efficient.
The fop takes two arguments offset and size. It zeroes out 'size' number
of bytes in an opened file starting from 'offset' position.
This patch adds zerofill support to the following areas:
- libglusterfs
- io-stats
- performance/md-cache,open-behind
- quota
- cluster/afr,dht,stripe
- rpc/xdr
- protocol/client,server
- io-threads
- marker
- storage/posix
- libgfapi
Client applications can exloit this fop by using glfs_zerofill introduced in
libgfapi.FUSE support to this fop has not been added as there is no system call
for this fop.
Changes from previous version 3:
* Removed redundant memory failure log messages
Changes from previous version 2:
* Rebased and fixed build error
Changes from previous version 1:
* Rebased for latest master
TODO :
* Add zerofill support to trace xlator
* Expose zerofill capability as part of gluster volume info
Here is a performance comparison of server offloaded zeofill vs zeroing
out using repeated writes.
[root@llmvm02 remote]# time ./offloaded aakash-test log 20
real 3m34.155s
user 0m0.018s
sys 0m0.040s
[root@llmvm02 remote]# time ./manually aakash-test log 20
real 4m23.043s
user 0m2.197s
sys 0m14.457s
[root@llmvm02 remote]# time ./offloaded aakash-test log 25;
real 4m28.363s
user 0m0.021s
sys 0m0.025s
[root@llmvm02 remote]# time ./manually aakash-test log 25
real 5m34.278s
user 0m2.957s
sys 0m18.808s
The argument log is a file which we want to set for logging purpose and
the third argument is size in GB .
As we can see there is a performance improvement of around 20% with this
fop.
Change-Id: I081159f5f7edde0ddb78169fb4c21c776ec91a18
BUG: 1028673
Signed-off-by: Aakash Lal Das <aakash@linux.vnet.ibm.com>
Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com>
Reviewed-on: http://review.gluster.org/5327
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|