| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: commit c34e4161f3cb6539ec83a9020f3d27eb4759a975 set log-level
per xlator during reconfigure only for a brick process not for
the client process.
Solution: 1) Change per xlator log-level only if brick_mux is enabled.To make sure
about brick multiplex introudce a flag brick_mux at ctx->cmd_args.
Note: There are two other changes done with this patch
1) Ignore client-log-level option to attach a brick with
already running brick if brick_mux is enabled
2) Add a log to print pid of the running process to make easier
debugging
> Change-Id: I39e85de778e150d0685cd9a79425ce8b4783f9c9
> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
> Fixes: bz#1696046
> (Cherry picked from commit 798aadbe51a9a02dd98a0f861cc239ecf7c8ed57)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22495/)
Change-Id: If91682830f894ab8f6857f19dcb1797fc15ca64c
Fixes: bz#1699715
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
| |
|
|
|
|
| |
updates bz#1650403
Change-Id: Ib5a11e691599ce4bd93c1ed5aca6060592893961
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove the options to load old symbol.
* keep only 'xlator_api' symbol from being exported using xlator.sym
* add xlator_api to all the xlators where its missing
NOTE: This covers all the xlators which has at least a test case
to validate its loading. If there is a translator, which doesn't
have any test, then we should probably remove that from codebase.
fixes: #164
Change-Id: Ibcdc8c9844cda6b4463d907a15813745d14c1ebb
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* libglusterfs changes to add new fop
* Fuse changes:
- Changes in fuse bridge xlator to receive and send responses
* posix changes to perform the op on the backend filesystem
* protocol and rpc changes for sending and receiving the fop
* gfapi changes for performing the fop
* tools: glfs-copy-file-range tool for testing copy_file_range fop
- Although, copy_file_range support has been added to the upstream
fuse kernel module, no release has been made yet of a kernel
which contains the support. It is expected to come in the
upcoming release of linux-4.20
So, as of now, executing copy_file_range fop on a fused based
filesystem results in fuse kernel module sending read on the
source fd and write on the destination fd.
Therefore a small gfapi based tool has been written to be able
test the copy_file_range fop. This tool is similar (in functionality)
to the example program given in copy_file_range man page.
So, running regular copy_file_range on a fuse mount point and
running gfapi based glfs-copy-file-range tool gives some idea about
how fast, the copy_file_range (or reflink) can be.
On the local machine this was the result obtained.
mount -t glusterfs workstation:new /mnt/glusterfs
[root@workstation ~]# cd /mnt/glusterfs/
[root@workstation glusterfs]# ls
file
[root@workstation glusterfs]# cd
[root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new
real 0m6.495s
user 0m0.000s
sys 0m1.439s
[root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr
OPEN_SRC: opening /file is success
OPEN_DST: opening /rrr is success
FSTAT_SRC: fstat on /rrr is success
copy_file_range successful
real 0m0.309s
user 0m0.039s
sys 0m0.017s
This tool needs following arguments
1) hostname
2) volume name
3) log file path
4) source file path (relative to the gluster volume root)
5) destination file path (relative to the gluster volume root)
"glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>"
- Added a testcase as well to run glfs-copy-file-range tool
* io-stats changes to capture the fop for profiling
* NOTE:
- Added conditional check to see whether the copy_file_range syscall
is available or not. If not, then return ENOSYS.
- Added conditional check for kernel minor version in fuse_kernel.h
and fuse-bridge while referring to copy_file_range. And the kernel
minor version is kept as it is. i.e. 24. Increment it in future
when there is a kernel release which contains the support for
copy_file_range fop in fuse kernel module.
* The document which contains a writeup on this enhancement can be found at
https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit
Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367
updates: #536
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
|
| |
|
|
|
|
| |
Fixes: #164
Change-Id: I93ad6f0232a1dc534df099059f69951e1339086f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.
Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs
This change although big, is just moving around the headers and
making it correct when including these headers from other sources.
This helps us correctly include libglusterfs includes without
namespace conflicts.
Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
| |
|
|
|
|
|
|
|
|
| |
Summary: The json being output by the io-stats debug xlator
quotes the numbers. This is not necessary and makes parsing
in strongly typed languages more difficult.
Change-Id: I3ac13700e2c52dbdc29d0bcdd39896d7871f36fe
fixes: bz#1654521
Signed-off-by: Chris Holcombe <xfactor973@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The issue pertains to checking if conf is NULL in BUMP_FOP
but not providing that safety in io_stats_release when using
conf to lock and bump nr_opens.
This is now corrected to check if conf is non-NULL before
attempting the lock and bump nr_opens.
Tested with local clang analyzer to ensure this fixes the
problem.
Change-Id: Iffd6a97c2060d0a6930a8dc5914b1956c192cab1
Updates: bz#1622665
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, there are possibilities in few places, where a user-controlled
(like filename, program parameter etc) string can be passed as 'fmt' for
printf(), which can lead to segfault, if the user's string contains '%s',
'%d' in it.
While fixing it, makes sense to make the explicit check for such issues
across the codebase, by making the format call properly.
Fixes: CVE-2018-14661
Fixes: bz#1644763
Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
By allowing clients taking dump in a file on brick process, we are
allowing compromised clients to create io-stats dumps on server,
which can exhaust all the available inodes.
Fixes: CVE-2018-14659
Fixes: bz#1644757
Change-Id: I32bfde9d4fe646d819a45e627805b928cae2e1ca
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: In brick_mux environment, while a user has enabled brick-log-level
for anyone volume, it automatically enables for other volumes
also those are attached with same brick.
Solution: A log-level option is automatically enabled for other volumes
because log-level saved in glusterfsd_ctx and ctx is common for
volumes those are attached with same brick. To resolve it
set log level for all children xlator's at the time of the graph
reconfigure at io-stat xlator.
Change-Id: Id9a6efa05d286e0bea2d47f49292d084e7bb2fcf
fixes: bz#1640495
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As dict_unserialize does not null terminate the value,
using snprintf adds garbage characters to the buffer
used to create the filename.
The code also used this->name in the filename which
will be the same for all bricks for a volume. The
files were thus overwritten if a node contained
multiple bricks for a volume. The code now uses
the conf->unique instead if available.
Change-Id: I2c72534b32634b87961d3b3f7d53c5f2ca2c068c
fixes: bz#1640165
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
While it may slightly reduce accuracy, I think it's better to
acquire the time outside a lock and then memcpy the value while
under lock.
Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ic8fb94a56c4fa2d3b13e59767e13646fb39342ba
|
| |
|
|
|
|
|
|
|
|
| |
When compiling in other architectures there appear many warnings. Some
of them are actual problems that prevent gluster to work correctly on
those architectures.
Change-Id: Icdc7107a2bc2da662903c51910beddb84bdf03c0
fixes: bz#1632717
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
| |
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
| |
|
|
| |
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It wouldn't make sense to allow iostats file to be written in
*any* directory. While the formating makes sure we try to append
io-stats-name for the file, so overwriting existing file is slim,
but in any case it makes sense to restrict dumping to one directory.
Below are the sample commands, and files created for the corresponding
values:
$ setfattr -n trusted.io-stats-dump -v file-for-dump $M0
In this case, the file would be in /var/run/gluster/file-for-dump
$ setfattr -n trusted.io-stats-dump -v /dir1/dir2/file-for-dump $M0
In this case, then the dump file is in /var/run/gluster/dir1-dir2-file-for-dump
Note that the value passed for this virtual xattr would be treated as a
file, and even if the value has '/' in it, it would be changed to '-'
for sanity.
Fixes: bz#1625106
Change-Id: Id9ae6a40a190b8937c51662e6e1c2a0f6c86a0e0
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
| |
|
|
|
|
|
|
| |
Addresses CID: 1124492
Change-Id: I92f807b5e45537e68c318bcbf54ccaa23d18b0dd
Updates: bz#789278
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
|
| |
|
|
|
|
|
|
| |
Addresses CID: 1124492, 1124700, 1124701
Change-Id: If9916df1b0a4e67ad74fe0f6ea6f2544598ddacb
updates: bz#789278
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
It's not needed.
There's a good chance the compiler is smart enough to remove it
anyway, but it can't hurt - I hope.
Compile-tested only!
Change-Id: Id7c054e146ba630227affa591007803f3046416b
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
| |
|
|
|
|
|
|
| |
s/coverty/coverity/
Change-Id: Iac7c13176162eace4247dd3236373aa76d906380
updates: bz#789278
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...
Only compile-tested!
Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
This patch fixes compile warnings that appear with newer compilers. The
solution applied is only to remove the warnings, but it doesn't always
solve the problem in the best way. It assumes that the problem will never
happen, as the previous code assumed.
Change-Id: I6e8470d6c2e2dbd3bd7d324b5fd2f92ffdc3d6ec
updates: bz#1193929
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
_ios_dump_thread is not terminated by the function
_ios_destroy_dump_thread when the diagnostic interval is set
to 0 (which means disable auto dumping).
During reconfigure, if the value changes from 0 to another
then the thread is started, but on reconfiguring this to 0
the thread is not being terminated.
Further, if the value is changed from 0 to X to 0 to Y, where
X and Y are 2 arbitrary duration numbers, the reconfigure
code ends up starting one more thread (for each change from
0 to a valid interval).
This patch fixes the same by terminating the thread when the
value changes from non-zero to 0.
NOTE: It would seem nicer to use conf->dump_thread and check
its value for thread presence etc. but there is no documented
invalid value for the same, and hence an invalid check is not
feasible, thus introducing a new running bool to determine the
same.
Fixes: bz#1598548
Change-Id: I3e7d2ce8f033879542932ac730d57bfcaf14af73
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: At the time of stopping the volume while brick multiplex is
enabled memory is not cleanup from all server side xlators.
Solution: To cleanup memory for all server side xlators call fini
in glusterfs_handle_terminate after send GF_EVENT_CLEANUP
notification to top xlator.
BUG: 1544090
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Note: Run all test-cases in separate build (https://review.gluster.org/19574)
with same patch after enable brick mux forcefully, all test cases are
passed.
Change-Id: Ia10dc7f2605aa50f2b90b3fe4eb380ba9299e2fc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following release-3.8-fb branch patch is upstreamed:
> features/namespace: Add namespace xlator and link into brick graph
> Commit ID: dbd30776f26e
> https://review.gluster.org/#/c/18041/
> By Michael Goulet <mgoulet@fb.com>
Changes in this patch:
Removes extra config.h and namespace.h file in namespace.c
Adds default_getspec_cbk to libglusterfs.sym
Rename dict_for_each to dict_foreach_inline
Remove fd.h header file stack.h
Add test case for truncate, open and symlink
This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: Ib88c95b89eecee9b8957df8a4c8712c899c761d1
Signed-off-by: Varsha Rao <varao@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
There are still remain some code paths where cleanup is required while
brick mux is on.I will upload a new patch after resolve all code paths.
This reverts commit b313d97faa766443a7f8128b6e19f3d2f1b267dd.
BUG: 1544090
Change-Id: I26ef1d29061092bd9a409c8933d5488e968ed90e
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: At the time of stopping the volume while brick multiplex is
enabled memory is not cleanup from all server side xlators.
Solution: To cleanup memory for all server side xlators call fini
in glusterfs_handle_terminate after send GF_EVENT_CLEANUP
notification to top xlator.
BUG: 1544090
Change-Id: Ifa1525e25b697371276158705026b421b4f81140
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following release-3.8-fb branch patch is upstreamed:
> io-stats: Expose io-thread queue depths
> Commit ID: 69509ee7d2
> https://review.gluster.org/#/c/18143/
> By Shreyas Siravara <sshreyas@fb.com>
Changes in this patch:
- Replace iot_pri_t with gf_fop_pri_t
- Replace IOT_PRI_{HI, LO, NORMAL, MAX, LEAST} with
GF_FOP_PRI_{HI, LO, NORMAL, MAX, LEAST}
- Use dict_unref() instead of dict_destroy()
This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: I1b47a63185a441a30fbc423ca1015df7b36c2518
Signed-off-by: Varsha Rao <varao@redhat.com>
|
| |
|
|
|
|
|
| |
Updates #302
Change-Id: I8e7ff391cec88ea76f63ffe05b7404bdb31eaf8e
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
|
| |
|
|
|
|
|
| |
Updates #302
Change-Id: I8d04bb4f46b1e53a8d176a75da2382fdbf0cfc6f
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
|
| |
|
|
|
|
| |
Updates #302
Change-Id: I2c7ab85364337d0bc00428e0001ddc4038e08174
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
v1 of the patch started off as adding new fields to iatt that can be
filled up using statx but the discussions were more around introducing
masks to check the validity of different fields from a RIO perspective.
To that extent, I have dropped the statx call in this version and
introduced a 64 bit mask for existing fields. The masks I have defined
are similar with the statx() flags' masks.
I have *not* changed iatt_to_stat() to use the macros IATT_TYPE_VALID,
IATT_GFID_VALID etc before blindly copying from struct iatt to struct.
Also fixed warnings in xlators because of atime/mtime/ctime seconds
field change from uint32_t to int64_t.
Change-Id: I4ac614f1e8d5c8246fc99d5bc2d2a23e7941512b
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
| |
|
|
|
|
|
|
|
| |
This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.
Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: In glusterfs code base we call mutex_lock/unlock to take
reference/dereference for a object.Sometime it could be
reason for lock contention also.
Solution: There is no need to use mutex to increase/decrease ref
counter, instead of using mutex use gcc builtin ATOMIC
operation.
Test: I have not observed yet how much performance gain after apply
this patch specific to glusterfs but i have tested same
with below small program(mutex and atomic both) and
get good difference.
static int numOuterLoops;
static void *
threadFunc(void *arg)
{
int j;
for (j = 0; j < numOuterLoops; j++) {
__atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL);
}
return NULL;
}
int
main(int argc, char *argv[])
{
int opt, s, j;
int numThreads;
pthread_t *thread;
int verbose;
int64_t n = 0;
if (argc < 2 ) {
printf(" Please provide 2 args Num of threads && Outer Loop\n");
exit (-1);
}
numThreads = atoi(argv[1]);
numOuterLoops = atoi (argv[2]);
if (1) {
printf("\tthreads: %d; outer loops: %d;\n",
numThreads, numOuterLoops);
}
thread = calloc(numThreads, sizeof(pthread_t));
if (thread == NULL) {
printf ("calloc error so exit\n");
exit (-1);
}
__atomic_store (&glob, &n, __ATOMIC_RELEASE);
for (j = 0; j < numThreads; j++) {
s = pthread_create(&thread[j], NULL, threadFunc, NULL);
if (s != 0) {
printf ("pthread_create failed so exit\n");
exit (-1);
}
}
for (j = 0; j < numThreads; j++) {
s = pthread_join(thread[j], NULL);
if (s != 0) {
printf ("pthread_join failed so exit\n");
exit (-1);
}
}
printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED));
exit(0);
}
time ./thr_count 800 800000
threads: 800; outer loops: 800000;
glob value is 640000000
real 1m10.288s
user 0m57.269s
sys 3m31.565s
time ./thr_count_atomic 800 800000
threads: 800; outer loops: 800000;
glob value is 640000000
real 0m20.313s
user 1m20.558s
sys 0m0.028
Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
- Our current approach to measuring "average fop latency" is badly
flawed in that it doesn't weight the FOPs correctly according to how
many occurred in the time interval. This makes Statisticians very
sad. This patch adds an internally computed weighted average
latency which will be far more efficient to display via ODS, as well
as having the benefit of not being complete nonsense.
Reviewers: kvigor, dph, sshreyas
Reviewed By: sshreyas
Change-Id: Ie3618f279b545610b7ed1a8482243fcc8dc53217
BUG: 1523353
Reviewed-on: https://review.gluster.org/18192
Reviewed-by: Shreyas Siravara <sshreyas@fb.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Signed-off-by: Ana M. Neri <amnerip@fb.com>
|
| |
|
|
|
|
|
|
|
| |
specify ctx in gf_log_set_loglevel, instead of getting it from a thread
specific variable.
Change-Id: I498f826e8e32231235a6b0005026a27c327727fd
BUG: 1521213
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
|
| |
|
|
|
|
|
|
|
| |
This reverts commit fc73ae5f81ef5926e3dc2311db116250d0f2a321.
See bug: https://bugzilla.redhat.com/show_bug.cgi?id=1513692
Change-Id: I00d5989b042d4e345621c596d5370d324948557f
Bug: 1513692
Signed-off-by: Chris Holcombe <xfactor973@gmail.com>
|