| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
| |
Have removed the deadcode found by the coverity id:1356503
Change-Id: Ieaa41e864538fb82dc967b4a214d4db09e267098
Updates: #1060
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
|
|
|
|
|
|
|
|
|
| |
All these checks are done after analyzing clang-scan report produced
by the CI job @ https://build.gluster.org/job/clang-scan
updates: bz#1622665
Change-Id: I590305af4ceb779be952974b2a36066ffc4865ca
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
| |
After basic analysis, found that these methods were not being
used at all.
updates: bz#1693692
Change-Id: If9cfa1ab189e6e7b56230c4e1d8e11f9694a9a65
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
| |
also make minor changes for signature (int -> void) where return value
was not checked anywhere.
updates: bz#1693692
Change-Id: Iff117712eb65e0b6b8b441a779202a117fcdf1fb
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.
Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs
This change although big, is just moving around the headers and
making it correct when including these headers from other sources.
This helps us correctly include libglusterfs includes without
namespace conflicts.
Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In a previous patch (https://review.gluster.org/20769) we've
added the key length to be passed to dict_* funcs, to remove the need
to strlen() it. This patch moves some code to use it.
Please review carefully.
Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: If4f425a9827be7c36ccfbb9761006ae824a818c6
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes CIDs: 1388779,1390242,1390057
The patch does the following:
1. Changes the prototype of mq_inspect_file_xattr() and mq_xattr_state()
to have a parameter of struct iatt * instead of struct iatt.
2. Remove unneeded struct iatt parameter from
mq_inspect_directory_xattr()
Change-Id: I1f6eabda042f0fe56d8c9bca010302f5e518dbe4
updates: bz#789278
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
|
|
| |
Updates: bz#1193929
Change-Id: I549f741fbf97ee3fa2d63edacee5b705ef37c49a
Signed-off-by: sanoj-unnikrishnan <sunnikri@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
xlators/storage/posix/src/posix-inode-fd-ops.c:
xlators/storage/posix/src/posix-helpers.c:
xlators/storage/bd/src/bd.c:
xlators/protocol/client/src/client-lk.c:
xlators/performance/quick-read/src/quick-read.c:
xlators/performance/io-cache/src/page.c
xlators/nfs/server/src/nfs3-helpers.c
xlators/nfs/server/src/nfs-fops.c
xlators/nfs/server/src/mount3udp_svc.c
xlators/nfs/server/src/mount3.c
xlators/mount/fuse/src/fuse-helpers.c
xlators/mount/fuse/src/fuse-bridge.c
xlators/mgmt/glusterd/src/glusterd-utils.c
xlators/mgmt/glusterd/src/glusterd-syncop.h
xlators/mgmt/glusterd/src/glusterd-snapshot.c
xlators/mgmt/glusterd/src/glusterd-rpc-ops.c
xlators/mgmt/glusterd/src/glusterd-replace-brick.c
xlators/mgmt/glusterd/src/glusterd-op-sm.c
xlators/mgmt/glusterd/src/glusterd-mgmt.c
xlators/meta/src/subvolumes-dir.c
xlators/meta/src/graph-dir.c
xlators/features/trash/src/trash.c
xlators/features/shard/src/shard.h
xlators/features/shard/src/shard.c
xlators/features/marker/src/marker-quota.c
xlators/features/locks/src/common.c
xlators/features/leases/src/leases-internal.c
xlators/features/gfid-access/src/gfid-access.c
xlators/features/cloudsync/src/cloudsync-plugins/src/cloudsyncs3/src/libcloudsyncs3.c
xlators/features/bit-rot/src/bitd/bit-rot.c
xlators/features/bit-rot/src/bitd/bit-rot-scrub.c
bxlators/encryption/crypt/src/metadata.c
xlators/encryption/crypt/src/crypt.c
xlators/performance/md-cache/src/md-cache.c:
Move to GF_MALLOC() instead of GF_CALLOC() when possible
It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.
In some cases, also changed allocation size to be sizeof some
struct or type instead of a pointer - easier to read.
In some cases, removed redundant strlen() calls by saving the result
into a variable.
1. Only done for the straightforward cases. There's room for improvement.
2. Please review carefully, especially for string allocation, with the
terminating NULL string.
Only compile-tested!
.. and allocate memory as much as needed.
xlators/nfs/server/src/mount3.c :
Don't blindly allocate PATH_MAX, but strlen() the string and allocate
appropriately.
Also, align error messges.
updates: bz#1193929
Original-Author: Yaniv Kaul <ykaul@redhat.com>
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ibda6f33dd180b7f7694f20a12af1e9576fe197f5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
strncpy may not be very efficient for short strings copied into
a large buffer: If the length of src is less than n,
strncpy() writes additional null bytes to dest to ensure
that a total of n bytes are written.
Instead, use snprintf(). Check for truncated output
where applicable.
Also:
- save the result of strlen() and re-use it when possible.
- move from strlen to SLEN (sizeof() ) for const strings.
Compile-tested only!
Change-Id: I54e80d4f4a80e98d3775e376efe05c51af0b29eb
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
| |
Fixes CID: 1325591 1356503 1325540 1351704
Change-Id: I325b1184baa81402eaa009cb36ab95e5de812f6c
updates: bz#789278
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Coverity false positive.
Solution:
Added a comment to ignore the false positive.
CID: 1325591
BUG: 789278
Change-Id: I95037efd12c059efcfc04f3c4c13f60c530150b4
Signed-off-by: Shwetha Acharya <shwetha174@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...
Only compile-tested!
Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Currently it is not possible to capture the xattrs values which
are set on the bricks by calling syncop_(f)xattrop, because the
response dict is not being assigned to any of the dictionaries.
Fix:
In the xattrop callback capture the response dict and send it
back to the caller if it is requested.
Change-Id: I9de9bcd97d6008091c9b060bcca3676cb9ae8ef9
fixes: bz#1572076
Signed-off-by: karthik-us <ksubrahm@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
0 0x00007f1482f1f1d7 in raise () from /lib64/libc.so.6
1 0x00007f1482f208c8 in abort () from /lib64/libc.so.6
2 0x00007f1482f18146 in __assert_fail_base () from /lib64/libc.so.6
3 0x00007f1482f181f2 in __assert_fail () from /lib64/libc.so.6
4 0x00007f148484986a in __inode_link (inode=inode@entry=0x7f14742404d4,
parent=parent@entry=0x7f14742404d4,
name=name@entry=0x7f1460001c48 "testfile5308",
iatt=iatt@entry=0x7f1460001bc8) at inode.c:954
5 0x00007f1484849969 in inode_link (inode=0x7f14742404d4,
parent=parent@entry=0x7f14742404d4,
name=name@entry=0x7f1460001c48 "testfile5308",
iatt=iatt@entry=0x7f1460001bc8) at inode.c:1060
6 0x00007f147591b895 in quota_build_ancestry_cbk (
frame=frame@entry=0x7f1482315e80, cookie=<optimized out>,
this=0x7f147000e910, op_ret=op_ret@entry=6904, op_errno=op_errno@entry=0,
entries=entries@entry=0x7f1474731c00, xdata=xdata@entry=0x0) at quota.c:779
7 0x00007f1475b2f505 in marker_build_ancestry_cbk (frame=0x7f1482315988,
cookie=<optimized out>, this=<optimized out>, op_ret=<optimized out>,
op_errno=<optimized out>, entries=0x7f1474731c00, xdata=0x0)
at marker.c:3055
8 0x00007f14848b9cd9 in default_readdirp_cbk (
frame=frame@entry=0x7f1482315b30, cookie=<optimized out>,
this=<optimized out>, op_ret=op_ret@entry=6904, op_errno=op_errno@entry=0,
entries=entries@entry=0x7f1474731c00, xdata=xdata@entry=0x0)
at defaults.c:1403
9 0x00007f1475f68132 in pl_readdirp_cbk (frame=0x7f1482315dac,
cookie=<optimized out>, this=<optimized out>, op_ret=6904, op_errno=0,
entries=0x7f1474731c00, xdata=0x0) at posix.c:2700
10 0x00007f1476e26819 in posix_readdirp (frame=0x7f1482315f54,
this=<optimized out>, fd=<optimized out>, size=<optimized out>,
off=<optimized out>, dict=<optimized out>) at posix.c:6282
11 0x00007f1475f6599a in pl_readdirp (frame=0x7f1482315dac,
this=0x7f147000a200, fd=0x7f1484b5106c, size=0, offset=0,
xdata=0x7f1481ab4f34) at posix.c:2711
12 0x00007f14848ce954 in default_readdirp_resume (frame=0x7f1482315b30,
this=0x7f147000b690, fd=0x7f1484b5106c, size=0, off=0,
xdata=0x7f1481ab4f34) at defaults.c:2019
13 0x00007f148485c92d in call_resume (stub=0x7f1481b65710) at call-stub.c:2508
14 0x00007f1475d54743 in iot_worker (data=0x7f147004e7d0) at io-threads.c:210
15 0x00007f148369cdc5 in start_thread () from /lib64/libpthread.so.0
16 0x00007f1482fe173d in clone () from /lib64/libc.so.6
Change-Id: I740dc691e7be1bc2a9ae3a0cb14bbf566ea77bc5
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
Reviewed-on: https://review.gluster.org/17730
Tested-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In function mq_update_dirty_inode_task we do readdirp
on a dirty directory and for entry we again do
lookup to fecth the contribution xattr.
We can fetch this contribution as part of readdirp
Change-Id: I766593c0dba793f1ab3b43625acce1c7d9af8d7f
BUG: 1320818
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/13892
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* If a "rm -rf" is performed by a client, we initiate a
marker background operation mq_reduce_parent_size_txn
for rmdir and unlink.
mq_reduce_parent_size_txn can fail when updating
size on the ancestor directories, if these directories
are removed during the txn as the child-parent association
removed in the dentry list.
So execute mq_reduce_parent_size_txn in foreground
and then do the UNWIND for rmdir and unlink FOP
Change-Id: Iefcdced4c6ae0dbd43f92814d0ddcd1e33825864
BUG: 1322489
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/13874
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* quota-enforcer doesn't execute build_ancestry in the below
code path
1) Special client (PID < 0)
2) unlink
3) rename within the same directory
4) link within the same directory
In these cases, marker accounting can fail as parent not found.
We need to build_ancestry in marker if it doesn't find parent
during update txn
Change-Id: Idb7a2906500647baa6d183ba859b15e34769029c
BUG: 1320818
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/13857
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a file is unlinked before it was accounted by the marker,
then we may see error "ctx for the node ... is NULL" at many places.
This is actually not an error and can be ignored.
It is better to set the inode ctx before lookup/create is unwind
back to protocol server
Change-Id: I462b5542951f2fc2964f59af7a31978979dab1de
BUG: 1318158
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/13748
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With unlink, rename, rmdir, contribution xattrs
are removed. If the file is a last link
then remove_xattr will fail with ENOENT.
So it better to perform remove_xattr
only if there are more links to the file
Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d
BUG: 1257694
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12033
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a quota is disable and the clean-up process terminated
without completely cleaning-up the quota xattrs.
Now when quota is enabled again, this can mess-up the accounting
A version number is suffixed for all quota xattrs and this version
number is specific to marker xaltor, i.e when quota xattrs are
requested by quotad/client marker will remove the version suffix in the
key before sending the response
Change-Id: I1ca2c11460645edba0f6b68db70d476d8d26e1eb
BUG: 1272411
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12386
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Consider below scenario:
Quota enabled on pre-existing data
Now quota-crawl process will start healing xattrs
Now if write is performed where healing is not complete, there is a
possibility that 'update txn' is started before 'create xattr txn', in
this case dir count can be missed on a dir where quota size xattr is not
yet created.
Solution is to get size xattr and if xattr is missing, add 1 for
dir_count, this requires one additional fop if done in marker during
each update iteration
Better solution is to us xattrop GF_XATTROP_ADD_ARRAY64_WITH_DEFAULT
Change-Id: Idc8978860a3914e70c98f96effeff52e9a24e6ba
BUG: 1243798
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11694
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
marker is re-factored with syncop approach,
remove unused old code
Change-Id: I36e670e63b6c166db5e64d3149d2978981e2f7c2
BUG: 1240581
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11560
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During a DHT re-balance operation, quota accounts for the destination.
Problem of accounting this destination file are:
1) Migration is an internal operation, 'quota list' shows more usage on the CLI
and this will come to the normal numbers once the migration is complete
2) If the usage is close to the limit set, then we can get 'Disk Quota
Exceeded' errors in the I/O path during file migration
Solution is we should not account of the usage on the destination file
during migration, at the end of the migration. We need to reduce size of
the source directory and accounting for the migrated dest file
We assume that there are sufficent disk space in the back-end. DHT
migrator should make sure that there are sufficient disk space before it
starts the migration process.
Change-Id: Ie3cfe3e4ab5241c2a127ba0edc599a053d30c3a0
BUG: 1260545
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12113
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: Dan Lambright <dlambrig@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In case if dir has become dirty because of brick crash,
this dirty flag on disk was getting reset in the
next update txn.
This patch now gets the dirty flag before setting the flag
in the update txn and if this value is dirty, it keeps
the flag dirty, so that inspect_directory can fix the
dirty dir
Change-Id: Iab2c343dbe19bd3b291adbfeebe6d9785b6bb9e3
BUG: 1251454
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/12032
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- fd_unref should decrement fd->inode->fd_count only if it is present in the
inode's fd list.
- successful open/opendir should perform fd_bind.
Change-Id: I81dd04f330e2fee86369a6dc7147af44f3d49169
BUG: 1207735
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/11044
Reviewed-by: Anoop C S <anoopcs@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch does the following
1) Set loc.parent if it is NULL
Don't log warning in txn if parent is NULL
2) Don't initiate txn when inode gfid is NULL
3) optimize invoking dirty txn with status flag
Change-Id: I67dd9e6268014b0b257c136e951e6ded0a2e911f
BUG: 1251454
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11863
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are three problems with marker-rename which
is fixed in this patch
Problem 1)
1) mq_reduce_parent_size is not handling inode-quota contribution
2) When dest files exists and IO is happening
Now renaming will overwrite existing file
mq_reduce_parent_size called on dest file
with saved contribution, this can be
a problem is IO is still happening
contribution might have changed
Problem 2)
There is a small race between rename and in-progress write
Consider below scenario
1) rename FOP invoked on file 'x'
2) write is still in progress for file 'x'
3) rename takes a lock on old-parent
4) write-update txn blocked on old-parent to acquire lock
5) in rename_cbk, contri xattrs are removed and contribution is deleted and
lock is released
6) now write-update txn gets the lock and updates the wrong parent
as it was holding lock on old parent
so validate parent once the lock is acquired
Problem 3)
when a rename operation is performed, a lock is
held on old parent. This lock is release before
unwinding the rename operation.
This can be a problem if there are in-progress
writes happening during rename, where update txn
can take a lock and update the old parent
as inode table is not updated with new parent
Change-Id: Ic3316097c001c33533f98592e8fcf234b1ee2aa2
BUG: 1240991
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11578
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For all marker txn, it is possible that while the txn in progress
file can be removed.
So log all ENOENT/ESTALE error in debug level
Change-Id: I776ba3cda33fe992f9dcc7baede7206c4ce19553
BUG: 1245544
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11736
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dict_set_bin() is handling the pointer that it passed inconsistently.
Depending on the errors that can occur, the pointer passed to the dict
can be free'd, but there is no guarantee.
It is cleaner to have the caller free the pointer that allocated it and
dict_set_bin() returned an error. When dict_set_bin() returned success,
the given pointer will be free'd when dict_unref() calls data_destroy().
Many callers of dict_set_bin() already take care of free'ing the pointer
on error. The ones that did not, are corrected with this change too.
Change-Id: I39a4f7ebc0cae6d403baba99307d7ce408f25966
BUG: 1242280
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11638
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a possibility that contribution is removed twice from list
during unlink operation (with hard links) or during rename operation
Use list_del_init for a thread safe deltion of member from list
Change-Id: Iff5e0c03cc8f0ed85da0db1739b84b695abf9ea6
BUG: 1244109
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11706
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During quota-update process if inode info is present in size-xattr and
missing in contri-xattrs, then in function '_mq_get_metadata', we set
contri-size as zero (on error -2, which means usage info present, but inode info missing).
With this we are calculating wrong delta and updating the same.
With this patch we are ignoring errors if inode info in xattrs are missing
Change-Id: I7940a0e299b8bb425b5b43746b1f13f775c7fb92
BUG: 1241153
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11583
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Default stacksize that synctask uses is 2M.
For marker we set it to 16k
Also move market xlator close to io-threads
to have smaller stack
Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11499
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem-1)
Now the marker accounting happens in background,
There is a possibility that before completing
create_xattr_txn another create txn can be initiated
for the same inode.
suppose if few 100 txns are initiated
before completion, this can block all synctask threads
waiting on a lock and this can also consume lot of memory
and can take more time to complete the background accounting
operation.
This patch improves the locking mechanism which
can improve the performance as well reduce memory
consumption
Problem-2)
For every lookup and for all inodes in readdirp
we were initiating a new txn, this can result
in more txn pending in synctask queue and
lead to huge memory consumption. inspect
file/dir should start a txn only if there
is some delta
Problem-3)
When there are multiple write operations on
same inode and all the synctask threads are busy.
As we are checking for updation_status
flag in background, all txn will be move to synctask queue.
This can increase the mem usage.
Only one txn for inode in a queue will be sufficient,
so check and set updation falg before moving txn to
background
Change-Id: Ic42ce00f0a50ce51c7128ba68a1b6a0699a1cd14
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11457
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With below test-case, marker accounting becomes bad:
1) Create a volume with 1 brick
2) fuse mount
3) on one terminal write some data
dd if=/dev/zero of=f1 bs=1M count=500 oflag=sync
4) on another terminal execute below rename operation while the write is
still in progress
for i in {1..50}; do
ii=`expr $i + 1`;
mv f$i f$ii;
done
remove-xattr is already on while doing rename operation,
we should not be doing again in background when reducing the
parent size
Change-Id: I969a64bb559e2341315928b55b99203e9ddee3f2
BUG: 1235195
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11403
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When removing contribution xattr, we also need to free
contribution node in memory.
Use ref/unref mechanism to handle contribution node memory
local->xdata should be freed in mq_local_unref
There is another huge memory consumption happens
in function mq_inspect_directory_xattr_task
where dirty flag is not set.
Change-Id: Ieca3ab4bf410c51259560e778bce4e81b9d888bf
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11361
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit 8ab6608accb62d6320d1fc1fbe651fcafd376270.
This patch is resulting in memory corruption:
http://build.gluster.org/job/rackspace-regression-2GB-triggered/11193/consoleFull
contribution object might be being used by some other transaction when
we free it. The correct way to handle this is to have a reference based
scheme to manage the contribution object.
Change-Id: Idf9993ed8268029073a3e2d699865587f20d9aea
BUG: 1207735
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/11362
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
When removing contribution xattr, we also need to free
contribution node in memory
Change-Id: I5fe97813a8f39e2f00401976046bd280f2eea54d
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11311
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a rename operation is performed, we are renaming
the file first and performing remove-xattr when reducing
the contri size from parents.
This remove-xattr fails as the file is alreday renamed,
this failure causes reduce-parent-size to abort resulting
in double quota accounting
This patch fixes the problem. We don't need to perform remove-xattr
operation on a file when performing reduce-parent-size txn as this
will be alreday done before starting reduce-parent-size txn
Change-Id: If86e3dbb0233f6deaaa90bee72cb0ec1689c7325
BUG: 1232572
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11264
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Missing loc_wipe() for error paths in mq_readdir_cbk() can
cause memory leaks. loc_wipe() is now done for both happy
and unhappy paths.
Change-Id: I882aa5dcca06e25b56a828767fb2b91a1efaf83b
BUG: 1227904
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/11074
Reviewed-by: Sachin Pandit <spandit@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
inode quota is a new feature implemented in glusterfs-3.7
if quota is enabled in the older version and is upgraded
to a new version, we can hit setxattr spike during self-heal
of inode quotas. So, when a quota is enabled, turn off
inode-quotas with a xlator option.
With this patch, we still account for inode quotas but only
when a write operation is performed for a particular file.
User will be able to query inode quotas once the Inode-quota
xlator option is enabled.
Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a
BUG: 1209430
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10152
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a problem during upgrade where, inode quotas are not healed in
the contri xattrs.
Healing happens if contri xattrs are missing.
But healing doesn't happen if contri xattrs are present and inode quota
values are missing in the contri xattrs.
This patch fixes the problem
Change-Id: I6c88b74b5bb333a97c5419e24cc4ada82839f474
BUG: 1211808
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/10239
Tested-by: NetBSD Build System
Reviewed-by: Sachin Pandit <spandit@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch adds support for xdata in both the
request and response path of syncops.
Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.
xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.
There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.
xdata support at gfapi level would be introduced
in subsequent patches.
Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.
Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.
A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.
BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CIDs:
1124601
1124690
1134008
1134009
Change-Id: I93992b11bb6d8f7edd065f602aec2cd7a8b433d0
BUG: 789278
Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com>
Reviewed-on: http://review.gluster.org/9614
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
==========================================================================
Inode quota
==========================================================================
= Currently, the only way to retrieve the number of files/objects in a =
= directory or volume is to do a crawl of the entire directory/volume. =
= This is expensive and is not scalable. =
= =
= The proposed mechanism will provide an easier alternative to determine =
= the count of files/objects in a directory or volume. =
= =
= The new mechanism proposes to store count of objects/files as part of =
= an extended attribute of a directory. Each directory's extended =
= attribute value will indicate the number of files/objects present =
= in a tree with the directory being considered as the root of the tree. =
= =
= The count value can be accessed by performing a getxattr(). =
= Cluster translators like afr, dht and stripe will perform aggregation =
= of count values from various bricks when getxattr() happens on the key =
= associated with file/object count. =
A new interface is introduced:
------------------------------
limit-objects : limit the number of inodes at directory level
list-objects : list the directories where the limit is set
remove-objects : remove the limit from the directory
==========================================================================
CLI COMMAND:
gluster volume quota <volname> limit-objects <path> <number> [<percent>]
* <number> is a hard-limit for number of objects limitation for path "<path>"
If hard-limit is exceeded, creation of file/directory is no longer
permitted.
* <percent> is a soft-limit for number of objects creation for path "<path>"
If soft-limit is exceeded, a warning is issued for each creation.
CLI COMMAND:
gluster volume quota <volname> remove-objects [path]
==========================================================================
CLI COMMAND:
gluster volume quota <volname> list-objects [path] ...
Sample output:
------------------
Path Hard-limit Soft-limit Used Available
Soft-limit exceeded?
Hard-limit exceeded?
------------------------------------------------------------------------
--------------------------------------
/dir 10 80% 10 0
Yes
Yes
==========================================================================
[root@snapshot-28 dir]# ls
a b file11 file12 file13 file14 file15 file16 file17
[root@snapshot-28 dir]# touch a1
touch: cannot touch `a1': Disk quota exceeded
* Nine files are created in directory "dir" and directory is included in
* the
count too. Hence the limit "10" is reached and further file creation
fails
==========================================================================
Note: We have also done some re-factoring in cli for volume name
validation. New function cli_validate_volname is created
==========================================================================
Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b
BUG: 1190108
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9769
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
marker-quota.c: In function 'mq_inspect_directory_xattr_task':
marker-quota.c:3451:31: warning: variable 'buf' set but not
used [-Wunused-but-set-variable]
struct iatt buf = {0,};
Change-Id: I211378328bdb2509a5d2a186d173f7f30a670c8a
BUG: 1198849
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/9928
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, the only way to retrieve the number of files/objects in a
directory or volume is to do a crawl of the entire directory/volume.
This is expensive and is not scalable.
The new mechanism proposes to store count of objects/files as part of
an extended attribute of a directory. Each directory's extended
attribute value will indicate the number of files/objects present
in a tree with the directory being considered as the root of the tree.
Currently file usage is accounted in marker by doing multiple FOPs
like setting and getting xattrs. Doing this with STACK WIND and
UNWIND can be harder to debug as involves multiple callbacks.
In this code we are replacing current mechanism with syncop approach
as syncop code is much simpler to follow and help us implement inode
quota in an organized way.
Change-Id: Ibf366fbe07037284e89a241ddaff7750fc8771b4
BUG: 1188636
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/9567
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|