| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- fd_unref should decrement fd->inode->fd_count only if it is present in the
inode's fd list.
- successful open/opendir should perform fd_bind.
Change-Id: I81dd04f330e2fee86369a6dc7147af44f3d49169
BUG: 1207735
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/11044
Reviewed-by: Anoop C S <anoopcs@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch does the following
1) Set loc.parent if it is NULL
Don't log warning in txn if parent is NULL
2) Don't initiate txn when inode gfid is NULL
3) optimize invoking dirty txn with status flag
Change-Id: I67dd9e6268014b0b257c136e951e6ded0a2e911f
BUG: 1251454
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11863
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are three problems with marker-rename which
is fixed in this patch
Problem 1)
1) mq_reduce_parent_size is not handling inode-quota contribution
2) When dest files exists and IO is happening
Now renaming will overwrite existing file
mq_reduce_parent_size called on dest file
with saved contribution, this can be
a problem is IO is still happening
contribution might have changed
Problem 2)
There is a small race between rename and in-progress write
Consider below scenario
1) rename FOP invoked on file 'x'
2) write is still in progress for file 'x'
3) rename takes a lock on old-parent
4) write-update txn blocked on old-parent to acquire lock
5) in rename_cbk, contri xattrs are removed and contribution is deleted and
lock is released
6) now write-update txn gets the lock and updates the wrong parent
as it was holding lock on old parent
so validate parent once the lock is acquired
Problem 3)
when a rename operation is performed, a lock is
held on old parent. This lock is release before
unwinding the rename operation.
This can be a problem if there are in-progress
writes happening during rename, where update txn
can take a lock and update the old parent
as inode table is not updated with new parent
Change-Id: Ic3316097c001c33533f98592e8fcf234b1ee2aa2
BUG: 1240991
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11578
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For all marker txn, it is possible that while the txn in progress
file can be removed.
So log all ENOENT/ESTALE error in debug level
Change-Id: I776ba3cda33fe992f9dcc7baede7206c4ce19553
BUG: 1245544
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11736
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dict_set_bin() is handling the pointer that it passed inconsistently.
Depending on the errors that can occur, the pointer passed to the dict
can be free'd, but there is no guarantee.
It is cleaner to have the caller free the pointer that allocated it and
dict_set_bin() returned an error. When dict_set_bin() returned success,
the given pointer will be free'd when dict_unref() calls data_destroy().
Many callers of dict_set_bin() already take care of free'ing the pointer
on error. The ones that did not, are corrected with this change too.
Change-Id: I39a4f7ebc0cae6d403baba99307d7ce408f25966
BUG: 1242280
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11638
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a possibility that contribution is removed twice from list
during unlink operation (with hard links) or during rename operation
Use list_del_init for a thread safe deltion of member from list
Change-Id: Iff5e0c03cc8f0ed85da0db1739b84b695abf9ea6
BUG: 1244109
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11706
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CID: 1124494
The variable was sent as a parameter to the function without the null check,
hence added a check on null and error log accordingly.
Change-Id: I11a649fc9a1bedad17422aeddb8dd8fa225d404e
BUG: 789278
Signed-off-by: arao <arao@redhat.com>
Reviewed-on: http://review.gluster.org/9641
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Free local in error paths
Change-Id: I76f69e7d746af8eedea34354ff5a6bf50234e50e
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11617
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
quota xattrs requested
In a lookup and build ancestry quota xattrs are
not requested, it gives wrong assumption that
quota xattrs are missing and tries to do
healing process
Change-Id: I7135101ec0edc72a6310dbb304227eaa3b16cb46
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11616
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During quota-update process if inode info is present in size-xattr and
missing in contri-xattrs, then in function '_mq_get_metadata', we set
contri-size as zero (on error -2, which means usage info present, but inode info missing).
With this we are calculating wrong delta and updating the same.
With this patch we are ignoring errors if inode info in xattrs are missing
Change-Id: I7940a0e299b8bb425b5b43746b1f13f775c7fb92
BUG: 1241153
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11583
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Default stacksize that synctask uses is 2M.
For marker we set it to 16k
Also move market xlator close to io-threads
to have smaller stack
Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11499
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem-1)
Now the marker accounting happens in background,
There is a possibility that before completing
create_xattr_txn another create txn can be initiated
for the same inode.
suppose if few 100 txns are initiated
before completion, this can block all synctask threads
waiting on a lock and this can also consume lot of memory
and can take more time to complete the background accounting
operation.
This patch improves the locking mechanism which
can improve the performance as well reduce memory
consumption
Problem-2)
For every lookup and for all inodes in readdirp
we were initiating a new txn, this can result
in more txn pending in synctask queue and
lead to huge memory consumption. inspect
file/dir should start a txn only if there
is some delta
Problem-3)
When there are multiple write operations on
same inode and all the synctask threads are busy.
As we are checking for updation_status
flag in background, all txn will be move to synctask queue.
This can increase the mem usage.
Only one txn for inode in a queue will be sufficient,
so check and set updation falg before moving txn to
background
Change-Id: Ic42ce00f0a50ce51c7128ba68a1b6a0699a1cd14
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11457
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before doing a rename operation marker takes inode lock on the file
parent,
here lk_owner is NULL and this can cause accounting problem with
multiple rename on the same directory
This patch fix the problem by setting lk_owner
Change-Id: Ibb789e39b2833e425d0a5fca85282ff1465206cb
BUG: 1240598
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11561
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With below test-case, marker accounting becomes bad:
1) Create a volume with 1 brick
2) fuse mount
3) on one terminal write some data
dd if=/dev/zero of=f1 bs=1M count=500 oflag=sync
4) on another terminal execute below rename operation while the write is
still in progress
for i in {1..50}; do
ii=`expr $i + 1`;
mv f$i f$ii;
done
remove-xattr is already on while doing rename operation,
we should not be doing again in background when reducing the
parent size
Change-Id: I969a64bb559e2341315928b55b99203e9ddee3f2
BUG: 1235195
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11403
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When removing contribution xattr, we also need to free
contribution node in memory.
Use ref/unref mechanism to handle contribution node memory
local->xdata should be freed in mq_local_unref
There is another huge memory consumption happens
in function mq_inspect_directory_xattr_task
where dirty flag is not set.
Change-Id: Ieca3ab4bf410c51259560e778bce4e81b9d888bf
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11361
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit 8ab6608accb62d6320d1fc1fbe651fcafd376270.
This patch is resulting in memory corruption:
http://build.gluster.org/job/rackspace-regression-2GB-triggered/11193/consoleFull
contribution object might be being used by some other transaction when
we free it. The correct way to handle this is to have a reference based
scheme to manage the contribution object.
Change-Id: Idf9993ed8268029073a3e2d699865587f20d9aea
BUG: 1207735
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/11362
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
When removing contribution xattr, we also need to free
contribution node in memory
Change-Id: I5fe97813a8f39e2f00401976046bd280f2eea54d
BUG: 1207735
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11311
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a rename operation is performed, we are renaming
the file first and performing remove-xattr when reducing
the contri size from parents.
This remove-xattr fails as the file is alreday renamed,
this failure causes reduce-parent-size to abort resulting
in double quota accounting
This patch fixes the problem. We don't need to perform remove-xattr
operation on a file when performing reduce-parent-size txn as this
will be alreday done before starting reduce-parent-size txn
Change-Id: If86e3dbb0233f6deaaa90bee72cb0ec1689c7325
BUG: 1232572
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/11264
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The problem was in marker xlator, where during
rename a NULL value is passed during STACK_WIND.
Marker needs to pass the xdata un-modified to next
translator if marker does not rely on that.
Change-Id: I9e47e504fd241263987645abfed7ca13c0d54a80
BUG: 1228492
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/11089
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Missing loc_wipe() for error paths in mq_readdir_cbk() can
cause memory leaks. loc_wipe() is now done for both happy
and unhappy paths.
Change-Id: I882aa5dcca06e25b56a828767fb2b91a1efaf83b
BUG: 1227904
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/11074
Reviewed-by: Sachin Pandit <spandit@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During ancestry build, loc path was set to invalid
path. path was set to one of its child instead
of itself. Because of this quota accounting was
going wrong
This patch fix the issue
Below mentioned tests removed from bad test list
as part of patch# 10930
./tests/basic/ec/quota.t
./tests/basic/quota-nfs.t
./tests/bugs/quota/bug-1035576.t
Change-Id: Iaa65b2d968c04c9abcd476d0e9f588cb7fd39294
BUG: 1223798
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/10918
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
inode quota is a new feature implemented in glusterfs-3.7
if quota is enabled in the older version and is upgraded
to a new version, we can hit setxattr spike during self-heal
of inode quotas. So, when a quota is enabled, turn off
inode-quotas with a xlator option.
With this patch, we still account for inode quotas but only
when a write operation is performed for a particular file.
User will be able to query inode quotas once the Inode-quota
xlator option is enabled.
Change-Id: I52fb28bf7024989ce7bb08ac63a303bf3ec1ec9a
BUG: 1209430
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/10152
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In quota readdirp_cbk, inode ctx filled for the all entries
received.
In marker readdirp_cbk, files/directories are inspected for
dirty
There is no guarantee that entry->inode is populated.
If entry->inode is NULL, this needs to be treated as readdir
Change-Id: Id2d17bb89e4770845ce1f13d73abc2b3c5826c06
BUG: 1215550
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/10416
Tested-by: NetBSD Build System
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a problem during upgrade where, inode quotas are not healed in
the contri xattrs.
Healing happens if contri xattrs are missing.
But healing doesn't happen if contri xattrs are present and inode quota
values are missing in the contri xattrs.
This patch fixes the problem
Change-Id: I6c88b74b5bb333a97c5419e24cc4ada82839f474
BUG: 1211808
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/10239
Tested-by: NetBSD Build System
Reviewed-by: Sachin Pandit <spandit@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.
libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.
Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.
Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch adds support for xdata in both the
request and response path of syncops.
Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.
xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.
There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.
xdata support at gfapi level would be introduced
in subsequent patches.
Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.
Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.
A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.
BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CIDs:
1124601
1124690
1134008
1134009
Change-Id: I93992b11bb6d8f7edd065f602aec2cd7a8b433d0
BUG: 789278
Signed-off-by: Nandaja Varma <nandaja.varma@gmail.com>
Reviewed-on: http://review.gluster.org/9614
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
==========================================================================
Inode quota
==========================================================================
= Currently, the only way to retrieve the number of files/objects in a =
= directory or volume is to do a crawl of the entire directory/volume. =
= This is expensive and is not scalable. =
= =
= The proposed mechanism will provide an easier alternative to determine =
= the count of files/objects in a directory or volume. =
= =
= The new mechanism proposes to store count of objects/files as part of =
= an extended attribute of a directory. Each directory's extended =
= attribute value will indicate the number of files/objects present =
= in a tree with the directory being considered as the root of the tree. =
= =
= The count value can be accessed by performing a getxattr(). =
= Cluster translators like afr, dht and stripe will perform aggregation =
= of count values from various bricks when getxattr() happens on the key =
= associated with file/object count. =
A new interface is introduced:
------------------------------
limit-objects : limit the number of inodes at directory level
list-objects : list the directories where the limit is set
remove-objects : remove the limit from the directory
==========================================================================
CLI COMMAND:
gluster volume quota <volname> limit-objects <path> <number> [<percent>]
* <number> is a hard-limit for number of objects limitation for path "<path>"
If hard-limit is exceeded, creation of file/directory is no longer
permitted.
* <percent> is a soft-limit for number of objects creation for path "<path>"
If soft-limit is exceeded, a warning is issued for each creation.
CLI COMMAND:
gluster volume quota <volname> remove-objects [path]
==========================================================================
CLI COMMAND:
gluster volume quota <volname> list-objects [path] ...
Sample output:
------------------
Path Hard-limit Soft-limit Used Available
Soft-limit exceeded?
Hard-limit exceeded?
------------------------------------------------------------------------
--------------------------------------
/dir 10 80% 10 0
Yes
Yes
==========================================================================
[root@snapshot-28 dir]# ls
a b file11 file12 file13 file14 file15 file16 file17
[root@snapshot-28 dir]# touch a1
touch: cannot touch `a1': Disk quota exceeded
* Nine files are created in directory "dir" and directory is included in
* the
count too. Hence the limit "10" is reached and further file creation
fails
==========================================================================
Note: We have also done some re-factoring in cli for volume name
validation. New function cli_validate_volname is created
==========================================================================
Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b
BUG: 1190108
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9769
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
marker-quota.c: In function 'mq_inspect_directory_xattr_task':
marker-quota.c:3451:31: warning: variable 'buf' set but not
used [-Wunused-but-set-variable]
struct iatt buf = {0,};
Change-Id: I211378328bdb2509a5d2a186d173f7f30a670c8a
BUG: 1198849
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/9928
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Marker can fail or can account incorrect numbers when it doesn't find a
ancestry for a inode.
Solution:
Current build_ancestry is done only on demand in the write/create FOPs
in quota enforcer.
It is good to do this in the quota_lookup as well.
Change-Id: I8aaf5b3e05a3ca51e7ab1eaa1b636a90f659a872
BUG: 1184885
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9478
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, the only way to retrieve the number of files/objects in a
directory or volume is to do a crawl of the entire directory/volume.
This is expensive and is not scalable.
The new mechanism proposes to store count of objects/files as part of
an extended attribute of a directory. Each directory's extended
attribute value will indicate the number of files/objects present
in a tree with the directory being considered as the root of the tree.
Currently file usage is accounted in marker by doing multiple FOPs
like setting and getting xattrs. Doing this with STACK WIND and
UNWIND can be harder to debug as involves multiple callbacks.
In this code we are replacing current mechanism with syncop approach
as syncop code is much simpler to follow and help us implement inode
quota in an organized way.
Change-Id: Ibf366fbe07037284e89a241ddaff7750fc8771b4
BUG: 1188636
Signed-off-by: vmallika <vmallika@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/9567
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* in readdirp callbak marker is calling inode_path on the inodes that
are not yet linked to the inode table.
Change-Id: I7f5db29c6a7e778272044f60f8e73c60574df3a9
BUG: 1176393
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/9320
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. Changed log messages to be more appropriate.
2. Changed loglevel of failures in fop_cbks to be recorded as TRACE.
Logging of failures at higher loglevels is unessential in non-endpoint
translators.
3. Removed a log message related to memory allocation failure.
BUG: 1174087
Change-Id: I63c560c3bbd12706357fb3f696378c1a1e1efb44
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/8168
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Iec89e7141f6ff3216355d8a6b971617e16df679b
BUG: 1075417
Signed-off-by: Jan-Hendrik Zab <zab@l3s.de>
Reviewed-on: http://review.gluster.org/9238
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Afr should ignore quota-size-key as part of self-heal
but should heal quota-limit key.
Change-Id: Ic0b06bd20a563a00d6bfdc2dc5a76c661e533ecb
BUG: 1161106
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9061
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unref dicts in quota and marker functions before they return.
Change-Id: I772edc7b46657a70fb136e037576c65e9beeaca8
BUG: 1139327
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/8655
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Linux defines ENODATA and ENOATTR with the same value, which means that
code can miss on on the two without breaking.
FreeBSD does not have ENODATA and GlusterFS defines it as ENOATTR just
like Linux does.
On NetBSD, ENODATA != ENOATTR, hence we need to check for both values
to get portable behavior.
BUG: 764655
Change-Id: I003a3af055fdad285d235f2a0c192c9cce56fab8
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/8447
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Geo-replicatoin does a full xsync crawl after snapshot
restoration of slave and master. It does not do history crawl.
Analysis:
Marker creates 'marker.tstamp' file when geo-rep is started
for the first time. The virtual extended attribute
'trusted.glusterfs.volume-mark' is maintained and whenever
it is queried on gluster mount point, marker fills it on
the fly and returns the combination of uuid, ctime of
marker.tstamp and others. So ctime of marker.tstamp, in other
sense 'volume-mark' marks the geo-rep start time when the
session is freshly created.
From the above, after the first filesystem crawl(xsync) is
done during first geo-rep start, stime should always be less
than 'volume-mark'. So whenever stime is less than volume-mark,
it does full filesystem crawl (xsync).
Root Cause:
When snapshot is restored, marker.tstamp file is freshly
created losing the timestamps, it was originally created with.
Solution:
1. Change is made to depend on mtime instead of ctime.
2. mtime and atime of marker.tstamp is restored back when
snapshot is created and restored.
Change-Id: I4891b112f4aedc50cfae402832c50c5145807d7a
BUG: 1125918
Signed-off-by: Kotresh H R <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/8401
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
The xattr healing part of the marker requires path to be present in the loc.
Currently path is not filled while triggering from the readdirp_cbk.
Solution:
Current patch tries to fill the loc with path.
Change-Id: I5c7dc9de60fa79ca0fe9b58d2636fd1355add0d3
BUG: 1118591
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/8296
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs
Working functionality on MacOSX
- GlusterD (management daemon)
- GlusterCLI (management cli)
- GlusterFS FUSE (using OSXFUSE)
- GlusterNFS (without NLM - issues with rpc.statd)
Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac
BUG: 1089172
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Signed-off-by: Dennis Schafroth <dennis@schafroth.com>
Tested-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Dennis Schafroth <dennis@schafroth.com>
Reviewed-on: http://review.gluster.org/7503
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
To be used in afr metadata self-heal
Change-Id: I8dac4b19d61e331702427eeb5b606aab3d20b328
BUG: 1021686
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6941
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If an error occurs and op_errno is not set to non-zero value, we can
end up in loosing a frame resulting in a hung syscall. This patch
adds code setting op_errno appropriately in storage/posix and
makes marker to set err to a default non-zero value in case of
op_errno being zero.
Change-Id: Idc2c3e843b932709a69b32ba67deb284547168f2
BUG: 833586
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/5032
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Quota and marker uses 'trusted.glusterfs.quota*' and 'trusted.pgfid*' xattrs to
store its configurations and accounting information and also to build the
parent inode chain in case of absense of path.
Problem:
After disabling and then enabling quota back, the xattrs may contain stale data
leading to impaired accounting and thus improper enforcement.
Solution:
Clean up all the quota related xattrs after quota disable.
Marker xlator implements a virtual xattr to cleanup quota and pgfid xattrs. In
this approach glusterd mounts an auxiliary mount and sends the below command to
all the files by crawling the mountpoint.
#setfattr -n "glusterfs.quota-xattr-cleanup" -v 1 <path/to/file>
Credit:
Krishnan Parthasarathi <kparthas@redhat.com>
Varun Shastry <vshastry@redhat.com>
Change-Id: I9380eca58a285dc27dd572de1767aac8f2cd8049
BUG: 969461
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/6369
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Quota contributions of a file/directory are tracked by quota
xlator using xattrs on the file. Quota allows these xattrs to be
healed as part of metadata self-heal. This leads to
wrong quota calculations on this brick after self-heal because
quota xattrs don't represent the actual contributions on the
brick anymore.
Fix:
Don't let self-heal of this xattr happen as part of self-heal
by filtering quota xattrs on file in listxattr.
Change-Id: Iea68a116595ba271e58c6fdcc3dd21c7bb55ebb3
BUG: 1035576
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/6374
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is needed for two reasons:
* since dht-linkfiles are internal, they shouldn't be accounted.
* hardlink handling in marker is broken. link/unlink of hardlinks
present in same directory can break marker accounting. Hence, if src
and dst are in same directory in case of rename, dht - if it breaks
rename into link/unlink operations - should instruct marker to not to
do accounting.
Change-Id: I9c9f7384569f75a2792f6450ee7a5279bf751ae7
BUG: 1022995
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: http://review.gluster.org/6203
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I3239f5e8477664dcc04434e4d455ae447493a7ac
BUG: 1022995
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: http://review.gluster.org/6153
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* handles renames on dht linkfiles correctly
* nameless lookup friendly changes. uses gfid-to-path conversion
functionality from storage/posix to build ancestry till root.
* log message cleanup.
* build inode contexts in readdirp
* Accounting still not correct with hardlinks.
Credits:
========
Vijay Bellur <vbellur@redhat.com>
Raghavendra Bhat <rabhat@redhat.com>
Change-Id: I415b6fbbc9691f5a38d9fd3c5d083a61e578bb81
BUG: 969461
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: http://review.gluster.org/5953
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glfs_zerofill() can be potentially called to zero-out entire file and
hence allow for bigger value of length parameter.
Change-Id: I75f1d11af298915049a3f3a7cb3890a2d72fca63
BUG: 1028673
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-on: http://review.gluster.org/6266
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: M. Mohan Kumar <mohan@in.ibm.com>
Tested-by: M. Mohan Kumar <mohan@in.ibm.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|