| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
| |
This directory holds the extended attribute backing store on FFSv1
filesystems. It should be ignored by glusterfs just like the .glusterfs
directory.
Fixes: #1361
Change-Id: Ia259be2af722a651473a312b5044b431c2b01b10
Type: Bug
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Before executing a fop in POSIX xlator it builds an internal
path based on GFID.To validate the path it call's (l)stat
system call and while .glusterfs is heavily loaded kernel takes
time to lookup inode and due to that performance drops
Solution: In this patch we followed two ways to improve the performance.
1) Keep open fd specific to first level directory(gfid[0])
in .glusterfs, it would force to kernel keep the inodes
from all those files in cache. In case of memory pressure
kernel won't uncache first level inodes. We need to open
256 fd's per brick to access the entry faster.
2) Use at based call's to access relative path to reduce
path based lookup time.
Note: To verify the patch we have executed kernel untar 100 times on 6
different clients after enabling metadata group-cache and some
other option.We were getting more than 20 percent improvement in
kenel untar after applying the patch.
Credits: Xavi Hernandez <xhernandez@redhat.com>
Change-Id: I1643e6b01ed669b2bb148d02f4e6a8e08da45343
updates: #891
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: With lookup-optimize set to on by default, a client with
stale-layout can create a new file on a wrong subvol. This will lead to
possible duplicate files if two different clients attempt to create the
same file with two different layouts.
Solution: Send in-memory layout to be cross checked at posix before
commiting a "create". In case of a mismatch, sync the client layout with
that of the server and attempt the create fop one more time.
test: Manual, testcase(attached)
fixes: bz#1786679
Change-Id: Ife0941f105113f1c572f4363cbcee65e0dd9bd6a
Signed-off-by: Susant Palai <spalai@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
The macro already fetches from the dictionary the gfid-req variable.
Instead of throwing it away, keep it and in 2 cases,
re-use it later.
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Id6d42535132805ab69e7e3b802c140b90f2f8958
|
|
|
|
|
|
|
|
| |
Do not include ftw.h twice.
Change-Id: Id9e8d1813aafd890940adcd6883d90fa1b4beaf9
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Updates: bz#1193929
|
|
|
|
|
|
|
|
|
|
| |
1. Move them to posix-entry-ops.c and make them static.
2. Remove useless 'val' parameter allocation and snprintf() in
posix_set_gfid2path_xattr(), as it was identical to pgfid_bname.
Change-Id: I41f3946c6c42aee9d7cf1150e21b442438fbcce2
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Sometime fops like posix_writev, posix_fallocate, posix_zerofile
failed and throw error ENOSPC if storage.reserve threshold limit
has reached even fops is overwriting the data
Solution: Retry the fops in case of overwrite if diskspace check
is failed
Credits: kinsu <vpolakis@gmail.com>
Change-Id: I987d73bcf47ed1bb27878df40c39751296e95fe8
Updates: #745
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Remove TIER_LINKFILE_GFID related code from posix
Tier xlator was removed, but there are some code related to it scattered
around in DHT and Posix xlators. Remove some of it.
Change-Id: I3a878b31ed4a045ed419f936aa1d567ded1a273f
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Pass the maximum buffer (PATH_MAX) to posix_handle_path to
avoid the posix_handle_path call again.
Change-Id: I7d18313870218dc028c5f7fc94d6ec85a2bdb332
Updates #761
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
| |
Avoid one function call to set the gfid_path in buffer
Change-Id: If9b95801b05c34d262fac9a275492c794d12bf58
Updates #748
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As was found out in a different patch, passing a pointer
to a constant string still calculates the size of the pointer
and not the string. This is not catastrophic as the dictionary
is matching not just hashes but key names, but is certainly what
was intended to be done.
Instead, pass explicitly the key lenth.
I've looked for additional cases for such an issue and did not
find them, luckily.
Change-Id: I644a07a77dd541e9cf4886811ab54897e0c9d483
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The code is simplified to avoid needless copy as well as simplified
overall for readability.
Such changes are needed elsewhere too (see
https://github.com/gluster/glusterfs/issues/720 )
Few other minor changes here and there, nothing functional.
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I14f9dd2c32a8932bfcc80ebe92c9aa77701095ff
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After add-brick and rebalance, the ctime xattr is not present
on rebalanced directories on new brick. This patch fixes the
same.
Note that ctime still doesn't support consistent time across
distribute sub-volume.
This patch also fixes the in-memory inconsistency of time attributes
when metadata is self healed.
Change-Id: Ia20506f1839021bf61d4753191e7dc34b31bb2df
fixes: bz#1734026
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
| |
warning: ‘%s’ directive argument is null [-Wformat-overflow=]
Change-Id: I69b8d47f0002c58b00d1cc947fac6f1c64e0b295
updates: bz#1193929
Signed-off-by: SheetalPamecha <spamecha@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are many include statements that are not needed.
A previous more ambitious attempt failed because of *BSD plafrom
(see https://review.gluster.org/#/c/glusterfs/+/21929/ )
Now trying a more conservative reduction.
It does not solve all circular deps that we have, but it
does reduce some of them. There is just too much to handle
reasonably (dht-common.h includes dht-lock.h which includes
dht-common.h ...), but it does reduce the overall number of lines
of include we need to look at in the future to understand and fix
the mess later one.
Change-Id: I550cd001bdefb8be0fe67632f783c0ef6bee3f9f
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The way delta_blocks is computed in shard is incorrect, when a file
is truncated to a lower size. The accounting only considers change
in size of the last of the truncated shards.
FIX:
Get the block-count of each shard just before an unlink at posix in
xdata. Their summation plus the change in size of last shard
(from an actual truncate) is used to compute delta_blocks which is
used in the xattrop for size update.
Change-Id: I9128a192e9bf8c3c3a959e96b7400879d03d7c53
fixes: bz#1705884
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Main changes include logic to update iatt buf
with file size from extended attributes in posix
rather than having this logic in cloudsync xlator.
Change-Id: I44f5f8df7a01e496372557fe2f4eff368dbdaa33
fixes: bz#1642168
Signed-off-by: Anuradha Talur <atalur@commvault.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
nr_files is supposed to represent the number of files opened in posix.
Present logic doesn't seem to handle anon-fds because of which the
counts would always be wrong.
I don't remember anyone using this value in debugging any problem probably
because we always have 'ls -l /proc/<pid>/fd' which not only prints the
fds that are active but also prints their paths. It also handles directories
and anon-fds which actually opened the file. So removing this code
instead of fixing the buggy logic to have the nr_files.
fixes bz#1688106
Change-Id: Ibf8713fdfdc1ef094e08e6818152637206a54040
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
| |
Change-Id: I629698d8ddf6f15428880bdc1501d36bc37b8ebb
fixes: bz#1657607
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With brick mux, the number of threads increases as the number of
bricks increases. As an initiative to reduce the number of
threads in brick mux scenario, replacing janitor thread to use
synctask infra.
Now close() and closedir() handle by separate janitor
thread which is linked with glusterfs_ctx.
Updates #475
Change-Id: I0c4aaf728125ab7264442fde59f3d08542785f73
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.
Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs
This change although big, is just moving around the headers and
making it correct when including these headers from other sources.
This helps us correctly include libglusterfs includes without
namespace conflicts.
Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes coverity
CID : 1356537
https://scan6.coverity.com/reports.htm#v42907/p10714/fileInstanceId=87389108&defectInstanceId=26791927&mergedDefectId=1356537
CID : 1395666
https://scan6.coverity.com/reports.htm#v42907/p10714/fileInstanceId=87389187&defectInstanceId=26791932&mergedDefectId=1395666
CID : 1351707
https://scan6.coverity.com/reports.htm#v42907/p10714/fileInstanceId=87389027&defectInstanceId=26791973&mergedDefectId=1351707
CID : 1396910
https://scan6.coverity.com/reports.htm#v42907/p10714/fileInstanceId=87389027&defectInstanceId=26791973&mergedDefectId=13596910
Change-Id: I8094981a741f4d61b083c05a98df23dcf5b022a2
updates: bz#789278
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch fixes CID:
1. 1389762 : Explicit null dereferenced
2. 1390462 : Argument cannot be negative
3. 1124552 : Explicit null dereferenced
4. 1356522 : Argument cannot be negative
updates: bz#789278
Change-Id: I1262f3b4b61a6e65bb34884f46df9a24b8dd03f1
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
* Buffer overflow issue in glusterfsd
* Null argument passed to function expecting non-null (event-epoll)
* Make sure the op_ret value is set in macro (posix)
Updates: bz#1622665
Change-Id: I32b378fc40a5e3ee800c0dfbc13335d44c9db9ac
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
The value of trusted.pgfid.xx was always set to 1
in posix_mknod. This is incorrect if posix_mknod
calls posix_create_link_if_gfid_exists.
Change-Id: Ibe87ca6f155846b9a7c7abbfb1eb8b6a99a5eb68
fixes: bz#1619720
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
If the gfid link file inside .glusterfs is not present for a file,
the operations which are dependent on the gfid will fail,
complaining the link file does not exists inside .glusterfs.
Fix:
If the link file creation fails, fail the entry creation operation
and delete the original file.
Change-Id: Id767511de2da46b1f45aea45cb68b98d965ac96d
fixes: bz#1612037
Signed-off-by: karthik-us <ksubrahm@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
BZ 1564071 complains of directories with missing gfid symlinks and
corresponding "Found stale gfid handle" messages in the logs. Hence
add a check to see if the symlink points to an actual directory before
removing it.
Note: Removing stale symlinks was added via commit
3e9a9c029fac359477fb26d9cc7803749ba038b2
Change-Id: I5d91fab8e5f3a621a9ecad4a1f9c898a3c2d346a
Updates: bz#1564071
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since setxattr and removexattr fops cbk do not carry poststat,
the stat cache was being invalidated in setxatr/remoxattr cbk.
Hence the further lookup wouldn't be served from cache.
To prevent this invalidation, md-cache is modified to get
the poststat in set/removexattr_cbk in dict.
Co-authored with Xavi Hernandez.
Change-Id: I6b946be2d20b807e2578825743c25ba5927a60b4
fixes: bz#1586018
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
1) snprintf into linkname_expected should happen with PATH_MAX
2) comparison should happen with linkname_actual with complete
string linkname_expected
fixes bz#1595190
Change-Id: Ic3b3c362dc6c69c046b9a13e031989be47ecff14
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. Successful rename was not updating ctime.
Fixed the same.
2. Successful unlink when link count is more than 1
was not updating ctime. Fixed the same.
3. Copy ctime and flags during frame copy.
fixes: bz#1580020
Change-Id: Ied47275a36aea60254b2add7a59128a9c83b3645
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
This patch uses the ctime posix APIs to set consistent
time across replica on disk. It also stores the time
attributes in the inode context.
Credits: Rafi KC <rkavunga@redhat.com>
Updates: #208
Change-Id: I1a8d74d1e251f1d6d142f066fc99258025c0bcdd
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch uses the ctime posix APIs to get consistent
time across replica. The time attributes are got from
from inode context or from on disk if not found and
merged with iatt to be returned.
Credits: Rafi KC <rkavunga@redhat.com>
Updates: #208
Change-Id: Id737038ce52468f1f5ebc8a42cbf9c6ffbd63850
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is part of the effort to provide consistent time
across distribute and replica set for time attributes
(ctime, atime, mtime) of the object. This patch contains
the APIs to set and get the attributes from on disk
and in inode context.
Credits: Rafi KC <rkavunga@redhat.com>
Updates: #208
Change-Id: I5d3cba53eef90ac252cb8299c0da42ebab3bde9f
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In context of Cloudsync:
In scenarios where a data modification fop e.g. a write landed in
POSIX thinking that the file is local, while the file was actually
remote, can be dangerous. Ofcourse we don’t want to take inodelk
for every read/write operation to check the archival status or
coordinate with an upload or a download of a file. To avoid inodelk,
we will check the status of the file in POSIX it self, before we
resume the fop. This helps us avoiding any races mentioned above.
Now e.g. if a write reached POSIX for a file which was actually remote,
it can check the status of the file and will get to know that the file
is remote. It can error out with this status “remote” and cloudsync
xlator will retry the same operation, once it finished downloading the
file.
This patch includes the setxattr changes to do the post processing of
upload i.e. truncate and setting the remote xattr
"trusted.glusterfs.cs.remote" to indicate the file is REMOTE
Each file will have no xattr if the file is LOCAL, one remote xattr if
the file is REMOTE and a combination of REMOTE and DOWNLOADING xattr if
the file is getting downloaded. There is healing logic of these xattrs
to recover from crash inconsitencies.
Fixes: #387
Change-Id: Ie93c2d41aa8d6a798a39bdbef9d1669f057e5fdb
Signed-off-by: Susant Palai <spalai@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: At the time of stopping the volume while brick multiplex is
enabled memory is not cleanup from all server side xlators.
Solution: To cleanup memory for all server side xlators call fini
in glusterfs_handle_terminate after send GF_EVENT_CLEANUP
notification to top xlator.
BUG: 1544090
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Note: Run all test-cases in separate build (https://review.gluster.org/19574)
with same patch after enable brick mux forcefully, all test cases are
passed.
Change-Id: Ia10dc7f2605aa50f2b90b3fe4eb380ba9299e2fc
|
|
|
|
|
|
|
|
|
|
| |
Due to merging https://review.gluster.org/#/c/19131 before
https://review.gluster.org/18804 the build started failing.
Fixing with this patch.
Change-Id: Ia1eedf481a0f6d25f1357ee56d6bbb7a7e9d9793
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
| |
Updates #353
Change-Id: I1410222529ff42a810d62825bd6eebe8913fad1e
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For more details on this issue see
https://github.com/gluster/glusterfs/issues/308
Solution:
This is a restrictive solution where a file will not be migrated
if a client writes to it during the migration. This does not
check if the writes from the rebalance and the client actually
do overlap.
If dht_writev_cbk finds that the file is being migrated (PHASE1)
it will set an xattr on the destination file indicating the file
was updated by a non-rebalance client.
Rebalance checks if any other client has written to the dst file
and aborts the file migration if it finds the xattr.
updates gluster/glusterfs#308
Change-Id: I73aec28bc9dbb8da57c7425ec88c6b6af0fbc9dd
Signed-off-by: Susant Palai <spalai@redhat.com>
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
|
|
|
| |
..in order for self-heal of symlinks to work properly (see BZ for
details).
Change-Id: I9a011d00b07a690446f7fd3589e96f840e8b7501
BUG: 1529488
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Added 2 more types which are present in gluster codebase, mainly
IATT and UUID.
Updates #203
Change-Id: Ib6d6d6aefb88c3494fbf93dcbe08d9979484968f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
| |
xdata is freed when calling STACK_UNWIND_STRICT, so that,
posix_gfid_unset uses xdata after freed.
Change-Id: Id0ff2da3d6d9be204e50d9fb37ba57558936c85c
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
directory
Summary:
- We may have found an issue where certain directories were being moved into .landfill and then being quickly purged via nftw().
- We would like to have an emergency option to disable these purges.
> Reviewed-on: https://review.gluster.org/18253
> Reviewed-by: Shreyas Siravara <sshreyas@fb.com>
Fixes #371
Change-Id: I90b54c535930c1ca2925a928728199b6b80eadd9
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
Too may hard links blow up btrfs by exceeding max xattr size (recordign
pgfid for each hardlink). Add a limit to prevent this explosion.
> Reviewed-on: https://review.gluster.org/18232
> Reviewed-by: Shreyas Siravara <sshreyas@fb.com>
Fixes gluster/glusterfs#370
Signed-off-by: ShyamsundarR <srangana@redhat.com>
Change-Id: I614a247834fb8f2b2743c0c67d11cefafff0dbaa
|
|
1. Split out entry and inode/fd based FOPs into
separate files from posix.c
2. Split out common routines (init, fini, reconf,
and such) into its own file, from posix.c
3. Retain just the method assignments in posix.c
(such that posix2 for RIO can assign its own methods in
the future for entry operations and such)
4. Based on the split in (1) and (2) split out
posix-handle.h into 2 files, such that macros that are
needed for inode ops are in one and rest are in the other
If the split is done as above, posix2 can compile with
its own entry ops, and hence not compile, the entry ops
as split in (1) above.
The split described in (4) can again help posix2 to
define its own macros to make entry and inode handles,
thus not impact existing POSIX xlator code.
Noted problems
- There are path references in certain cases where
quota is used (in the xattr FOPs), and thus will fail
on reuse in posix2, this needs to be handled when we
get there.
- posix_init does set root GFID on the brick root,
and this is incorrect for posix2, again will need
handling later when posix2 evolves based on this
code (other init checks seem fine on current inspection)
Merge of experimental branch patches with the following
gerrit change-IDs
> Change-Id: I965ce6dffe70a62c697f790f3438559520e0af20
> Change-Id: I089a4d9cf470c2f9c121611e8ef18dea92b2be70
> Change-Id: I2cec103f6ba8f3084443f3066bcc70b2f5ecb49a
Fixes gluster/glusterfs#327
Change-Id: I0ccfa78559a7c5a68f5e861e144cf856f5c9e19c
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|