| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
ec_getxattr_heal_cbk was called with NULL as second argument
in case heal was failing.
This function was dereferencing "cookie" argument which caused crash.
Solution:
Cookie is changed to carry the value that was supposed to be
stored in fop->data, so even in the case when fop is NULL in error
case, there won't be any NULL dereference.
Thanks to Xavi for the suggestion about the fix.
Change-Id: I0798000d5cadb17c3c2fbfa1baf77033ffc2bb8c
fixes: bz#1805057
|
|
|
|
|
|
|
|
|
|
| |
When discard/truncate performs write fop, it should do so
after updating lock->good_mask to make sure readv happens
on the correct mask
fixes: bz#1805056
Change-Id: Idfef0bbcca8860d53707094722e6ba3f81c583b7
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
when a file needs to be re-opened O_APPEND and O_EXCL
flags are not filtered in EC.
- O_APPEND should be filtered because EC doesn't send O_APPEND below EC for
open to make sure writes happen on the individual fragments instead of at the
end of the file.
- O_EXCL should be filtered because shd could have created the file so even
when file exists open should succeed
- O_CREAT should be filtered because open happens with gfid as parameter. So
open fop will create just the gfid which will lead to problems.
Fix:
Filter out these two flags in reopen.
Change-Id: Ia280470fcb5188a09caa07bf665a2a94bce23bc4
Fixes: bz#1805055
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
There are cases where fop->mask may have fop->healing added
and readv shouldn't be wound on fop->healing. To avoid this
always wind readv to lock->good_mask
fixes: bz#1805054
Change-Id: I2226ef0229daf5ff315d51e868b980ee48060b87
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
EC doesn't allow concurrent writes on overlapping areas, they are
serialized. However non-overlapping writes are serviced in parallel.
When a write is not aligned, EC first needs to read the entire chunk
from disk, apply the modified fragment and write it again.
The problem appears on sparse files because a write to an offset
implicitly creates data on offsets below it (so, in some way, they
are overlapping). For example, if a file is empty and we read 10 bytes
from offset 10, read() will return 0 bytes. Now, if we write one byte
at offset 1M and retry the same read, the system call will return 10
bytes (all containing 0's).
So if we have two writes, the first one at offset 10 and the second one
at offset 1M, EC will send both in parallel because they do not overlap.
However, the first one will try to read missing data from the first chunk
(i.e. offsets 0 to 9) to recombine the entire chunk and do the final write.
This read will happen in parallel with the write to 1M. What could happen
is that half of the bricks process the write before the read, and the
half do the read before the write. Some bricks will return 10 bytes of
data while the otherw will return 0 bytes (because the file on the brick
has not been expanded yet).
When EC tries to recombine the answers from the bricks, it can't, because
it needs more than half consistent answers to recover the data. So this
read fails with EIO error. This error is propagated to the parent write,
which is aborted and EIO is returned to the application.
The issue happened because EC assumed that a write to a given offset
implies that offsets below it exist.
This fix prevents the read of the chunk from bricks if the current size
of the file is smaller than the read chunk offset. This size is
correctly tracked, so this fixes the issue.
Also modifying ec-stripe.t file for Test #13 within it.
In this patch, if a file size is less than the offset we are writing, we
fill zeros in head and tail and do not consider it strip cache miss.
That actually make sense as we know what data that part holds and there is
no need of reading it from bricks.
Change-Id: Ic342e8c35c555b8534109e9314c9a0710b6225d6
Fixes: bz#1805053
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The ec_manager_open/opendir memsets ctx->loc which causes
memory/inode leak, and ec_fheal uses ctx->loc out of fd->lock
that loc_copy may copy bad data when memset it.
This patch skips updating ctx->loc when it is initilizaed.
With it, ctx->loc is filled once, and never updated.
Change-Id: I3bf5ffce4caf4c1c667f7acaa14b451d37a3550a
fixes: bz#1805052
Signed-off-by: Kinglong Mee <mijinlong@horiscale.com>
|
|
|
|
|
|
|
|
|
| |
If lock has info, fop should inherit healing mask from it.
Otherwise, fop cannot inherit right healing when changed_flags is zero.
Change-Id: Ife80c9169d2c555024347a20300b0583f7e8a87f
fixes: bz#1805051
Signed-off-by: Kinglong Mee <mijinlong@horiscale.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Race:
Thread-1 Thread-2
1) Does ec_get_size_version() to perform
pre-op fxattrop as part of write-1
2) Calls ec_set_dirty_flag() in
ec_get_size_version() for write-2.
This sets dirty[] to 1
3) Completes executing
ec_prepare_update_cbk leading to
ctx->dirty[] = '1'
4) Takes LOCK(inode->lock) to check if there are
any flags and sets dirty-flag because
lock->waiting_flag is 0 now. This leads to
fxattrop to increment on-disk dirty[] to '2'
At the end of the writes the file will be marked for heal even when it doesn't need heal.
Fix:
Perform ec_set_dirty_flag() and other checks inside LOCK() to prevent dirty[] to be marked
as '1' in step 2) above
Fixes: bz#1805050
Change-Id: Icac2ab39c0b1e7e154387800fbededc561612865
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Doing re-open with O_TRUNC will truncate the fragment even when it is not
needed needing extra heals
Fix:
At the time of re-open don't use O_TRUNC.
fixes: bz#1805049
Change-Id: Idc6408968efaad897b95a5a52481c66e843d3fb8
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently EC tries to reopen fd's that have been opened while a brick
was down. This is done as part of regular write operations, just after
having acquired the locks, and it's sent as a sub-fop of the main write
fop.
There were two problems:
1. The reopen was attempted on all UP bricks, even if a previous lock
didn't succeed. This is incorrect because most probably the open will
fail.
2. If reopen is sent and fails, the error is propagated to the main
operation, causing it to fail when it shouldn't.
To fix this, we only attempt reopens on bricks where the current fop
owns a lock, and we prevent any error to be propagated to the main
fop.
To implement this behaviour an argument used to indicate the minimum
number of required answers has overloaded to also include some flags. To
make the change consistent, it has been necessary to rename the
argument, which means that a lot of files have been changed. However
there are no functional changes.
This change has also uncovered a problem in discard code, which didn't
correctely process requests of small sizes because no real discard fop
was being processed, only a write of 0's on some region. In this case
some fields of the fop remained uninitialized or with incorrect values.
To fix this, a new function has been created to simulate success on a
fop and it's used in the discard case.
Thanks to Pranith for providing a test script that has also detected an
issue in this patch. This patch includes a small modification of this
script to force data to be written into bricks before stopping them.
Change-Id: If272343873369186c2fb8f43c1d9c52c3ea304ec
Fixes: bz#1805047
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
EC was ignoring lock contention notifications received while a lock was
being acquired. When a lock is partially acquired (some bricks have
granted the lock but some others not yet) we can receive notifications
from acquired bricks, which should be honored, since we may not receive
more notifications after that.
Since EC was ignoring them, once the lock was acquired, it was not
released until the eager-lock timeout, causing unnecessary delays on
other clients.
This fix takes into consideration the notifications received before
having completed the full lock acquisition. After that, the lock will
be releaed as soon as possible.
Backport of:
> BUG: bz#1708156
> Change-Id: I2a306dbdb29fb557dcab7788a258bd75d826cc12
> Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Fixes: bz#1717282
Change-Id: I2a306dbdb29fb557dcab7788a258bd75d826cc12
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
ec_truncate_clean does writing under the lock granted for truncate,
but the lock is calculated by ec_adjust_offset_up, so that,
the write in ec_truncate_clean is out of lock.
Updates: bz#1699500
Change-Id: I15ed1b0807d75c5eb817323f1c227e97d03e0e7c
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
(cherry picked from commit 0e1223491e964096384edfae5032ed0d50d028ad)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There was a problem in commit 7f81067 that caused infinite loop when
full heal was triggered.
The previous commit was made to prevent self-heal to go idle after a
replace brick operation. One of the changes consisted on setting a
flag to force an immediate scan of the dirty directory if a heal on
a directory succeeded (assuming it could have generated newer entries).
However that change was causing an issue with a full self-heal, since
every time an already healed directory was checked and it returned
suceessfully, it was also setting the flag, forcing self-heal to start
over again.
This patch fixes this issue by only setting the flag if the heal is not
full. It's assumed that a full self-heal will already traverse all
entries automatically, so there's no need to force a new scan later.
>Change-Id: Id12dbfc04e622b18183e796cc6cc87ccc30a6d55
>fixes: bz#1636631
>Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
(cherry picked from commit 7150c51ad75ccba22045a35fc31e5037612d1ad4)
Change-Id: Id12dbfc04e622b18183e796cc6cc87ccc30a6d55
fixes: bz#1651525
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As lookup is not a locked fop, we can not trust the
data received in this to be same.
Changing the log level to DEBUG in case lookup finds any
difference.
(cherry picked from commit 9be6bf3d90e3783b3ba559c93d41b933f8d53f03)
Change-Id: I39499c44688a2455c7c6c69a798762d045d21b39
updates: bz#1644622
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, there are possibilities in few places, where a user-controlled
(like filename, program parameter etc) string can be passed as 'fmt' for
printf(), which can lead to segfault, if the user's string contains '%s',
'%d' in it.
While fixing it, makes sense to make the explicit check for such issues
across the codebase, by making the format call properly.
Fixes: CVE-2018-14661
Fixes: bz#1647666
Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
| |
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
|
|
|
|
| |
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
|
|
|
|
|
|
|
|
|
|
| |
the same macro is defined in common-utils.h, which seems to be
much better place for the same.
Updates: bz#1193929
Change-Id: I409b719c291102136500b955e5827a550142ed96
Signed-off-by: Amar Tumballi <amarts@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a fop has succeeded on all the bricks and trying to release
the lock, there is no need to update the version for the
file/entry. All it will do is to increase the version from
x to x+1 on all the bricks.
If this update (x to x+1) fails on some brick, this will indicate
that the entry is unhealthy while in realty everything is fine
with the entry.
Avoiding this update will help to not to send one xattrop
at the end of the fops. Which will decrease the chances
of entries being in unhealthy state and also improve the
performance.
Change-Id: Id9fca6bd2991425db6ed7d1f36af27027accb636
fixes: bz#1623759
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
| |
Change-Id: I037e52a3467467b81a1ba5416317870864060d4d
updates: bz#1615703
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix following coverity issues-
CID:
1382378
1382459
https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=85091670&defectInstanceId=25915064&mergedDefectId=1382459
https://scan6.coverity.com/reports.htm#v42607/p10714/fileInstanceId=85091670&defectInstanceId=25915063&mergedDefectId=1382378
Problem:
ASSERT_LOCAL(this, healer) function is supposed to get the local healer
so that we can take advantage of it while healing and reading data.
However, we are not using healer->local anywhere.
Also, this is not as useful in context of EC as it is in AFR.
In EC we have to raed fragments from 4 bricks to heal a bad
fragment on other brick.
Change-Id: Iea8ce127ea02cc84e3823cb2be82a47872217b33
updates: bz#789278
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a squash of multiple commits:
contrib/fuse-lib/misc.c: remove unneeded memset()
All flock variables are properly set, no need to memset it.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I8e0512c5a88daadb0e587f545fdb9b32ca8858a2
libglusterfs/src/{client_t|fd|inode|stack}.c: remove some memset()
I don't think there's a need for any of them.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I2be9ccc3a5cb5da51a92af73488cdabd1c527f59
libglusterfs/src/xlator.c: remove unneeded memset()
All xl->mem_acct members are properly set,
no need to memset it.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I7f264cd47e7a06255a3f3943c583de77ae8e3147
xlators/cluster/afr/src/afr-self-heal-common.c: remove unneeded memset()
Since we are going over the whole array anyway, initialize it
properly, to either 1 or 0.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ied4210388976b6a7a2e91cc3de334534d6fef201
xlators/cluster/dht/src/dht-common.c: remove unneeded memset()
Since we are going over the whole array anyway it is initialized
properly.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Idc436d2bd0563b6582908d7cbebf9dbc66a42c9a
xlators/cluster/ec/src/ec-helpers.c: remove unneeded memset()
Since we are going over the whole array anyway it is initialized
properly.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I81bf971f7fcecb4599e807d37f426f55711978fa
xlators/mgmt/glusterd/src/glusterd-volgen.c: remove some memset()
I don't think there's a need for any of them.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I476ea59ba53546b5153c269692cd5383da81ce2d
xlators/mgmt/glusterd/src/glusterd-geo-rep.c: read() in 4K blocks
The current 1K seems small. 4K is usually better (in Linux).
Also remove a memset() that I don't think is needed between reads.
Only compile-tested!
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: I5fb7950c92d282948376db14919ad12e589eac2b
xlators/storage/posix/src/posix-{gfid-path|inode-fd-ops}.c: remove memset()
before sys_*xattr() functions.
I don't see a reason to memset the array sent to the functions
sys_llistxattr(), sys_lgetxattr(), sys_lgetxattr(), sys_flistxattr(),
sys_fgetxattr().
(Note: it's unclear to me why we are calling sys_*txattr() functions with
XATTR_VAL_BUF_SIZE-1 size instead of XATTR_VAL_BUF_SIZE ).
Only compile-tested!
Change-Id: Ief2103b56ba6c71e40ed343a93684eef6b771346
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Addresses CID 1370939
In ec_code_x64_epilog(), there is a possibility of reading from an
incorrect index of ec_code_x64_regmap array
Change-Id: Ib8a228bbe13631188343634b2bde5919cdaab5a4
Updates: bz#789278
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
{ec-heal|ec-combine|ec-helpers|ec-inode-read}.c
For const strings, just do compile time size calc instead of runtime.
Compile-tested only!
Change-Id: If92ba0a7a20f64b898d01c6e3b6708190ca93e04
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Fixing FORWARD_NULL coverify errors with EC.
CID: 1394650
BUG: 789278
Change-Id: I52c99dac3483ca31a86cd7e3a959d4010b195f32
updates: bz#789278
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mkstemp as per the Linux man page, uses 0600 as the permission
bits when creating the file. This is hence safe and a Coverity
warning that should be ignored.
Further, we are mostly a multi-threaded program in all our daemons
and cannot set and unset umask at will in a multi-threaded
program, to address the coverity issue.
This change attempts to nudge coverity to ignore this warning,
using the pattern,
/* coverity[EVENT_TAG_NAME] ... */
<line of code that has the issue>
This commit is an experiment, if post merge the next coverity
report ignores these errors, the above pattern (as found using
an internet search) works and can be applied to certain other
warnings as well.
Change-Id: I73a184ce1a54dd9e66542952b1190a74438c826a
Updates: bz#789278
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...
Only compile-tested!
Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since setxattr and removexattr fops cbk do not carry poststat,
the stat cache was being invalidated in setxatr/remoxattr cbk.
Hence the further lookup wouldn't be served from cache.
To prevent this invalidation, md-cache is modified to get
the poststat in set/removexattr_cbk in dict.
Co-authored with Xavi Hernandez.
Change-Id: I6b946be2d20b807e2578825743c25ba5927a60b4
fixes: bz#1586018
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Signed-off-by: Poornima G <pgurusid@redhat.com>
|
|
|
|
|
|
| |
fixes bz#1597156
Change-Id: I323eb9190e40b12df216698dcdba74a6d336beeb
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Multiple pre-op xattrop can be simultaneously being processed. On the cbk
it was checked if the fop was waiting for some specific data (like size and
version) and, if so, it was assumed that this answer should contain that
data.
This is not true, since a fop can be waiting for some data, but it may come
from the xattrop of another fop.
This patch differentiates between needing some information and providing it.
This is related to parallel writes. Disabling them fixed the problem, but
also prevented concurrent reads. A change has been made so that disabling
parallel writes still allows parallel reads.
Fixes: bz#1578325
Change-Id: I74772ad6b80b7b37805da93d5ec3ae099e96b041
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Currently it is not possible to capture the xattrs values which
are set on the bricks by calling syncop_(f)xattrop, because the
response dict is not being assigned to any of the dictionaries.
Fix:
In the xattrop callback capture the response dict and send it
back to the caller if it is requested.
Change-Id: I9de9bcd97d6008091c9b060bcca3676cb9ae8ef9
fixes: bz#1572076
Signed-off-by: karthik-us <ksubrahm@redhat.com>
|
|
|
|
|
|
| |
Change-Id: I0a290396c30c635b13ee73004d20259efb76a954
fixes: bz#1563945
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
The xattr trusted.glusterfs.list-node-uuids was only sent to a single
subvolume. This was returning null uuids from the other subvolumes as
if they were down.
This fix forces that xattr to be requested from all subvolumes.
Change-Id: If62eb39a6857258923ba625e153d4ad79018ea2f
fixes: bz#1561406
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the self-heal daemon is doing a full sweep it uses readdirp to
get extra stat information from each file. This information is
obtained in two steps by the posix xlator: first the directory is
read to get the entries and then each entry is stated to get additional
info. Between these two steps, it's possible that the file is removed
by the user, so we'll get an error, leaving stat info empty.
EC's heal daemon was using the gfid blindly, causing an assert failure
when protocol/client was trying to encode the gfid.
To fix the problem a check has been added. If we detect a null gfid, we
simply ignore it and continue healing.
Change-Id: I2e4acdcecd0b6951055e50d1c37d686a2186a228
BUG: 1558016
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Whenever we read data from file over NFS, NFS reads
more data then requested and caches it. Based on the
stat information it makes sure that the cached/pre-read
data is valid or not.
Consider 4 + 2 EC volume and all the bricks are on
differnt nodes.
In EC, with round-robin read policy, reads are sent on
different set of data bricks. This way, it balances the
read fops to go on all the bricks and avoid heating UP
(overloading) same set of bricks.
Due to small difference in clock speed, it is possible
that we get minor difference for atime, mtime or ctime
for different bricks. That might cause a different stat
returned to NFS based on which NFS will discard
cached/pre-read data which is actually not changed and
could be used.
Solution:
Change read policy for EC as gfid-hash. That will force
all the read to go to same set of bricks.
Change-Id: I825441cc519e94bf3dc3aa0bd4cb7c6ae6392c84
BUG: 1554743
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Self-heal creates a thread per brick to sweep the index looking for
files that need to be healed. These threads are started before the
volume comes online, so nothing is done but waiting for the next
sweep. This happens once per minute.
When a replace brick command is executed, the new graph is loaded and
all index sweeper threads started. When all bricks have reported, a
getxattr request is sent to the root directory of the volume. This
causes a heal on it (because the new brick doesn't have good data),
and marks its contents as pending to be healed. This is done by the
index sweeper thread on the next round, one minute later.
This patch solves this problem by waking all index sweeper threads
after a successful check on the root directory.
Additionally, the index sweep thread scans the index directory
sequentially, but it might happen that after healing a directory entry
more index entries are created but skipped by the current directory
scan. This causes the remaining entries to be processed on the next
round, one minute later. The same can happen in the next round, so
the heal is running in bursts and taking a lot to finish, specially
on volumes with many directory levels.
This patch solves this problem by immediately restarting the index
sweep if a directory has been healed.
Change-Id: I58d9ab6ef17b30f704dc322e1d3d53b904e5f30e
BUG: 1547662
Signed-off-by: Xavi Hernandez <jahernan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
ec_link_has_lock_conflict() is traversing over only owner_list
but the function is also getting called with wait_list.
Fix:
Modify ec_link_has_lock_conflict() to traverse lists correctly.
Updated the callers to reflect the changes.
BUG: 1540669
Change-Id: Ibd7ea10f4498e7c2761f9a6faac6d5cb7d750c91
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
| |
Updates #302
Change-Id: I31b4648f7b1a394fceece5cba8120c579c66edd9
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The locks xlator now is able to send a contention notification to
the current owner of the lock.
This is only a notification that can be used to improve performance
of some client side operations that might benefit from extended
duration of lock ownership. Nothing is done if the lock owner decides
to ignore the message and to not release the lock. For forced
release of acquired resources, leases must be used.
Change-Id: I7f1ad32a0b4b445505b09908a050080ad848f8e0
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Existing EC code doesn't try to heal the OpenFD to
avoid unnecessary healing of the data later.
Fix implements the healing of open FDs before
carrying out file operations on them by making an
attempt to open the FDs on required up nodes.
BUG: 1431955
Change-Id: Ib696f59c41ffd8d5678a484b23a00bb02764ed15
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
v1 of the patch started off as adding new fields to iatt that can be
filled up using statx but the discussions were more around introducing
masks to check the validity of different fields from a RIO perspective.
To that extent, I have dropped the statx call in this version and
introduced a 64 bit mask for existing fields. The masks I have defined
are similar with the statx() flags' masks.
I have *not* changed iatt_to_stat() to use the macros IATT_TYPE_VALID,
IATT_GFID_VALID etc before blindly copying from struct iatt to struct.
Also fixed warnings in xlators because of atime/mtime/ctime seconds
field change from uint32_t to int64_t.
Change-Id: I4ac614f1e8d5c8246fc99d5bc2d2a23e7941512b
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
A coverity scan has revelaed a potential shift overflow while scanning
the bitmap of available subvolumes. The actual overflow cannot happen,
but I've changed to test used to control the limit to make it explicit.
Change-Id: Ieb55f010bbca68a1d86a93e47822f7c709a26e83
BUG: 789278
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
At the moment in EC, [f]getxattr operations wait to acquire a lock
while other operations are in progress even when it is in the same mount with a
lock on the file/directory. This happens because [f]getxattr operations
follow the model where the operation is wound on 'k' of the bricks and are
matched to make sure the data returned is same on all of them. This consistency
check requires that no other operations are on-going while [f]getxattr
operations are wound to the bricks. We can perform [f]getxattr in
another way as well, where we find the good_mask from the lock that is already
granted and wind the operation on any one of the good bricks and unwind the
answer after adjusting size/blocks to the parent xlator. Since we are taking
into account good_mask, the reply we get will either be before or after a
possible on-going operation. Using this method, the operation doesn't need to
depend on completion of on-going operations which could be taking long time (In
case of some slow disks and writes are in progress etc). Thus we reduce the
time to serve [f]getxattr requests.
I changed [f]getxattr to dispatch-one and added extra logic in
ec_link_has_lock_conflict() to not have any conflicts for fops with
EC_MINIMUM_ONE as fop->minimum to achieve the effect described above.
Modified scripts to make sure READ fop is received in EC to trigger heals.
Updates gluster/glusterfs#368
Change-Id: I3b4ebf89181c336b7b8d5471b0454f016cdaf296
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
| |
Updates #302
Change-Id: Ifccc6a64c2b2a3b3a0133e6c8042cdf61a0838ce
Signed-off-by: Kamal Mohanan <kmohanan@redhat.com>
|
|
|
|
|
|
|
|
|
| |
This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.
Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1 - This patch fixes a bug in ec_update_stripe()
that prevented some stripes to be updated after a write.
2 - This patch also include code modification for the
case in which a file does not exist and we write on
unaligned offset and user size, the last stripe on
which "end" will fall should also be cached.
Change-Id: I069cb4be1c8d59c206e3b35a6991e1fbdbc9b474
BUG: 1520758
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
DISCARD operation on EC volume was punching hole of lesser
size than the specified size in some cases.
Solution:
EC was not handling punch hole for tail part in some cases.
Updated the code to handle it appropriately.
BUG: 1516206
Change-Id: If3e69e417c3e5034afee04e78f5f78855e65f932
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the volume is being stopped, PARENT_DOWN event is received.
This instructs EC to wait until all pending operations are completed
before declaring itself down. However heal operations are ignored
and allowed to continue even after having said it was down.
This may cause unexpected results and crashes.
To solve this, heal operations are considered exactly equal as any
other operation and EC won't propagate PARENT_DOWN until all
operations, including healing, are complete. To avoid big delays
if this happens in the middle of a big heal, a check has been
added to quit current heal if shutdown is detected.
Change-Id: I26645e236ebd115eb22c7ad4972461111a2d2034
BUG: 1515266
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When there are big directories or files that need to be healed,
other shds are stuck on getting lock on self-heal domain for these
directories/files. If there is a tie-breaker logic, other shds
can heal some other files/directories while 1 of the shds is healing
the big file/directory.
Before this patch:
96.67 4890.64 us 12.89 us 646115887.30us 340869 INODELK
After this patch:
40.76 42.35 us 15.09 us 6546.50us 438478 INODELK
Fixes gluster/glusterfs#354
Change-Id: Ia995b5576b44f770c064090705c78459e543cc64
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
| |
The op-version used for the new option was wrong. It has been set
to 3.13.0.
Change-Id: I88fbd7834e4a8018c8906303e734c251e90be8cf
BUG: 1502610
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
|