| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Objects allocated from a per-thread memory pool keep a reference to it
to be able to return the object to the pool when not used anymore. The
object holding this reference can have a long life cycle that could
survive a glfs_fini() call.
This means that it's unsafe to destroy memory pools from glfs_fini().
Another side effect of destroying memory pools from glfs_fini() is that
the TLS variable that points to one of those pools cannot be reset for
all alive threads. This means that any attempt to allocate memory from
those threads will access already free'd memory, which is very
dangerous.
To fix these issues, mem_pools_fini() doesn't destroy pool lists
anymore. They should be destroyed when the library is unloaded or the
process is terminated, but this cannot be done right now because
gluster doesn't stop other threads before calling exit(), which could
cause some races.
This patch is the backport of 2 master patches:
> Change-Id: Ib189a5510ab6bdac78983c6c65a022e9634b0965
> Fixes: bz#1801684
> Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
>
> Change-Id: Id7cfb4407fcf208e28f03a7c3cdc3ef9c1f3bf9b
> Fixes: bz#1801684
> Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Change-Id: Id7cfb4407fcf208e28f03a7c3cdc3ef9c1f3bf9b
Fixes: bz#1805671
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
ec_getxattr_heal_cbk was called with NULL as second argument
in case heal was failing.
This function was dereferencing "cookie" argument which caused crash.
Solution:
Cookie is changed to carry the value that was supposed to be
stored in fop->data, so even in the case when fop is NULL in error
case, there won't be any NULL dereference.
Thanks to Xavi for the suggestion about the fix.
Change-Id: I0798000d5cadb17c3c2fbfa1baf77033ffc2bb8c
fixes: bz#1806836
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
In a hyperconverged setup with granular-entry-heal enabled, if a file is
recreated while one of the bricks is down, and an index heal is triggered
(with the brick still down), entry-self heal was doing a spurious heal
with just the 2 good bricks. It was doing a post-op leading to removal
of the filename from .glusterfs/indices/entry-changes as well as
erroneous setting of afr xattrs on the parent. When the brick came up,
the xattrs were cleared, resulting in the renamed file not getting
healed and leading to gfid split-brain and EIO on the mount.
Fix:
Proceed with entry heal only when shd can connect to all bricks of the replica,
just like in data and metadata heal.
fixes: bz#1804594
Change-Id: I916ae26ad1fabf259bc6362da52d433b7223b17e
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 06453d77d056fbaa393a137ca277a20e38d2f67e)
|
|
|
|
|
|
|
|
|
|
|
| |
> updates: bz#1712322
> Change-Id: I120a1d23506f9ebcf88c7ea2f2eff4978a61cf4a
> Signed-off-by: Susant Palai <spalai@redhat.com>
> (cherry picked from commit 2bb1807879493cb77ec9b5088485d88f13b84828)
updates: bz#1797985
Change-Id: I120a1d23506f9ebcf88c7ea2f2eff4978a61cf4a
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
When we mount a ta volume, as soon as 2 data bricks are connected
we consider that the mount is done and then send a lookup/create on
ta file on ta node. However, this connection with ta node might not
have been completed.
Due to this delay, ta replica id file will not be created and we
will see ENOTCONN error in log file if we do lookup.
Solution:
As we know that this ta node could have a higher latency, we should
wait for reasonable time for connection to happen before sending
lookup/create on replica id file.
fixes: bz#1804546
Change-Id: I36f90865afe617e4e84cee57fec832a16f5dd6cc
(cherry picked from commit a7fa54ddea3fe429f143b37e4de06a93b49d776a)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of:
> Upstream Patch: https://review.gluster.org/#/c/glusterfs/+/24011/
>fixes: bz#1790748
>Change-Id: I1cb12c975142794139456d0f8e99fbdbb03c53a1
>Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
>(cherry picked from commit d73872e764214f8071c8915536a75bdac1e5e685)
fixes: bz#1790850
Change-Id: I1cb12c975142794139456d0f8e99fbdbb03c53a1
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
glfsheal program uses unix-socket-based volfile server.
volfile server will be the path to socket in this case.
gf_event expects this to be hostname in all cases. So getaddrinfo
will fail on the unix-socket path, events won't be sent in this case.
Fix:
In case of unix sockets, default to localhost
fixes: bz#1793096
Change-Id: I60d27608792c29d83fb82beb5fde5ef4754bece8
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The ec_manager_open/opendir memsets ctx->loc which causes
memory/inode leak, and ec_fheal uses ctx->loc out of fd->lock
that loc_copy may copy bad data when memset it.
This patch skips updating ctx->loc when it is initilizaed.
With it, ctx->loc is filled once, and never updated.
Change-Id: I3bf5ffce4caf4c1c667f7acaa14b451d37a3550a
fixes: bz#1806838
Signed-off-by: Kinglong Mee <mijinlong@horiscale.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
We currently don't have a roll-back/undoing of post-ops if quorum is not met.
Though the FOP is still unwound with failure, the xattrs remain on the disk.
Due to these partial post-ops and partial heals (healing only when 2 bricks
are up), we can end up in metadata split-brain purely from the afr xattrs
point of view i.e each brick is blamed by atleast one of the others for
metadata. These scenarios are hit when there is frequent connect/disconnect
of the client/shd to the bricks.
Fix:
Pick a source based on the xattr values. If 2 bricks blame one, the blamed
one must be treated as sink. If there is no majority, all are sources. Once
we pick a source, self-heal will then do the heal instead of erroring out
due to split-brain.
This patch also adds restriction of all the bricks to be up to perform
metadata heal to avoid any metadata loss.
Removed the test case tests/bugs/replicate/bug-1468279-source-not-blaming-sinks.t
as it was doing metadata heal even when only 2 of 3 bricks were up.
Change-Id: I07a9d62f84ceda329dcab1f02a33aeed258dcb09
fixes: bz#1805097
Signed-off-by: karthik-us <ksubrahm@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: At the time of coming up one server node(1x3) after reboot
client is unmounted.The client is unmounted because a client
is getting AUTH_FAILED event and client call fini for the graph.The
client is getting AUTH_FAILED because brick is not attached with a
graph at that moment
Solution: To avoid the unmounting the client graph throw ENOENT error
from server in case if brick is not attached with server at
the time of authenticate clients.
> Credits: Xavi Hernandez <xhernandez@redhat.com>
> Change-Id: Ie6fbd73cbcf23a35d8db8841b3b6036e87682f5e
> Fixes: bz#1793852
> Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
> (cherry picked from commit > f6421dff22a6ddaf14134f6894deae219948c89d)
Change-Id: Ie6fbd73cbcf23a35d8db8841b3b6036e87682f5e
Fixes: bz#1794020
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Glusterfs client process has memory leak if create serveral files under one folder, and delete the folder.
According to statedump, the ref counts of readdir-ahead is bigger than zero in the inode table. Readdir-ahead get parent inode by inode_parent in rda_mark_inode_dirty when each rda_writev_cbk,the inode ref count of parent folder will be increased in inode_parent, but readdir-ahead do not unref it later.
The correction is unref the parent inode at the end of rda_mark_inode_dirty
Backport of:
> Change-Id: Iee68ab1089cbc2fbc4185b93720fb1f66ee89524
> Fixes: bz#1779055
> Signed-off-by: HuangShujun <549702281@qq.com>
Change-Id: Iee68ab1089cbc2fbc4185b93720fb1f66ee89524
(cherry picked from commit 99044a5cedcff9a9eec40a07ecb32bd66271cd02)
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Fixes: bz#1789337
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
While we delete gluster volume the hook script 'S57glusterfind-delete-post.py'
is failed to execute and error message can be observed in glusterd log.
Traceback:
File "/var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post", line 69, in <module>
main()
File "/var/lib/glusterd/hooks/1/delete/post/S57glusterfind-delete-post", line 39, in main
glusterfind_dir = os.path.join(get_glusterd_workdir(), "glusterfind")
File "/usr/lib64/python3.7/posixpath.py", line 94, in join
genericpath._check_arg_types('join', a, *p)
File "/usr/lib64/python3.7/genericpath.py", line 155, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
Solution:
Added the 'universal_newlines' flag to Popen to support backward compatibility.
Backport of:
> Change-Id: Ie5655b11b55535c5ad2338108d0448e6fdaacf4f
> Fixes: bz#1789478
> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
> (cherry picked from commit 33c3cbe71b67f523538b04334f1ef962953281ed)
Change-Id: Ie5655b11b55535c5ad2338108d0448e6fdaacf4f
Fixes: bz#1790449
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
glusterfind is unable to copy remote output file to local node when a
remove-brick is in progress on the remote node. After copying remote
files, in the --full output listing path, a "sort -u" command is run on
the collected files. However, "sort" exits with an error code if it
finds any file missing.
Solution:
Maintain a map of (pid, output file) when the node commands are started
and remove the mapping for the pid for which the command returns an
error. Use the list of files present in the map for the "sort" command.
Backport of:
> Change-Id: Ie6e019037379f4cb163f24b1c65eb382efc2fb3b
> fixes: bz#1410439
> Signed-off-by: Milind Changire <mchangir@redhat.com>
> Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
> (cherry picked from commit 42c1605f42b89520d4d05806d7074e9e93b63640)
Change-Id: Ie6e019037379f4cb163f24b1c65eb382efc2fb3b
Fixes: bz#1790445
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added logrotate support for user serviceable snapshot's logs.
Backport of:
>Change-Id: Ic920eaa8ab5e44daf5937a027c6913d7bb26d517
>Fixes: bz#1786722
>Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
Change-Id: Ic920eaa8ab5e44daf5937a027c6913d7bb26d517
Fixes: bz#1786754
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The fd processing loops in the
dht_migration_complete_check_task and the
dht_rebalance_inprogress_task functions were unsafe
and could cause an open to be sent on an already freed
fd. This has been fixed.
> Change-Id: I0a3c7d2fba314089e03dfd704f9dceb134749540
> Fixes: bz#1757399
> Signed-off-by: N Balachandran <nbalacha@redhat.com>
> (cherry picked from commit 9b15867070b0cc241ab165886292ecffc3bc0aed)
Change-Id: I0a3c7d2fba314089e03dfd704f9dceb134749540
Fixes: bz#1786983
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
| |
Fixes: bz#1780540
Change-Id: I02ef8c24bc19c321ab2db1d13224ea9e89325e3a
Signed-off-by: hari gowtham <hgowtham@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Current slot allocation/deallocation code path is not
synchronized.There are scenario when due to race condition
in slot allocation/deallocation code path brick is crashed.
Solution: Synchronize slot allocation/deallocation code path to
avoid the issue
> Change-Id: I4fb659a75234218ffa0e5e0bf9308f669f75fc25
> Fixes: bz#1763036
> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
> (cherry picked from commit faf5ac13c4ee00a05e9451bf8da3be2a9043bbf2)
Change-Id: I4fb659a75234218ffa0e5e0bf9308f669f75fc25
Fixes: bz#1778182
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Geo-rep fails to start on python2 only machine like
centos6. It fails with "ImportError no module named _io".
This patch fixes the same.
Backport of:
> Patch: https://review.gluster.org//23702/
> BUG: 1771577
> Change-Id: I8228458a853a230546f9faf29a0e9e0f23b3efec
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 9595ecca3de49fdf37d30b151f5c3e071e0a80d0)
fixes: bz#1771842
Change-Id: I8228458a853a230546f9faf29a0e9e0f23b3efec
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
On a freshly installed system non-root geo-rep test case gets blocked.
Solution:
On a freshly installed system, the remote key need to be accepted automatically by ssh-copy-id.
Credits: M. Scherer <mscherer@redhat.com>
Backport of:
> Change-Id: I5077f99a6681660f7e3e84c25ef216f521b7c29c
> Fixes: bz#1779742
> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
Change-Id: I5077f99a6681660f7e3e84c25ef216f521b7c29c
Fixes: bz#1784796
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When __socket_proto_state_machine() detected a problem in the size of
the request or it couldn't allocate an iobuf of the requested size, it
returned -ENOMEM (-12). However the caller was expecting only -1 in
case of error. For this reason the error passes undetected initially,
adding back the socket to the epoll object. On further processing,
however, the error is finally detected and the connection terminated.
Meanwhile, another thread could receive a poll_in event from the same
connection, which could cause races with the connection destruction.
When this happened, the process crashed.
To fix this, all error detection conditions have been hardened to be
more strict on what is valid and what not. Also, we don't return
-ENOMEM anymore. We always return -1 in case of error.
An additional change has been done to prevent destruction of the
transport object while it may still be needed.
Backport of:
> Change-Id: I6e59cd81cbf670f7adfdde942625d4e6c3fbc82d
> Fixes: bz#1782495
> Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Change-Id: I6e59cd81cbf670f7adfdde942625d4e6c3fbc82d
Fixes: bz#1749625
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: After Configure the Cgroup(CPU/MEM) limit to a gluster processes
resource(CPU/MEM) limits are not applicable to the gluster
processes.Cgroup limits are not applicable because all threads are
not moved into a newly created cgroup to apply restriction.
Solution: To move a gluster thread to newly created cgroup change the
condition in script
> Change-Id: I8ad81c69200e4ec43a74f6052481551cf835354c
> Fixes: bz#1764208
> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
> (cherry picked from commit f5811979935ce607391825ac6913a95f588818e3)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/23599/)
Change-Id: I8ad81c69200e4ec43a74f6052481551cf835354c
Fixes: bz#1766425
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Issue:
libgfchangelog.so: cannot open shared object file
Due to hardcoded shared library name runtime loader looks for particular version of
a shared library.
Solution:
Using find_library to locate shared library at runtime solves this issue.
Traceback (most recent call last):
File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 323, in main
func(args)
File "/usr/libexec/glusterfs/python/syncdaemon/subcmds.py", line 82, in subcmd_worker
local.service_loop(remote)
File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1261, in service_loop
changelog_agent.init()
File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 233, in __call__
return self.ins(self.meth, *a)
File "/usr/libexec/glusterfs/python/syncdaemon/repce.py", line 215, in __call__
raise res
OSError: libgfchangelog.so: cannot open shared object file: No such file or directory
Backport of:
> Patch: https://review.gluster.org/22557
> Change-Id: I3dd013d701ed1cd99ba7ef20d1898f343e1db8f5
> BUG: 1699394
> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit f316c8b797283818bd800569771870a4b9bf1310)
Change-Id: I3dd013d701ed1cd99ba7ef20d1898f343e1db8f5
fixes: bz#1770100
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When iot_worker terminates, its resources have not been reaped, which
will consumes lots of memory.
Detach iot_worker to automically release its resources back to the
system.
Change-Id: I71fabb2940e76ad54dc56b4c41aeeead2644b8bb
fixes: bz#1768726
Signed-off-by: Liguang Li <liguang.lee6@gmail.com>
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
When discard/truncate performs write fop, it should do so
after updating lock->good_mask to make sure readv happens
on the correct mask
fixes: bz#1739449
Change-Id: Idfef0bbcca8860d53707094722e6ba3f81c583b7
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
when a file needs to be re-opened O_APPEND and O_EXCL
flags are not filtered in EC.
- O_APPEND should be filtered because EC doesn't send O_APPEND below EC for
open to make sure writes happen on the individual fragments instead of at the
end of the file.
- O_EXCL should be filtered because shd could have created the file so even
when file exists open should succeed
- O_CREAT should be filtered because open happens with gfid as parameter. So
open fop will create just the gfid which will lead to problems.
Fix:
Filter out these two flags in reopen.
Change-Id: Ia280470fcb5188a09caa07bf665a2a94bce23bc4
Fixes: bz#1739450
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
There are cases where fop->mask may have fop->healing added
and readv shouldn't be wound on fop->healing. To avoid this
always wind readv to lock->good_mask
updates: bz#1739449
Change-Id: I2226ef0229daf5ff315d51e868b980ee48060b87
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
| |
If lock has info, fop should inherit healing mask from it.
Otherwise, fop cannot inherit right healing when changed_flags is zero.
Change-Id: Ife80c9169d2c555024347a20300b0583f7e8a87f
updates: bz#1739449
Signed-off-by: Kinglong Mee <mijinlong@horiscale.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Race:
Thread-1 Thread-2
1) Does ec_get_size_version() to perform
pre-op fxattrop as part of write-1
2) Calls ec_set_dirty_flag() in
ec_get_size_version() for write-2.
This sets dirty[] to 1
3) Completes executing
ec_prepare_update_cbk leading to
ctx->dirty[] = '1'
4) Takes LOCK(inode->lock) to check if there are
any flags and sets dirty-flag because
lock->waiting_flag is 0 now. This leads to
fxattrop to increment on-disk dirty[] to '2'
At the end of the writes the file will be marked for heal even when it doesn't need heal.
Fix:
Perform ec_set_dirty_flag() and other checks inside LOCK() to prevent dirty[] to be marked
as '1' in step 2) above
Updates bz#1739446
Change-Id: Icac2ab39c0b1e7e154387800fbededc561612865
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
|
|
|
|
|
|
|
| |
Fixes: bz#1762237
Change-Id: If339505ca0436947b814d9b0bc0b78f92d5d3317
Signed-off-by: hari gowtham <hgowtham@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Export of env variable is required for ssh-copy-id command.
Backport of:
> fixes: bz#1765426
> Change-Id: Icaf7a848cb8f4ae9f887d885a8c5bb71f26633b4
> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
fixes: bz#1765433
Change-Id: Icaf7a848cb8f4ae9f887d885a8c5bb71f26633b4
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit bfebfa9f2ec9dfc5dbf4a68c3518f98364ebc461)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
While syncing rename of directory in hybrid crawl, geo-rep
crashes as below.
Traceback (most recent call last):
File "/usr/local/libexec/glusterfs/python/syncdaemon/repce.py", line 118, in worker
res = getattr(self.obj, rmeth)(*in_data[2:])
File "/usr/local/libexec/glusterfs/python/syncdaemon/resource.py", line 588, in entry_ops
src_entry = get_slv_dir_path(slv_host, slv_volume, gfid)
File "/usr/local/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 687, in get_slv_dir_path
[ENOENT], [ESTALE])
File "/usr/local/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap
return call(*arg)
PermissionError: [Errno 13] Permission denied: '/bricks/brick1/b1/.glusterfs/8e/c0/8ec0fcd4-d50f-4a6e-b473-a7943ab66640'
Cause:
Conversion of gfid to path for a directory uses readlink on backend
.glusterfs gfid path. But this fails for non root user with
permission denied.
Fix:
Use gfid2path interface to get the path from gfid
Backport of:
> Patch: https://review.gluster.org/23570
> Change-Id: I9d40c713a1b32cea95144cbc0f384ada82972222
> BUG: 1763439
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
Change-Id: I9d40c713a1b32cea95144cbc0f384ada82972222
fixes: bz#1764183
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
EC doesn't allow concurrent writes on overlapping areas, they are
serialized. However non-overlapping writes are serviced in parallel.
When a write is not aligned, EC first needs to read the entire chunk
from disk, apply the modified fragment and write it again.
The problem appears on sparse files because a write to an offset
implicitly creates data on offsets below it (so, in some way, they
are overlapping). For example, if a file is empty and we read 10 bytes
from offset 10, read() will return 0 bytes. Now, if we write one byte
at offset 1M and retry the same read, the system call will return 10
bytes (all containing 0's).
So if we have two writes, the first one at offset 10 and the second one
at offset 1M, EC will send both in parallel because they do not overlap.
However, the first one will try to read missing data from the first chunk
(i.e. offsets 0 to 9) to recombine the entire chunk and do the final write.
This read will happen in parallel with the write to 1M. What could happen
is that half of the bricks process the write before the read, and the
half do the read before the write. Some bricks will return 10 bytes of
data while the otherw will return 0 bytes (because the file on the brick
has not been expanded yet).
When EC tries to recombine the answers from the bricks, it can't, because
it needs more than half consistent answers to recover the data. So this
read fails with EIO error. This error is propagated to the parent write,
which is aborted and EIO is returned to the application.
The issue happened because EC assumed that a write to a given offset
implies that offsets below it exist.
This fix prevents the read of the chunk from bricks if the current size
of the file is smaller than the read chunk offset. This size is
correctly tracked, so this fixes the issue.
Also modifying ec-stripe.t file for Test #13 within it.
In this patch, if a file size is less than the offset we are writing, we
fill zeros in head and tail and do not consider it strip cache miss.
That actually make sense as we know what data that part holds and there is
no need of reading it from bricks.
Backport of:
> Patch:https://review.gluster.org/#/c/glusterfs/+/23066/
> Change-Id: Ic342e8c35c555b8534109e9314c9a0710b6225d6
> BUG: 1730715
> Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
(cherry picked from commit b01a43586c5abc23a874e5528a063c508f952cbd)
Change-Id: Ic342e8c35c555b8534109e9314c9a0710b6225d6
Fixes: bz#1739451
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
|
|
|
|
|
|
|
| |
Change-Id: I0cebaaf55c09eb1fb77a274268ff564e871b743b
fixes bz#1737141
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
(cherry picked from commit 51237eda7c4b3846d08c5d24d1e3fe9b7ffba1d4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After upgrade, if the config files are of old format, it
gets migrated to new format. Monitor process migrates it.
Since monitor doesn't run on nodes where bricks are not
hosted, it doesn't get migrated there. So this patch fixes
the config upgrade on nodes which doesn't host bricks.
This happens during config either on get/set/reset.
Backport of:
> Patch: https://review.gluster.org/23555
> Change-Id: Ibade2f2310b0f3affea21a3baa1ae0eb71162cba
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
> BUG: 1762220
Change-Id: Ibade2f2310b0f3affea21a3baa1ae0eb71162cba
fixes: bz#1763028
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added test case for non-root geo-rep setup.
Backport of:
> Patch: https://review.gluster.org/22902
> Change-Id: Ib6ebee79949a9f61bdc5c7b5e11b51b262750e98
> BUG: 1717827
> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
Change-Id: Ib6ebee79949a9f61bdc5c7b5e11b51b262750e98
fixes: bz#1764178
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background:
The processed changelogs are archived each month in a single tar file.
The default format is "archive_YYYYMM.tar" which is specified as "%%Y%%m"
in configuration file.
Problem:
The created changelog archive file didn't have corresponding year
and month. It created as "archive_%Y%m.tar" on python2 only systems.
Cause and Fix:
Geo-rep expects "%Y%m" after the ConfigParser reads it from config file.
Since it was "%%Y%%m" in config file, geo-rep used to get correct value
"%Y%m" in python3 and "%%Y%%m" in python2 which is incorrect.
The fix can be to use "%Y%m" in config file but that fails in python3.
So the fix is to use "RawConfigParser" in geo-rep and use "%Y%m". This
works both in python2 and python3.
Backport of:
> Patch: https://review.gluster.org/23248
> Change-Id: Ie5b7d2bc04d0d53cd1769e064c2d67aaf95d557c
> BUG: 1741890
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
Change-Id: Ie5b7d2bc04d0d53cd1769e064c2d67aaf95d557c
fixes: bz#1764176
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When two threads(sync jobs) in Geo-rep worker calls `gconf.get` and
`gconf.getr`(realtime) at the sametime, `getr` resets the conf object
and other one gets None. Thread Lock is introduced to fix the issue.
```
File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py",
line 368, in twrap
tf(*aargs)
File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1987,
in syncjob
po = self.sync_engine(pb, self.log_err)
File "/usr/libexec/glusterfs/python/syncdaemon/resource.py",
line 1444, in rsync
rconf.ssh_ctl_args + \
AttributeError: 'NoneType' object has no attribute 'split'
```
Backport of:
> Patch: https://review.gluster.org/23158
> Change-Id: I9c245e5c36338265354e158f5baa32b119eb2da5
> BUG: 1737484
> Signed-off-by: Aravinda VK <avishwan@redhat.com>
Change-Id: I9c245e5c36338265354e158f5baa32b119eb2da5
fixes: bz#1764174
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added test case for the patch
https://review.gluster.org/#/c/glusterfs/+/22894/4
Also updated if else structure in gsyncdconfig.py to avoid
repeated occurance of values in new configfile.
Backport of:
> Patch: https://review.gluster.org/22982
> BUG: 1707731
> Change-Id: If97e1d37ac52dbd17d47be6cb659fc5a3ccab6d7
> Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
fixes: bz#1764171
Change-Id: If97e1d37ac52dbd17d47be6cb659fc5a3ccab6d7
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
gluster command not found.
Cause:
In Volinfo class we issue command 'gluster vol info' to get information
about volume like getting brick_root to perform various operation.
When geo-rep session is configured for non-root user Volinfo class
fails to issue gluster command due to unavailability of gluster
binary path for non-root user.
Solution:
Use config value 'slave-gluster-command-dir'/'gluster-command-dir' to get path
for gluster command based on caller.
Backport of:
> Patch: https://review.gluster.org/22920
> BUG: 1722740
> Change-Id: I4ec46373da01f5d00ecd160c4e8c6239da8b3859
> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
fixes: bz#1764172
Change-Id: I4ec46373da01f5d00ecd160c4e8c6239da8b3859
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- configuration handling is enhanced with patch
https://review.gluster.org/#/c/glusterfs/+/18257/
- hence, the old configurations are not applied when
Geo-rep session is created in the old version and upgraded.
This patch solves the issue. It,
- checks if the config file is old.
- parses required values from old config file and stores in new
config file, which ensures that configerations are applied on
upgrade.
- stores old config file as backup.
- handles changes in options introduced in
https://review.gluster.org/#/c/glusterfs/+/18257/
Backport of:
> Patch: https://review.gluster.org/22894
> BUG: bz#1707731
> Change-Id: Iad8da6c1e1ae8ecf7c84dfdf8ea3ac6966d8a2a0
> Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
updates: bz#1764171
Change-Id: Iad8da6c1e1ae8ecf7c84dfdf8ea3ac6966d8a2a0
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem : When a file is migrated, dht attempts to re-open all open
fds on the new cached subvol. Earlier, if dht had not opened the fd,
the client xlator would be unable to find the remote fd and would
fall back to using an anon fd for the fop. That behavior changed with
https://review.gluster.org/#/c/glusterfs/+/15804, causing fops to fail
with EBADFD if the fd was not available on the cached subvol.
The client xlator returns EBADFD if the remote fd is not found but
dht only checks for EBADF before re-opening fds on the new cached subvol.
Solution: Handle EBADFD at dht code path to avoid the issue
>Change-Id: I43c51995cdd48d05b12e4b2889c8dbe2bb2a72d8
>Fixes: bz#1758579
>(cherry picked from commit 9314a9fbf487614c736cf6c4c1b93078d37bb9df)
>(Reviewed on upstream https://review.gluster.org/#/c/glusterfs/+/23518/)
Change-Id: I43c51995cdd48d05b12e4b2889c8dbe2bb2a72d8
Fixes: bz#1761907
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
In a situation where B1 blames B2, B2 blames B1 and B3 doesn't blame
anything for entry heal, heal will not complete even though we have
clear source and sinks. This will happen because while doing
afr_selfheal_find_direction() only the bricks which are blamed by
non-accused bricks are considered as sinks. Later in
__afr_selfheal_entry_finalize_source() when it tries to mark all the
non-sources as sinks it fails to do so because there won't be any
healed_sinks marked, no witness present and there will be a source.
Fix:
If there is a source and no healed_sinks, then reset all the locked
sources to 0 and healed sinks to 1 to do conservative merge.
Change-Id: If40d8bc95d52a52b2730f55bdcf135109b421548
Fixes: bz#1760706
Signed-off-by: karthik-us <ksubrahm@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ever since we added quorum checks for lookups in afr via commit
bd44d59741bb8c0f5d7a62c5b1094179dd0ce8a4, the split-brain resolution
commands would not work for replica 3 because there would be no
readables for the lookup fop.
The argument was that split-brains do not occur in replica 3 but we do
see (data/metadata) split-brain cases once in a while which indicate that there are
a few bugs/corner cases yet to be discovered and fixed.
Fortunately, commit 8016d51a3bbd410b0b927ed66be50a09574b7982 added
GF_CLIENT_PID_GLFS_HEALD as the pid for all fops made by glfsheal. If we
leverage this and allow lookups in afr when pid is GF_CLIENT_PID_GLFS_HEALD,
split-brain resolution commands will work for replica 3 volumes too.
Likewise, the check is added in shard_lookup as well to permit resolving
split-brains by specifying "/.shard/shard-file.xx" as the file name
(which previously used to fail with EPERM).
Change-Id: I3c543dea79caf7cfbc1633e9089cb1cdd2538ba9
Fixes: bz#1760792
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 47dbd753187f69b3835d2e42fdbe7485874c4b3e)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We need to safe-guard against possible zero'ing out of iatt
structure in acl ctx, which can cause many issues.
> fixes: bz#1668286
> Change-Id: Ie81a57d7453a6624078de3be8c0845bf4d432773
> Signed-off-by: Amar Tumballi <amarts@redhat.com>
> (cherry picked from commit 6bf9637a93011298d032332ca93009ba4e377e46)
Change-Id: I992d25f1c1282d50aa0232d01586d2df2216551c
fixes: bz#1741402
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
|
|
|
|
|
|
|
|
| |
$subject
Change-Id: Ifd8e3340ae6868ea1fd2d0b53f02b093cc23f2cb
Fixes: bz#1760361
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
WB saves the wb_inode in frame->local for the truncate and
ftruncate fops. This value is not cleared in case of error
on a conflicting write request. FRAME_DESTROY finds a non-null
frame->local and tries to free it using mem_put. However,
wb_inode is allocated using GF_CALLOC, causing the
process to crash.
credit: vpolakis@gmail.com
Change-Id: I217f61470445775e05145aebe44c814731c1b8c5
fixes: bz#1755679
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
| |
Change-Id: Id7e003e4a53d0a0057c1c84e1cd704c80a6cb015
fixes: bz#1753571
Signed-off-by: N Balachandran <nbalacha@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After add-brick and rebalance, the ctime xattr is not present
on rebalanced directories on new brick. This patch fixes the
same.
Note that ctime still doesn't support consistent time across
distribute sub-volume.
This patch also fixes the in-memory inconsistency of time attributes
when metadata is self healed.
Backport of:
> Patch: https://review.gluster.org/23127
> Change-Id: Ia20506f1839021bf61d4753191e7dc34b31bb2df
> BUG: 1734026
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
Patch: https://review.gluster.org/23127
Change-Id: Ia20506f1839021bf61d4753191e7dc34b31bb2df
fixes: bz#1752413
Signed-off-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Unable to setup mountbroker root directory while creating geo-replication
session for non-root user.
Casue:
With patch[1] which defines the max-port for glusterd one extra sapce
got added in field of 'option max-port'.
[1]. https://review.gluster.org/#/c/glusterfs/+/21872/
In geo-rep spliting of key-value pair form vol file was done on the
basis of space so this additional space caused "ValueError: too many values
to unpack".
Solution:
Use split so that it can treat consecutive whitespace as a single separator.
Backport of https://review.gluster.org/#/c/glusterfs/+/22716/
>Fixes: bz#1709248
>Change-Id: Ia22070a43f95d66d84cb35487f23f9ee58b68c73
>Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
>(cherry picked from commit 3dd03146bb7037ae2ebea0579d0b81be27fdd927)
Fixes: bz#1750228
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
Change-Id: Ia22070a43f95d66d84cb35487f23f9ee58b68c73
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Right now we have two separate APIs, one
- 'glfs_h_creat_handle' to create handle & another
- 'glfs_h_open' to create a glfd to return to application
Having two separate routines can result in access errors
while trying to create and write into a read-only file.
Since a fd is opened even during file/directory creation,
introducing a new API to make these two operations atomic i.e,
which can create both handle & fd and pass them to application
This is backport of below mainline patch -
- https://review.gluster.org/#/c/glusterfs/+/23448/
- bz#1753569
Change-Id: Ibf513fcfcdad175f4d7eb6fa7a61b8feec6d33b5
fixes: bz#1755785
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
|