| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add error logs if gf_history_changelog fails. If requested
changelog range is not available, log the error and exit
instead of continuing the loop and exiting in readdir
without logging. Also fixed the duplicate MSGID number in
'changelog-lib-messages.h'
> Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd
> BUG: 1362151
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
> Reviewed-on: http://review.gluster.org/15064
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
> Reviewed-by: Aravinda VK <avishwan@redhat.com>
Change-Id: Icd71b89ae23b48a71380657ba5649029c32fabfd
BUG: 1365877
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/15139
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/13658
PROBLEM:
1. Freeing up rpc_clnt object might lead to crashes. Well,
it was not a necessity to free rpc-clnt object till now
because all the existing use cases needs to reconnect
back on disconnects. Hence timer code was not taking
ref on rpc-clnt object.
Glusterd had some use-cases that led to crash due to
ping-timer and they fixed only those code paths that
involve ping-timer.
Now, since changelog has an use-case where rpc-clnt
need to be freed up, we need to fix timer code to take
refs
2. In changelog, because of issue 1, only mydata was being
freed which is incorrect. And there are races where
rpc-clnt object would access the freed mydata which
would lead to crashes.
Since changelog xlator resides on brick side and is long
living process, if multiple libgfchangelog consumers
register to changelog and disconnect/reconnect mulitple
times, it would result in leak of 'rpc-clnt' object
for every connect/disconnect.
SOLUTION:
1. Handle ref/unref of 'rpc_clnt' structure in timer
functions properly.
2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
after disabling timers and free mydata on RPC_CLNT_DESTROY.
RPC SETUP IN CHANGELOG:
1. changelog xlator initiates rpc server say 'changelog_rpc_server'
2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
4. In return changelog_rpc_server initiates a rpc client and connects back
to 'libgfchangelog_rpc_server'
REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
1. Take the ref before reigstering callback to timer queue
>>>> rpc_clnt_ref (say ref count becomes = 2)
2. Register a callback to timer say 'callback1'
3. If register fails:
>>>> rpc_clnt_unref (ref count = 1)
4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
in 'callback1'. This is corresponding to ref taken in step 1
>>>> rpc_clnt_unref (ref count = 1)
5. The cycle from step-1 to step-4 continues....until timer cancel event happens
6. timer cancel of say 'callback1'
If timer cancel fails:
Do nothing, Step-4 would have unrefd
If timer cancel succeeds:
>>>> rpc_clnt_unref (ref count = 1)
Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359363
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14993
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/14272
The barrier notification mechanism was fd based and 'select'
was being used. 'select' breaks when number of fds opened
by brick process exceeds 1024. To avoid this and also
the maintainance of pipe between notify and 'changelog_rollover',
the pipe has been replaced with pthread condition signal
and timed wait mechanism.
Change-Id: I530ea90d9a06953f8b23b4e12d122872ee1925de
BUG: 1341952
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/14615
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/#/c/13845/
If 'changelog' is enabled and 'changelog.capture-del-path' option is on
it calls 'resolve_pargfid_to_path' which modifies 'pargfid' sent by
caller. 'changelog_unlink' calls this routine directly with
'loc->pargfid' resulting it being modified and point to root instead of
actual pargfid. This is a nasty bug and could cause the deletion of
entry on root directory instead on actual parent when 'loc->path' is
not present. Hence this fix to make 'pargfid' a const pointer and
'resolve_pargfid' to work on copy of pargfid.
Glusterfind session creation enables these options by default to
capture deleted entry path in changelog.
Thanks Pranith for root causing this.
BUG: 1322552
Change-Id: I9f2bc44b5604b224462594c12b7d79e68198d693
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/13861
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
>Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36
>BUG: 1282761
>Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
>Reviewed-on: http://review.gluster.org/12598
>Tested-by: Gluster Build System <jenkins@build.gluster.com>
>Tested-by: NetBSD Build System <jenkins@build.gluster.org>
>Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
BUG: 1283757
Change-Id: Ic20d4ee031265305db1a6ed2cf591ce94b7d0749
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/12668
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are three kinds of inline functions: plain inline, extern inline,
and static inline. All three have been removed from .c files, except
those in "contrib" which aren't our problem. Inlines in .h files, which
are overwhelmingly "static inline" already, have generally been left
alone. Over time we should be able to "lower" these into .c files, but
that has to be done in a case-by-case fashion requiring more manual
effort. This part was easy to do automatically without (as far as I can
tell) any ill effect.
In the process, several pieces of dead code were flagged by the
compiler, and were removed.
backport of Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155,
http://review.gluster.org/11769, BUG: 1245331
Change-Id: Iba1efb0bc578ea4a5e9bf76b7bd93dc1be9eba44
BUG: 1283302
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/12646
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The master xlator needs to allocate 'logbuf_pool'
else 'gf_msg' fails with EINVAL.
BUG: 1293595
Change-Id: I51a1895b5ff9c8eaf6cc15a9cacc415fa8cd7bdd
Reviewed-on: http://review.gluster.org/12997
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/13064
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GEO-REP INTEROP WITH SHARD FEATURE
shard xlator updates size of the file using FXATTROP
or XATTROP. Hence record the same in changelog.
BUG: 1284453
Change-Id: I2f14b6075f863c0bed3ee2a159085b9b536a756d
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/12225
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/12732
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
changelog-helpers.c:1911:17: warning: Size argument is greater than the free
space in the destination buffer strncat (result, pre_dir_name, PATH_MAX);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
changelog-helpers.c:1919:17: warning: Size argument is greater than the free
space in the destination buffer strncat (result, bname, PATH_MAX);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Backport of:
> Change-Id: I60ca7fe762f07cb72fe7b69f0253835becaff7b9
> BUG: 1222238
> Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
> Reviewed-on: http://review.gluster.org/10802
> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
> Reviewed-by: Venky Shankar <vshankar@redhat.com>
> (cherry picked from commit 09530dfd822c8c3cc8da20a4600b5d2aec1ebf9d)
Change-Id: I46e1bf48b62f95e21f6615ac4afc22032f16f5a2
BUG:1252057
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/12494
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a series of patches which aims to fix geo-replication
in a Tiering Volume.
Problem:
Consider, a file is placed in volume initially and then hot tier is
attached. During any operation on the file, due to lookup a linkto
file is created in hot tier.
Now, any namespace operation carried out on the file is recorded in
both cold and hot tier.
There is a room for races when both changelogs are replayed.
Solution:
So, We are going to replay (namespace related)operations
only in the hot tier.
Why?
a. If the file is directly placed in Hot tier, all fops will be
recorded in HOT tier.
b. If the file is already present in Cold tier, and if any fop is
carried out, it creates linkto file in Hot tier.
Now, operations like UNLINK, RENAME are captured in Hot tier(by means of linkto file).
This way, we can get both tier's operation in HOT tier itself.
But, We may miss initial Data sync immediately after creating the
file as it is only recording MKNOD. So, if MKNOD encountered
with sticky bit set, queue DATA operation for the corresponding gfid.
( This geo-rep related changes are addressed in this patch: http://review.gluster.org/12326/ )
So, If tier-dht linkto is set, we need to record the corresponding
MKNOD. Earlier this was avoided as it was set as INTERNAL fop.
(This is addressed here in this patch)
Change-Id: I25514fe3e25f68592a8d6361507f8c8a4fcb70b1
BUG: 1275173
Reviewed-on: http://review.gluster.org/12417
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/12428
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Recording of tiering rebalance process's fops like Creation
and Deletion of file must be avoided.
Ignore the fops using corresponding pid.
Change-Id: Ifdc7765598d04d033f93e6339e9b188f7566cb65
BUG: 1275173
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/12239/
Reviewed-on: http://review.gluster.org/12425
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The @owner argument tells RPC layer the xlator that owns
the connection and to which xlator THIS needs be set during
network notifications like CONNECT and DISCONNECT.
Code paths that originate from the head of a (volume) graph and use
STACK_WIND ensure that the RPC local endpoint has the right xlator saved
in the frame of the call (callback pair). This guarantees that the
callback is executed in the right xlator context.
The client handshake process which includes fetching of brick ports from
glusterd, setting lk-version on the brick for the session, don't have
the correct xlator set in their frames. The problem lies with RPC
notifications. It doesn't have the provision to set THIS with the xlator
that is registered with the corresponding RPC programs. e.g,
RPC_CLNT_CONNECT event received by protocol/client doesn't have THIS set
to its xlator. This implies, call(-callbacks) originating from this
thread don't have the right xlator set too.
The fix would be to save the xlator registered with the RPC connection
during rpc_clnt_new. e.g, protocol/client's xlator would be saved with
the RPC connection that it 'owns'. RPC notifications such as CONNECT,
DISCONNECT, etc inherit THIS from the RPC connection's xlator.
Change-Id: I9dea2c35378c511d800ef58f7fa2ea5552f2c409
BUG: 1253212
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/11436
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
(cherry picked from commit f7668938cd7745d024f3d2884e04cd744d0a69ab)
Reviewed-on: http://review.gluster.org/11908
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/11148
This patch uses "cleanup, v1" infrastrcuture to cleanup scrubber
(data structures, threads, timers, etc..) on brick disconnection.
Signer is not cleaned up yet: probably would be done as part of
another patch.
Change-Id: I78a92b8a7f02b2f39078aa9a5a6b101fc499fd70
BUG: 1226666
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/11540
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Directory renames are being ignored as special renames. Special
renames can happen only on files. Hence always log directory
rename operations in changelog.
Change-Id: I4fbdb3e02e634a39a8846fb2f7a4c6cc2ba74400
BUG: 1235242
Reviewed-On: http://review.gluster.org/11356
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/11378
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Crash observed in gf_changelog_process and
gf_changelog_callback_invoker.
Cause:
Assignments to arguments passed to thread is done
post thread creation. If the thread created gets
scheduled before the assignment and access these
variables, it would crash with segmentation fault.
Solution:
Assignments to arguments are done prior to the thread
creation.
BUG: 1233044
Change-Id: I520599ab43026d25f4064ce71bd5a8b8e0d4b90a
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11273
Reviewed-on: http://review.gluster.org/11308
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
HTIME_KEY marks the last changelog rolled over. The xattr is
maintained on .glusterfs/changelog/htime/HTIME.TSTAMP file.
On every rollover of the changelog file, the xattr is updated.
It is being updated with XATTR_REPLACE flag as xattr gets
created during changelog enable. But it is once found that
the xattrs on the file is cleared and is not reproduced later
on. This patch protects that case, if it happens by setting
xattr without XATTR_REPLACE flag in failure case.
The reason behind doing this in failure case is not to mask
the actual cause of xattrs getting cleared. This provides
the log message if the original issue still exists but the
consequential effects are fixed.
Also changed the log messages to depict the events happened
during changelog enable.
Change-Id: I699ed09a03667fd823d01d65c9c360fa7bc0e455
BUG: 1230694
Reviewed-On: http://review.gluster.org/#/c/11150/
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/11181
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
When a file is renamed and the (renamed)file's Hashing
falls into a different brick, DHT creates a special file(linkto file)
in the brick(Hashed subvolume) and carries out setattr operation
on that file.
Currently, Changelog records this(setattr) operation in Hashed
subvolume. glusterfind in turn records this operation
as MODIFY operation.
So, there is a NEW entry in Cached subvolume and MODIFY entry
in Hashed subvolume for the same file.
Solution:
Avoid logging setattr operation carried out, by
marking the operation as internal fop using xdata.
In changelog translator, check whether setattr is set
as internal fop and skip accordingly.
Change-Id: I21b09afb5a638b88a4ccb822442216680b7b74fd
BUG: 1230687
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/11183
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Backport of http://review.gluster.org/11004
When compiled with gcc5, following warnings were displayed
and volume start failed:
changelog-helpers.h:499:1: warning: inline function 'changelog_dispatch_event'
declared but never defined
changelog_dispatch_event (xlator_t *, changelog_priv_t *, changelog_event_t *);
gf-changelog-journal-handler.c:692:17: warning: 'list_add_tail' is static but
used in inline function 'gf_changelog_queue_journal' which is not static
list_add_tail (&entry->list, &jnl_proc->entries);
Fix is to remove the keyword from function prototype and
definitions.
Change-Id: I188b35b7ca087a94d7a48a052b05a6d845e3b74b
BUG: 1226853
Signed-off-by: Anoop C S <achiraya@redhat.com>
Reviewed-on: http://review.gluster.org/11030
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During snapshot, changelog barrier is enabled and a
explicit rollover of changelog is initiated. During
rollover of changelog, if any error or changelog is
empty, the notification was not sent to reconfigure
and hence snapshot was failing because of timeout.
This patch addresses it by sending notification
irrespective of failures and sends error if any
back to barrier.
BUG: 1225543
Change-Id: I971ef3bdc63bb50bda0b655e55cd814e44254ba9
Reviewed-On: http://review.gluster.org/10951
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10988
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Background:
Glusterfs changelogs are stored in each brick, which records the changes
happened in that brick. Georep will run in all the nodes of master and processes
changelogs "independently". Processing changelogs is in brick level, but
all the fops will be replayed on "slave mount" point.
Problem:
With a DHT volume, in changelog "internal fops" are NOT recorded.
For Rename case, Rename is recorded in "hashed" brick changelog.
(DHT's internal fops like creating linkto file, unlink is NOT recorded).
This lead us to inconsistent rename operations.
For example,
Distribute volume created with Two bricks B1, B2.
//Consider master volume mounted @ /mnt/master and following operations
executed:
cd /mnt/master
touch f1 // f1 falls on B1 Hash
mv f1 f2 // f2 falls on B2 Hash
// Here, Changelogs are recorded as below:
@B1
CREATE f1
@B2
RENAME f1 f2
Here, race exists between Brick B1 and B2, say B2 will get executed first.
Source file f1 itself is "NOT PRESENT", so it will go ahead and create
f2 (Current implementation).
We have this problem When rename falls in another brick and
file is unlinked in Master.
Similar kind of issue exists in following case too(multiple rename):
CREATE f1
RENAME f1 f2
RENAME f2 f1
Solution:
Instead of carrying out "changelogging" at "HASHED volume",
carry out at the "CACHED volume".
This way we have rename operations carried out where actual files are present.
So,Changelog recorded as :
@B1
CREATE f1
RENAME f1 f2
Note:
This patch is dependent on dht changes from this patch.
http://review.gluster.org/10410/
changelog related changes are separated out for review.
In changelog, xdata passed from DHT is considered as:
1. In case of unlink (internal operation as part of rename), xdata value
is set , it is considered as RENAME and recorded accordingly.
2. In case of rename (Hash and Cache different), xdata value is NOT
set, recording rename operation is SKIPPED.
BUG: 1219412
Change-Id: I7691166c84991482b2cfe073df64e2317c935b13
Reviewed-On: http://review.gluster.org/#/c/10220/
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: http://review.gluster.org/10633
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
BUG: 1219823
Change-Id: I3a1c7b7742671847ed3fec13e06d861c3d09f7a9
Reviewed-On: http://review.gluster.org/#/c/10687/
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10688
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This issue introduced due to manual rebase.
Change-Id: I0589f4a0a1270190340f419b8022d6483bcf853d
Signed-off-by: Aravinda VK <avishwan@redhat.com>
BUG: 1219479
Reviewed-on: http://review.gluster.org/10685
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10686
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An empty changelog when rolled over gets unlinked and indexed with
a modified path-name in htime file. The modification is "changelog"
not "CHANGELOG" in basename of the empty changelog file.
BUG: 1219479
Change-Id: Ib5b825ab563fa34d8dcf4368cf6cbf4b25d78a6d
Original-Author: Ajeet Jha <ajha@redhat.com>
Original-Author: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-On: http://review.gluster.org/#/c/9572/
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10642
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As optional feature, during unlink, full path will be recorded.
Changelog Version number to be bumped up to 1.2.
With this patch, parser checks the version number before parsing
and handles accordingly.
Change-Id: Ic1ad98259c39e417029a08e26a1d4b467817e65a
BUG: 1218383
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10166
Reviewed-on: http://review.gluster.org/10620
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity IDs:
1214630
1214631
1214633
1234643
Backport of: http://review.gluster.org/9557
Change-Id: I172c4f49bf651b2324522f9e661023f73ca05339
BUG: 789278
Signed-off-by: Nandaja Varma <nvarma@redhat.com>
Reviewed-on: http://review.gluster.org/9557
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Sakshi Bansal
Reviewed-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7d7b80efe8c745f3ff7de76fc31c4977098cae01)
Reviewed-on: http://review.gluster.org/10595
Tested-by: NetBSD Build System
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The structure 'rpcsvc_state', which maintains rpc server
state had no separate pointer to track the translator.
It was using the mydata pointer itself. So callers were
forced to send xlator pointer as mydata which is opaque
(void pointer) by function prototype.
'rpcsvc_register_init' is setting svc->mydata with xlator
pointer. 'rpcsvc_register_notify' is overwriting svc->mydata
with mydata pointer. And rpc interprets svc->mydata as
xlator pointer internally. If someone passes non xlator
structure pointer to rpcsvc_register_notify as libgfchangelog
currently does, it might corrupt mydata. So interpreting opaque
mydata as xlator pointer is incorrect as it is caller's choice
to send mydata as any type of data to 'rpcsvc_register_notify'.
Maintaining two different pointers in 'rpcsvc_state' for xlator
and mydata solves the issue.
BUG: 1218381
Change-Id: I4c28937a30845e3f41b6fc7a09036149c816659b
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10366
Reviewed-on: http://review.gluster.org/10534
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PROBLEM:
There is no way to get the path of deleted file if we
have gfid from changelog since the file is already deleted.
SOLUTION:
Do a recursive readlink on parent gfid in backend .glusterfs
path to get the complete path in I/O callpath in changelog
translator and capture it in callback.
The path captured is relative from the brick root. The field
separator used is '\0'.
e.g.,
......\0<pgfid>/bname\0<relative-path>\0<next-record>
ADDITIONAL REQUIRED CHANGES:
1. The changelog translator option called "changelog.capture-del-path"
is introduced to enable or disable the capturing of deleted entry
path.
e.g.,
gluster vol set <vol-name> changelog.capture-del-path on/off
If capture-del-path is disabled, '\0' is captured instead of
relative path.
e.g.,
......\0<pgfid>/bname\0\0\0<next-record>
2. The minor number in the version of changelog is bumped up from v1.1
to v1.2.
3. If recursive readlink is failed for some reason, it will capture
\0 in place of <relative path>.
e.g.,
......\0<pgfid>/bname\0\0\0<next-record>
(same as when caputre-del-path option is disabled)
4. If bname argument passed to "resolve_pargfid_to_path" function
is NULL and pargfid is ROOT, "." is returned. This is not the
case with changelog, where bname is always passed. This is
applicable to other consumers of "resolve_pargfid_to_path"
routine.
NOTE:
Changelog parser should consider the above new changes
and should parse accordingly.
BUG: 1218383
Change-Id: I5d89cf4157befd207771f6c0248d2493fbf85832
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10288
Reviewed-on: http://review.gluster.org/10535
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Earlier, both chagelog on/off and brick restart were considered
to be changelog breakage and treated as changelog not being
continuous. As a result, new HTIME.TSTAMP file was created on
both the above cases. Now the change is made such that only
on changelog enable/disable, the changelog is considered to be
discontinuous. New HTIME.TSTAMP file is not created on brick
restart, the changelogs files are appended to last HTIME.TSTAMP
file.
Treating changelog as continuous in above scenario is important
as changelog history API will fail otherwise. It can successfully
get changes between start and end timestamps only when changelog
is continuous (Changelogs in single HTIME.TSTAMP file are treated
as continuous). Without this change, changelog history API would
fail, and it would become necessary to fallback to other mechanisms
like xsync FSCrawl in case geo-rep to detect changes in this time
window. But Xsync FSCrawl would not be applicable to other
consumers like glusterfind.
Rationale:
1. In plain distributed volume, if brick goes down, no I/O can
happen onto the brick. Hence changelog is intact with data
on disk.
2. In distributed replicate volume, if brick goes down, since
self-heal traffic is captured in changelog. Eventually,
I/O happened whend brick down is captured in changelog.
BUG: 1217944
Change-Id: Ifa6d932818fe1a3a914e87ac84f1d2ded01c1288
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10222
Reviewed-on: http://review.gluster.org/10507
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changelog xlator was capturing bitrot-stub's fsetxattr sent
for versioning. Since it was using the same frame as of the
create fop, there was inconsistency in fop number and gfid
of capturing metadata. So fix is to mark fsetxattr used for
versioning as internal and add internal fop filter in
changelog_fsetxattr.
Change-Id: I51ff468995139838b22bf293a59a0713a92ee7a5
BUG: 1170075
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10148
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.
libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.
Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.
Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288819
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.
Change-Id: Ie2d2970f37840146aa18724be3b89e93194c8160
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10062
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288821
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.
Change-Id: I5ecc60549864cf50e3facbb8a2d9228cd67b57b5
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10061
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.
Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.
A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.
BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When libgfchangelog is linked with non xlator application,
it should point to 'master' xlator which is initiated separately.
When ever a new thread is created, 'THIS' points to the global
xlator. 'THIS' should point to corresponding xlator even then.
This patch adjusts the pointer accordingly.
Change-Id: I2a199bb3c73146a0329540aedcbae697a00f6f0a
BUG: 1207643
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/10074
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288822 (#1 of 2)
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
In this case the strncpy is not needed due to the snprintf
with the same target buffer which follows immediately.
This patch also removes the now unneeded scratch_dir
argument to gf_changelog_init_history(), which is semantically
correct, since scratch_dir has previously been filled into
jnl->jnl_working_dir by the caller, and this is now used to
fill hist_scratch_dir.
Change-Id: Ib1ed3a1058e80e34191758921b49c29030d6c9db
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10058
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I948f85cb369206ee8ce8b8cd5e48cae9adb971c9
BUG: 1075417
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-on: http://review.gluster.org/9529
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Coverity CID 1288822 (#2 of 2)
strncpy executed with a limit equal to the target array
size potentially leaves the target string not null terminated.
Make sure the copied string is a valid 0 terminated string.
Change-Id: If283dd6f716912f65729edf6ea26a4331a697151
BUG: 789278
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-on: http://review.gluster.org/10059
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Poornima G <pgurusid@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CID: 1124481
The goto label 'out' was trying to
dereference a pointer which might be
null, hence a check on not null is
made before the use of it.
Change-Id: I813e523207b8218a7a28c2b3ac36a8e472e36c89
BUG: 789278
Signed-off-by: arao <arao@redhat.com>
Reviewed-on: http://review.gluster.org/9617
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The changelog xlator was modified to use a poller thread, which uses
the same event pool as the xlator stack. Unfortunately, such threaded
usage of event pool is not supported by event-poll code, only
event-epoll supports it. As a result, platforms such as NetBSD that
lack epoll support got broken.
The fix is to remove the poller thread, which does not cause any
functionnality loss because the xlator stack event poll is
functionnal. That lets NetBSD pass AFR tests again.
BUG: 1129939
Change-Id: I3d73cf58e2ed8d92d9e0191f7abda3c37dea4159
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/10030
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the "Signer" -- responsible for signing files with their
checksums upon last file descriptor close (last release()).
The event notification facility provided by the changelog xlator
is made use of.
Moreover, checksums are as of now SHA256 hash of the object data
and is the only available hash at this point of time. Therefore,
there is no special "what hash to use" type check, although it's
does not take much to add various hashing algorithms to sign
objects with. Signatures are stored in extended attributes of the
objects along with the the type of hashing used to calculate the
signature. This makes thing future proof when other hash types
are added. The signature infrastructure is provided by bitrot
stub: a little piece of code that sits over the POSIX xlator
providing interfaces to "get or set" objects signature and it's
staleness.
Since objects are signed upon receiving release() notification,
pre-existing data which are "never" modified would never be
signed. To counter this, an initial crawler thread is spawned
The crawler scans the entire brick for objects that are unsigned
or "missed" signing due to the server going offline (node reboots,
crashes, etc..) and triggers an explicit sign. This would also
sign objects when bit-rot is enabled for a volume and/or after
upgrade.
Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b
BUG: 1170075
Original-Author: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9711
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bitrot stub implements object versioning required for identifying
signature freshness. More details about versioning is explained
as a part of the "bitrot feature documentation" patch.
Change-Id: I2ad70d9eb109ba4a12148ab8d81336afda529ad9
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9709
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Translators which wish to send event notifications can send
"down" an IPC FOP with op_type as GF_IPC_TARGET_CHANGELOG
and xdata carrying event structures (changelog_event_t).
Change-Id: I0e5f8c9170161c186f0e58d07105813e34e18786
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9775
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[WIP patch as of now, just needs a little tweak]
A pending TODO in the code caused regressions to fail as
bitrot daemons are spawned during volume start (equivalent
to enabling bitrot by default). The problematic part that
casued such failures is during brick disconnections with
unsafe handling of event data structured in the code.
With this patch, data structures are properly cleaned up
with care taken to cleanup all accessors first. This also
fixes potential memory leaks which was bluntly ignored
before.
Change-Id: I70ed82cb1a0fb56c85ef390007e321a97a35c5ce
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
original-author: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9959
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
libgfchangelog initializes global xlator on library load (via
constructor: _ctor) and mangles it's xlator context thereby
messing with certain important members of the command structure.
On receiving an RPC disconnection event, if the point-of-execution
was in libgfchangelogs context, accessing ->cmd_args during RPC
notify resulted in a segfault.
Fix:
Since the libarary needs to be able to work with processes that
have a notion of an xlator (THIS in particular) and without it,
care needs to be taken to allocate the global xlator when needed.
Moreover, the actual fix is to use the correct xlator context
in both cases. A new API is introduces when needs to be invoked
by the conusmer (although this could have been done during
register() call, keeping it a separate API makes thing flexible
and easy).
Test:
The issue is observed when a brick process goes offline. This is
triggered when test cases (.t's) are run in bulk, since each
test essestially spawns bricks processes (on volume start) and
terminates them (volume stop). Since bitrot daemon, as of now,
spawns upon volume start, the issue is much observed when the
volume is taken offline at the end of each test case. With this
fix, running the basic and core test cases along with building
the linux kernel has passed without daemon segfaults.
Thanks to Johnny (rabhat@) for helping in debugging the issue
(and with the fix :)).
Change-Id: I8d3022bf749590b2ee816504ed9b1dfccc65559a
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9953
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch introduces RPC based communication between the changelog
translator and libgfchangelog. It replaces the old pathetic stream
based interaction that existed earlier (due to time constraints :-/).
Changelog, upon initialization starts a RPC server (rpcsvc) allowing
clients to invoke a probe API as a bootup mechanism to request for
event notifications. During probe, clients can choose an event
filter specifying the type(s) of events they are interested in. As
of now there is no way to change the event notification set once
the probe RPC call is made, but that is easier to implement.
The actual event notifications is done on a separate RPC session.
The client (libgfchangelog) itself starts and RPC server which the
changelog translator "connects back" during probe. Notifications
are dispatched by a bunch of threads from the server (translator)
and the client optionally orders them if ordered notifications
are requried. FOPs fill in their respective event details in a
buffer (rot-buffs to be particular) and a bunch of threads
(consumers) swap the buffers out of roatation and dispatch them
via RPC. To avoid writer starvation, then number of dispatcher
threads is one less than the number of buffer list in rot-buffs.x
libgfchangelog becomes purely callback based -- upon event
notification from the server (and re-ordering them if required)
invoke a callback routine specified by consumer(s).
A major part of the patch is also aimed at providing backward
compatibility for geo-replication, which was one of the main
consumer of the stream based API. Also, this patch does not\
"turn on" event notifications for all fops, just a bunch which
is currently in requirement. Another pain point is that the
server does not filter events before dispatching it to the
clients. That load is taken up by the client itself (although
it's done at the library layer rather than making it hard on
the callback implementor). This needs improvement and care
needs to be taken to not load the server up with expensive
filtering mechanisms.
Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395
BUG: 1170075
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/9708
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A gf_history_changelog_next_change() calls gf_readline() to fill a
buffer without checking buffer size. The size of maxlen is not verified
to be less than the lenght of buffer. This could result in the over
filling of buffer of maxlen is greater than PATH_MAX.
Check the size of maxlen to be less than PATH_MAX and return a fail code
as needed.
BUG: 1174017
Change-Id: Ic53b1a6e25af69a339bc15fb2d233dc1e457910f
Reported-by: Keith Schincke <kschinck@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9275
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes bad file descriptor issue while cleaning up
scratch directory during gf_changelog_register.
Change-Id: Ia6aa8d55dcc2209144b48b6583681a155d919c42
BUG: 1162057
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9495
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On changelog_register cleanup .processing, .history/.processing,
.current and .history/.current from the working directory.
Moved glusterd_recursive_rmdir and glusterd_for_each_entry to common
place(libglusterfs) and renamed as recursive_rmdir and
GF_FOR_EACH_ENTRY_IN_DIR respectively
BUG: 1162057
Change-Id: I1f98468a344cead039026762a805437b2f9e507b
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/9082
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is an improvement over the patch 'http://review.gluster.org/9337'
to trigger explicit geo-rep sync on regular files even if entry is not
present on the slave. An attempt is made to find the pargfid and
if available captures CREATE along with DATA in changelog.
CREATE is captured with default file permissions. Setting this virtual
setxattr on directories captures MKDIR in changelog. The value of
setxattr can be as follows.
If value = "1" : Both CREATE and DATA is captured in changelog if
pargfid is available, else on DATA is captured.
value = "any other: ENOTSUP is returned.
Usage:
setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <file-path>
NOTE: This patch supports explicit record of entries only for
directories and regular files.
Change-Id: Iedde8b2c8bc3b78db524050d8c866ff664811d01
BUG: 1176934
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9370
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A virtual xattr "glusterfs.geo-rep.trigger-sync" is provided
in glusterfs through changelog translator. Geo-rep triggers
a explicit data sync on setting this xattr on a file.
Changelog captures a DATA entry on file's gfid on setting this
virtual xattr on a file. This is supported only for files. It
doesn't support directories.
Usage: setfattr -n glusterfs.geo-rep.trigger-sync <file-path>
Change-Id: Ia689326ac2dcb31035ffbecad2c548eda4eb9245
BUG: 1176934
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9337
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
|