glusterfs.git/xlators/features/changelog/src, branch v3.7.4

rpc: add owner xlator argument to rpc_clnt_new

2015-08-14T10:10:47+00:00

The @owner argument tells RPC layer the xlator that owns
the connection and to which xlator THIS needs be set during
network notifications like CONNECT and DISCONNECT.

Code paths that originate from the head of a (volume) graph and use
STACK_WIND ensure that the RPC local endpoint has the right xlator saved
in the frame of the call (callback pair). This guarantees that the
callback is executed in the right xlator context.

The client handshake process which includes fetching of brick ports from
glusterd, setting lk-version on the brick for the session, don't have
the correct xlator set in their frames. The problem lies with RPC
notifications. It doesn't have the provision to set THIS with the xlator
that is registered with the corresponding RPC programs. e.g,
RPC_CLNT_CONNECT event received by protocol/client doesn't have THIS set
to its xlator. This implies, call(-callbacks) originating from this
thread don't have the right xlator set too.

The fix would be to save the xlator registered with the RPC connection
during rpc_clnt_new. e.g, protocol/client's xlator would be saved with
the RPC connection that it 'owns'. RPC notifications such as CONNECT,
DISCONNECT, etc inherit THIS from the RPC connection's xlator.

Change-Id: I9dea2c35378c511d800ef58f7fa2ea5552f2c409
BUG: 1253212
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/11436
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
(cherry picked from commit f7668938cd7745d024f3d2884e04cd744d0a69ab)
Reviewed-on: http://review.gluster.org/11908
Reviewed-by: Pranith Kumar Karampuri

features/bitrot: cleanup, v2

2015-07-09T10:13:25+00:00

    Backport of http://review.gluster.org/11148

This patch uses "cleanup, v1" infrastrcuture to cleanup scrubber
(data structures, threads, timers, etc..) on brick disconnection.
Signer is not cleaned up yet: probably would be done as part of
another patch.

Change-Id: I78a92b8a7f02b2f39078aa9a5a6b101fc499fd70
BUG: 1226666
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/11540
Reviewed-by: Raghavendra Bhat 
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System

features/changelog: Always log directory rename operations

2015-06-28T14:45:36+00:00

Directory renames are being ignored as special renames. Special
renames can happen only on files. Hence always log directory
rename operations in changelog.

Change-Id: I4fbdb3e02e634a39a8846fb2f7a4c6cc2ba74400
BUG: 1235242
Reviewed-On: http://review.gluster.org/11356
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/11378
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR

features/changelog: Do htime setxattr without XATTR_REPLACE flag

2015-06-12T18:37:43+00:00

HTIME_KEY marks the last changelog rolled over. The xattr is
maintained on .glusterfs/changelog/htime/HTIME.TSTAMP file.
On every rollover of the changelog file, the xattr is updated.
It is being updated with XATTR_REPLACE flag as xattr gets
created during changelog enable. But it is once found that
the xattrs on the file is cleared and is not reproduced later
on. This patch protects that case, if it happens by setting
xattr without XATTR_REPLACE flag in failure case.

The reason behind doing this in failure case is not to mask
the actual cause of xattrs getting cleared. This provides
the log message if the original issue still exists but the
consequential effects are fixed.

Also changed the log messages to depict the events happened
during changelog enable.

Change-Id: I699ed09a03667fd823d01d65c9c360fa7bc0e455
BUG: 1230694
Reviewed-On: http://review.gluster.org/#/c/11150/
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/11181
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog: Avoid setattr fop logging during rename

2015-06-12T05:26:16+00:00

Problem:
        When a file is renamed and the (renamed)file's Hashing
falls into a different brick, DHT creates a special file(linkto file)
in the brick(Hashed subvolume) and carries out setattr operation
on that file.

Currently, Changelog records this(setattr) operation in Hashed
subvolume. glusterfind in turn records this operation
as MODIFY operation.

So, there is a NEW entry in Cached subvolume and MODIFY entry
in Hashed subvolume for the same file.

Solution:
        Avoid logging setattr operation carried out, by
marking the operation as internal fop using xdata.

In changelog translator, check whether setattr is set
as internal fop and skip accordingly.

Change-Id: I21b09afb5a638b88a4ccb822442216680b7b74fd
BUG: 1230687
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/11183
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

features/changelog: Remove inline keyword to avoid warnings (gcc v5.1.1)

2015-06-01T13:56:04+00:00

        Backport of http://review.gluster.org/11004

When compiled with gcc5, following warnings were displayed
and volume start failed:

changelog-helpers.h:499:1: warning: inline function 'changelog_dispatch_event'
declared but never defined
 changelog_dispatch_event (xlator_t *, changelog_priv_t *, changelog_event_t *);

gf-changelog-journal-handler.c:692:17: warning: 'list_add_tail' is static but
used in inline function 'gf_changelog_queue_journal' which is not static
                 list_add_tail (&entry->list, &jnl_proc->entries);

Fix is to remove the keyword from function prototype and
definitions.

Change-Id: I188b35b7ca087a94d7a48a052b05a6d845e3b74b
BUG: 1226853
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/11030
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar

featuress/changelog: On snapshot, notify irrespective of failures

2015-06-01T02:39:24+00:00

During snapshot, changelog barrier is enabled and a
explicit rollover of changelog is initiated. During
rollover of changelog, if any error or changelog is
empty, the notification was not sent to reconfigure
and hence snapshot was failing because of timeout.
This patch addresses it by sending notification
irrespective of failures and sends error if any
back to barrier.

BUG: 1225543
Change-Id: I971ef3bdc63bb50bda0b655e55cd814e44254ba9
Reviewed-On: http://review.gluster.org/10951
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10988
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar

geo-rep: rename handling in dht volume(changelog changes)

2015-05-09T09:08:10+00:00

Background:

Glusterfs changelogs are stored in each brick, which records the changes
happened in that brick. Georep will run in all the nodes of master and processes
changelogs "independently".  Processing changelogs is in brick level, but
all the fops will be replayed on "slave mount" point.

Problem:

With a DHT volume, in changelog "internal fops" are NOT recorded.
For Rename case, Rename is recorded in "hashed" brick changelog.
(DHT's internal fops like creating linkto file, unlink is NOT recorded).
This lead us to inconsistent rename operations.

For example,
Distribute volume created with Two bricks B1, B2.

//Consider master volume mounted @ /mnt/master and following operations
executed:
cd /mnt/master
touch f1 // f1 falls on B1 Hash
mv f1 f2 // f2 falls on B2 Hash

// Here, Changelogs are recorded as below:
@B1
CREATE f1

@B2
RENAME f1 f2

Here, race exists between Brick B1 and B2, say B2 will get executed first.
Source file f1 itself is "NOT PRESENT", so it will go ahead and create
f2 (Current implementation).

We have this problem When rename falls in another brick and
file is unlinked in Master.

Similar kind of issue exists in following case too(multiple rename):
CREATE f1
RENAME f1 f2
RENAME f2 f1

Solution:

Instead of carrying out "changelogging" at "HASHED volume",
carry out  at the "CACHED volume".
This way we have rename operations carried out where actual files are present.

So,Changelog recorded as :
@B1
CREATE f1
RENAME f1 f2

Note:
This patch is dependent on dht changes from this patch.
http://review.gluster.org/10410/
changelog related changes are separated out for review.

In changelog, xdata passed from DHT is considered as:

1. In case of unlink (internal operation as part of rename), xdata value
is set , it is considered as RENAME and recorded accordingly.
2. In case of rename (Hash and Cache different), xdata value is NOT
set, recording rename operation is SKIPPED.

BUG: 1219412
Change-Id: I7691166c84991482b2cfe073df64e2317c935b13
Reviewed-On: http://review.gluster.org/#/c/10220/
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/10633
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog: Fix buffer overflow in snprintf

2015-05-09T03:54:38+00:00

BUG: 1219823
Change-Id: I3a1c7b7742671847ed3fec13e06d861c3d09f7a9
Reviewed-On: http://review.gluster.org/#/c/10687/
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10688
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK

features/changelog: Fixing compilation issue

2015-05-08T15:37:46+00:00

This issue introduced due to manual rebase.

Change-Id: I0589f4a0a1270190340f419b8022d6483bcf853d
Signed-off-by: Aravinda VK 
BUG: 1219479
Reviewed-on: http://review.gluster.org/10685
Reviewed-by: Niels de Vos 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10686