glusterfs.git/xlators/features/bit-rot/src, branch v3.8dev

features/bit-rot: Mark versioning fsetxattr as internal fop

2015-04-11T05:07:19+00:00

Changelog xlator was capturing bitrot-stub's fsetxattr sent
for versioning. Since it was using the same frame as of the
create fop, there was inconsistency in fop number and gfid
of capturing metadata. So fix is to mark fsetxattr used for
versioning as internal and add internal fop filter in
changelog_fsetxattr.

Change-Id: I51ff468995139838b22bf293a59a0713a92ee7a5
BUG: 1170075
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/10148
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/bitrot-stub: Packed format for version xattr

2015-04-09T08:54:37+00:00

Using __attribute__ ((__packed__)) for object signature xattr
saves some bytes (7 bytes to be particular) occupied by the
extended attribute on-disk as compared to the unpacked format.

Change-Id: I91a6a0a54aa60e6fd8c357d72f7601b6ed213f2d
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10161
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

libglusterfs/syncop: Add xdata to all syncop calls

2015-04-08T15:14:59+00:00

This patch adds support for xdata in both the
request and response path of syncops.

Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.

xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.

There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.

xdata support at gfapi level would be introduced
in subsequent patches.

Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/bitrot-stub: header file update in noinst_HEADERS

2015-04-08T10:50:17+00:00

Missing "bit-rot-object-version.h" causing devrpm failures.

Change-Id: I5af326c5871cc468a10dece4772b29eda06c4fa9
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10160
Reviewed-by: Raghavendra Bhat 
Tested-by: Vijay Bellur

bitrot/scrub: Scrubber fixes

2015-04-08T07:01:32+00:00

This patch fixes a handful of problem with scrubber which
are detailed below.

Scrubber used to skip objects for verification due to missing
fd iterface to fetch versioning extended attributes. Similar
to the inode interface, an fd based interface in POSIX is now
introduced.

Moreover, this patch also fixes potential false reporting by
scrubber due to:

  An object gets dirtied and signed when scrubber is busy
  calculatingobject checksum. This is fixed by caching the
  signed version when an object is first inspected for
  stalenes, i.e., during pre-compute stage. This version is
  used to verify checksum in the post-compute stage when the
  signatures are compared for possible corruption.

  Side effect of _not_ sending signature length during signing
  resulted in "truncated" signature to be set for an object.
  Now, at the time of signing, the signature length is sent
  and is used in place of invoking strlen() to get signature
  length (which could have possible 00s). The signature length
  itself is not persisted in the signature xattr, but is
  calculated on-the-fly by substracting the xattr length by
  the "structure" header size.

Some of the log entries are made more meaningful (as and aid
for debugging).

Change-Id: I938bee5aea6688d5d99eb2640053613af86d6269
BUG: 1207624
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10118
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

tests/bitrot-stub: Object versioning test(s)

2015-04-08T06:47:51+00:00

This patch introduces basic object versioning test(s) which
is required for bitrot detection to work correctly. Basic
test(s) such as opening a file in read-only mode, single
open, multiple open()s are covered on FUSE mount _only_ as
stub does not support anonymous fds yet. For this reason,
the test case disables open-behind.

Actual verification is implemented as a C source which
makes use of the same on-disk data structures as used by
the stub code. The data structures are moved to separate
header file which is included by the test script. Such
modularization helps in future enhancements to keep the
version "data type" opaque and provide handful of APIs
version checking (equal/greater/etc..).

[
  This is just a start and should grow over time as stub
  is enhanced and codebase matures.
]

Change-Id: Ibee20e65a15b56bbdd59fd2703f9305b115aec7a
BUG: 1201724
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10140
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/bitrot-stub: Enhancement to versioning protocol

2015-04-08T06:43:09+00:00

.. and potential bug fixes / memleak.

While assigning initial version to an object, both extended attributes
(namely, ongoing version and the default signing version) were persisted.
This is optimized to just persist the ongoing version along with safe
handling of xattr request(s) in it's absence. This is better than the
earlier approach as the two xattr sets were not atomic anyway (allowing
a request to sneak in between between two set operations). This also
allows to perform sanity checks on objects during lookup()/getxattr():
objects with missing ongoing version but presence of signature are
possible candidates of tampering (and catching implementation bugs).

There were couple of instances in the code where versioning xattrs
were incorrectly removed before in-memory versions were initialized,
which have been fixed with this patch. A memory leak in the IPC code
path is also fixed.

Change-Id: I01c690ccfe7156a883582275f40f79a7c10c0900
BUG: 1207054
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10117
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

features/bit-rot: fix CID 1124725 - use after free

2015-04-02T12:27:46+00:00

Coverity fixes:
CID 1124725
CID 1291742

The problem is that gf_tw_cleanup_timers() frees the handed
in priv->timer_wheel but it can not set the pointer to NULL,
so subsequent checks for priv->timer_wheel show it as not NULL
and allow for access after free.

The proper change might be to change gf_tw_cleanup_timers() to
take a reference to the pointer and set it to NULL after free,
but since it is under contrib/, I did not want to change that
function. Instead this patch uses the function's return code
which was not used previously. (Maybe this should even be done
in a wrapper macro or function?)

Change-Id: I31d80d3df2e4dc7503d62c7819429e1a388fdfdd
BUG: 789278
Signed-off-by: Michael Adam 
Reviewed-on: http://review.gluster.org/10056
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/bit-rot: fix assignment instead of check (Coverity fix)

2015-04-01T11:22:49+00:00

Fixes Coverity CIDs 1291728, 1291723, 1291732.

Change-Id: I62f3d540cac0f555fe2839b8418e59691c3ff4fd
BUG: 789278
Signed-off-by: Michael Adam 
Reviewed-on: http://review.gluster.org/10055
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar