glusterfs.git/xlators/features/bit-rot/src/stub/bit-rot-stub.h, branch v3.7.2

features/bitrot: serialize versioning

2015-05-31T12:16:10+00:00

    Backport of http://review.gluster.org/10832

Current signing interface (fsetxattr()) had couple of issues:

One, a signing request (by bitrot daemon) is denied if the version
against which an object is to be signed is unequal to the current
version of the object (cases where another subsequent modification
increments the version). Such request(s) are rejected with EINVAL
sent back to the signer resulting in a bunch of errors (in logs)
reported by bitrot daemon. Although, the object would be eventaully
signed with the version matching the current version, the "lagging"
request should be correctly handled.

Two, more than one signing request could race against each other
with the object getting signed with a version depending on which
request ended up last in the race. Although harmless to some extent,
such a case could end up marking the object's signature as stale
for infinity (if the object is *never* touched) thereby resulting
in scrubber skipping the object during verification.

This patch fixes these issues by ordering signing request(s) and
fixing version comparison checks at the time of signing.

Change-Id: I9fa83dfa3be664ba4db61d7f2edc408f4bde77dd
BUG: 1224650
Signed-off-by: Venky Shankar 
Tested-by: Gluster Build System 
Reviewed-on: http://review.gluster.org/10900
Tested-by: NetBSD Build System

features/bitrot: stub improvements and fixes

2015-05-31T04:12:48+00:00

This patch refactors the signing trigger mechanism used by bitrot
daemon as a "catch up" meachanism to sign files which _missed_
signing on the last run either due to bitrot being disabled and
enabled again or if bitrot is enabled for a volume with existing
data.

Existing implementation relies on overloading writev() to trigger
signing which just by the looks sounded dangerous and I hated it
to the core. This change moves all that business to the setxattr
interface thereby keeping the writev path strictly for client
IO.

Why not use IPC fop to trigger signing?
There's a need to access the object's inode to perform various
maintainance operations. inode is not _directly_ accessible in
the IPC fop (although, it can be found via inode_grep() for the
object's GFID - the inode just needs to be pinned in memory,
which is the case if there's an active fd on the inode). This
patch relies on good old technique of overloading fsetxattr()
to do the job instead of using IPC fop.

There are some pretty nice cleanups along the lines of memory
deallocations, unncessary allocations and redundant ref()ing
of structures (such as fd's) provided by this patch. All in
all - much improved code navigation.

Change-Id: Id93fe90b1618802d1a95a5072517dac342b96cb8
BUG: 1225709
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10953
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

features/bit-rot-stub: versioning of objects in write/truncate fop instead of open

2015-05-10T15:14:33+00:00

* This patch brings in the changes where object versioning is done in write and
  truncate fops instead of tracking them in open and create fops. This model
  works for both regular and anonymous fds. It also removes the race associated
  with open calls, create and lookups.

  This patch follows the below method for object versioning and notifications:

  Before sending writev on the fd, increase the ongoing
  version first. This makes anonymous fd write similar to the regular
  fd write by having the ongoing version increased before doing the
  write.

  Do following steps to do versioning:
  1) For anonymous fds set the fd context (so that release is invoked) and add
     the fd context to the list maintained in the inode context.
     For regular fds the above think would have been done in open itself.
  2) Increase the on-disk ongoing version
  3) Increase the in memory ongoing version and mark inode as non-dirty
  3) Once versioning is successfully done send write operation. If
     versioning fails, then fail the write fop.
  5) In writev_cbk mark inode as modified.

> Change-Id: I7104391bbe076d8fc49b68745d2ec29a6e92476c
> BUG: 1207979
> Signed-off-by: Raghavendra Bhat 
> Reviewed-on: http://review.gluster.org/10233
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I4bb86989b5fab02b9ed2950798b1a80e566f1024
BUG: 1220041
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/10722
Reviewed-by: Gaurav Kumar Garg 
Tested-by: NetBSD Build System
Tested-by: Gluster Build System

features/bitrot: Follow xattr naming conventions

2015-05-10T12:28:43+00:00

Instead of "trusted.glusterfs.bit-rot.*" use "trusted.bit-rot.*"

NOTE:
With this patch, data on existing volumes would be resigned
(which should be OK as of now since we do not expect many
users as of now :-))

> Change-Id: I926c7bca266a9c8f2cb35d57c4d0359aa5cecfa0
> BUG: 1170075
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10181
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I3c18d7dc2db4beaca6e8d8d231b4171a7b18795f
Signed-off-by: Venky Shankar 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10718
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg

bitrot/scrub: Scrubber fixes

2015-04-08T07:01:32+00:00

This patch fixes a handful of problem with scrubber which
are detailed below.

Scrubber used to skip objects for verification due to missing
fd iterface to fetch versioning extended attributes. Similar
to the inode interface, an fd based interface in POSIX is now
introduced.

Moreover, this patch also fixes potential false reporting by
scrubber due to:

  An object gets dirtied and signed when scrubber is busy
  calculatingobject checksum. This is fixed by caching the
  signed version when an object is first inspected for
  stalenes, i.e., during pre-compute stage. This version is
  used to verify checksum in the post-compute stage when the
  signatures are compared for possible corruption.

  Side effect of _not_ sending signature length during signing
  resulted in "truncated" signature to be set for an object.
  Now, at the time of signing, the signature length is sent
  and is used in place of invoking strlen() to get signature
  length (which could have possible 00s). The signature length
  itself is not persisted in the signature xattr, but is
  calculated on-the-fly by substracting the xattr length by
  the "structure" header size.

Some of the log entries are made more meaningful (as and aid
for debugging).

Change-Id: I938bee5aea6688d5d99eb2640053613af86d6269
BUG: 1207624
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10118
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Bitrot Stub

2015-03-24T13:37:19+00:00

Bitrot stub implements object versioning required for identifying
signature freshness. More details about versioning is explained
as a part of the "bitrot feature documentation" patch.

Change-Id: I2ad70d9eb109ba4a12148ab8d81336afda529ad9
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9709
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur