glusterfs.git/xlators/features/bit-rot/src, branch v3.11.2

feature/bitrot: Fix ondemand scrub

2017-06-19T15:50:33+00:00

The flag which keeps tracks of whether the scrub
frequency is changed from previous value should
not be considered for on-demand scrubbing. It
should be considered only for 'scrub-frequency'
where it should not be re-scheduled if it is
set to same value again. But in case ondemand
scrub, it should start the scrub immediately
no matter what the scrub-frequency.

Reproducer:
1. Enable bitrot
2. Set scrub-throttle
3. Set ondemand scrub
Make sure glusterd is not restarted while doing
below steps

> Change-Id: Ice5feaece7fff1579fb009d1a59d2b8292e23e0b
> BUG: 1461845
> Signed-off-by: Kotresh HR 
> Reviewed-on: https://review.gluster.org/17552
> Smoke: Gluster Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
> NetBSD-regression: NetBSD Build System 
(cherry picked from commit f0fb166078d59cab2a33583591b6448326247c40)


Change-Id: Ice5feaece7fff1579fb009d1a59d2b8292e23e0b
BUG: 1462127
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17555
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

features/bitrot: Fix glusterfsd crash

2017-05-29T13:57:11+00:00

With object versioning being optional, it can
so happen the bitrot stub context is not always
set. When it's not found, it's initialized. But
was not being assigned to use in the local
function. This was leading for brick crash.
Fixed the same.

> Change-Id: I0dab6435cdfe16a8c7f6a31ffec1a370822597a8
> BUG: 1454317
> Signed-off-by: Kotresh HR 
> Reviewed-on: https://review.gluster.org/17357
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Raghavendra Bhat 
(cherry picked from commit 6908e962f6293d38f0ee65c088247a66f2832e4a)

Change-Id: I0dab6435cdfe16a8c7f6a31ffec1a370822597a8
BUG: 1456331
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17406
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

core: make the per glusterfs_ctx_t timer-wheel refcounted

2017-05-12T13:32:32+00:00

xlators can use a 'global' timer-wheel for scheduling events. This
timer-wheel is managed per glusterfs_ctx_t, but does not need to be
allocated for every graph. When an xlator wants to use the timer-wheel,
it will be instanciated on demand, and provided to xlators that request
it later on.

By adding a reference counter to the glusterfs_ctx_t for the
timer-wheel, the threads and structures can be cleaned up when the last
xlator does not have a need for it anymore. In general, the xlators
request the timer-wheel in init(), and they should return it in fini().

Because the timer-wheel is managed per glusterfs_ctx_t, the functions
can be added to ctx.c and do not need to live in their very minimal
tw.[ch] files.


>Reported-by: Poornima G 
>Signed-off-by: Niels de Vos 
>Reviewed-on: https://review.gluster.org/17068
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Smoke: Gluster Build System 
>Reviewed-by: Amar Tumballi 
>Reviewed-by: Zhou Zhengping 
>Reviewed-by: Kaleb KEITHLEY 
>(cherry picked from commit 73fcf3a874b2049da31d01b8363d1ac85c9488c2)

Change-Id: I19d225b39aaa272d9005ba7adc3104c3764f1572
BUG: 1450267
Reviewed-on: https://review.gluster.org/17262
Tested-by: Poornima G 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

libglusterfs/stack.h: reduce duplication of code

2017-04-27T14:54:40+00:00

* Use STACK_UNWIND_STRICT everywhere.
* Provide STACK_WIND_COMMON as both STACK_WIND_COOKIE
  and STACK_WIND differ by just 1 line and 1 option.

Updates gluster/glusterfs#137

Change-Id: Ifbb6b9c4702b02f4a02834824f509fd10c78f0ce
Signed-off-by: Amar Tumballi 
Reviewed-on: https://review.gluster.org/16915
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

features/bit-rot-stub: bring in optional versioning

2017-04-19T03:29:46+00:00

* As of now bit-rot-stub does versioning always. This leads
  lots of getxattr calls being made in lookups. So make
  object versioning optional.

Change-Id: I83713e45ae59fb28004bb3cfa008f2d69edebbfa
BUG: 1359599
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/14442
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur

features/bit-rot-stub: use the correct spelling of quarantine for bad objects

2017-01-30T20:17:51+00:00

                       container

The directory for containing the list of bad objects was named "quanrantine"
instead of "quarantine"

Change-Id: I8c20381ac637201d9d1a224f5223e8dfbed53f1e
BUG: 1401571
Signed-off-by: Raghavendra Bhat 
Reviewed-on: https://review.gluster.org/16027
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kotresh HR

trivial: correct some spelling mistakes in comments and logs

2016-10-18T10:27:12+00:00

BUG: 1385593
Change-Id: Icfae9e557a284182c6c22e9606fdd641528906f0
Reported-by: Patrick Matthäi 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/15656
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Kotresh HR 
Smoke: Gluster Build System

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

feature/bitrot: Fix recovery of corrupted hardlink

2016-09-08T17:09:33+00:00

Problem:
When a file with hardlink is corrupted in ec volume,
the recovery steps mentioned was not working.
Only name and metadata was healing but not the data.

Cause:
The bad file marker in the inode context is not removed.
Hence when self heal tries to open the file for data
healing, it fails with EIO.

Background:
The bitrot deletes inode context during forget.

Briefly, the recovery steps involves following steps.
   1. Delete the entry marked with bad file xattr
      from backend. Delete all the hardlinks including
      .glusters hardlink as well.
   2. Access the each hardlink of the file including
      original from the mount.

The step 2 will send lookup to the brick where the files
are deleted from backend and returns with ENOENT. On
ENOENT, server xlator forgets the inode if there are
no dentries associated with it. But in case hardlinks,
the forget won't be called as dentries (other hardlink
files) are associated with the inode. Hence bitrot stube
won't delete it's context failing the data self heal.

Fix:
Bitrot-stub should delete the inode context on getting
ENOENT during lookup.

Change-Id: Ice6adc18625799e7afd842ab33b3517c2be264c1
BUG: 1373520
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15408
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat

bit-rot: fix unused variable warnings/errors

2016-08-29T16:21:41+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I566fcbcae5aec575bfca40975b941c53546d4d97
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15245
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Kotresh HR