glusterfs.git/xlators/features/bit-rot, branch v3.7.9

features/bit-rot-stub: get frame->local before unwinding

2016-03-09T19:29:26+00:00

In bit-rot-stub, if unlink fails, then it was unwinding
directly. Then it was trying to cleanup local. But local
would be NULL, since it was unwinding directly without getting
the value of frame->local. The NULL cleanup of local was
causing the brick process to crash.

Change-Id: I8544ba73b2e8dc0c50b1a53ff8027d85588d087b
BUG: 1315552
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/13630
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

features/bitrot: do not remove the quarantine handle in forget

2016-03-07T13:04:18+00:00

If an object is marked as bad, then an entry is corresponding to the
bad object is created in the .glusterfs/quarantine directory to help
scrub status. The entry name is the gfid of the corrupted object.
The quarantine handle is removed in below 2 cases.

1) When protocol/server revceives the -ve lookup on an entry whose inode
   is there in the inode table (it can happen when the corrupted object
   is deleted directly from the backend for recovery purpose) it sends a
   forget on the inode and bit-rot-stub removes the quarantine handle in
   upon getting the forget.
   refer to the below commit
   f853ed9c61bf65cb39f859470a8ffe8973818868:
   http://review.gluster.org/12743)

2) When bit-rot-stub itself realizes that lookup on a corrupted object
   has failed with ENOENT.

But with step1, there is a problem when the bit-rot-stub receives forget
due to lru limit exceeding in the inode table. In such cases, though the
corrupted object is not deleted (either from the mount point or from the
backend), the handle in the quarantine directory is removed and that object
is not shown in the bad objects list in the scrub status command.

So it is better to follow only 2nd step (i.e. bit-rot-stub removing the handle
from the quarantine directory in -ve lookups). Also the handle has to be removed
when a corrupted object is unlinked from the mount point itself.

Change-Id: Ibc3bbaf4bc8a5f8986085e87b729ab912cbf8cf9
BUG: 1313131
Original author: Raghavendra Bhat 
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13472
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Venky Shankar 
(cherry picked from commit 2102010edab355ac9882eea41a46edaca8b9d02c)
Reviewed-on: http://review.gluster.org/13552
Tested-by: Venky Shankar

features / bitrot: Prevent spurious pthread_cond_wait() wakeup

2016-01-28T13:29:08+00:00

        Backport of http://review.gluster.org/13302

pthread_cond_wait() is prone to spurious wakeups and it's utmost
necessarry to check a boolean predicate for thread continuation.

See man(3) pthread_cond_wait() for details.

The following is done in bitrot scrubber:

    if (list_empty (&fsscrub->scrublist))
       pthread_cond_wait (&fsscrub->cond, &fsscrub->mutex);

followed by:

    list_first_entry (&fsscrub->scrublist, ...)

A spurious wakeup from pthread_cond_wait() with the absence of
list_empty() check causes list_first_entry() to return garbage.

BUG: 1302199
Change-Id: I60151eabb8af257a35acd8e7c117876388166a0e
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/13307
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

features/bitrot: Fail node-uuid getxattr if file is marked bad

2016-01-27T11:46:01+00:00

If xattr is node-uuid and the inode is marked bad, fail getxattr
and fgetxattr with EIO. Returning EIO would result in AFR to
choose correct node-uuid coresponding to the subvolume where
the good copy of the file resides.

BUG: 1296795
Change-Id: I3f8dc807794f9a82867807e7c4c73ded6c64fd8a
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13116
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/13194
Tested-by: Venky Shankar 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

features/bitrot: add check for corrupted object in f{stat}

2016-01-27T02:44:58+00:00

         Backport of http://review.gluster.org/13120

Check for corrupted objects is done bt bitrot stub component
for data operations and such fops are denied processing by
returning EIO. These checks were not done for operations such
as get/set extended attribute, stat and the likes - IOW, stub
only blocked pure data operations.

However, its necessary to have these checks for certain other
fops, most importantly stat (and fstat). This is due to the
fact that clients could possibly get stale stat information
(such as size, {a,c,m}time) resulting in incorrect operation
of the application that rely on these fields. Note that, the
data that replication would take care of fetching good (and
correct) data, but the staleness of stat information could
lead to data inconsistencies (e.g., rebalance, tier).

Change-Id: I5a22780373b182a13f8d2c4ca6b7d9aa0ffbfca3
BUG: 1297213
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/13276
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

afr: handle bad objects during lookup/inode_refresh

2016-01-22T05:28:34+00:00

Backport of  http://review.gluster.org/12955,
http://review.gluster.org/#/c/13077/ and http://review.gluster.org/#/c/13185/

If an object (file) is marked bad by bitrot, do not consider the brick
on which the object is present  as a potential read subvolume for AFR
irrespective of the pending xattr values.

Also do not consider the brick containing the bad object while
performing afr_accuse_smallfiles(). Otherwise if the bad object's size
is bigger,we may end up considering that as the source.

Change-Id: I4abc68e51e5c43c5adfa56e1c00b46db22c88cf7
BUG: 1293300
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/13041
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
NetBSD-regression: NetBSD Build System 
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System

all: reduce "inline" usage

2016-01-18T09:02:34+00:00

There are three kinds of inline functions: plain inline, extern inline,
and static inline.  All three have been removed from .c files, except
those in "contrib" which aren't our problem.  Inlines in .h files, which
are overwhelmingly "static inline" already, have generally been left
alone.  Over time we should be able to "lower" these into .c files, but
that has to be done in a case-by-case fashion requiring more manual
effort.  This part was easy to do automatically without (as far as I can
tell) any ill effect.

In the process, several pieces of dead code were flagged by the
compiler, and were removed.

backport of Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155,
http://review.gluster.org/11769, BUG: 1245331

Change-Id: Iba1efb0bc578ea4a5e9bf76b7bd93dc1be9eba44
BUG: 1283302
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/12646
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

features/bit-rot-stub: delete the link for bad object in quarantine directory

2016-01-09T09:57:35+00:00

When the bad object is deleted (as of now manually from the backend itself),
along with its gfid handle, the entry for the bad object in the quarantne
directory is left as it is (it also can be removed manually though). But the
next lookup of the object upon not finding it in the backend, sends forget on
the in-memory inode. If the stale link for the gfid still exists in the
quarantine directory, bir-rot-stub will unlink the entry in its forget or in
the next failed lookup on that object with errno being ENOENT.

Change-Id: If84292d3e44707dfa11fa29023b3d9f691b8f0f3
BUG: 1293584
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/12743
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
(cherry picked from commit f853ed9c61bf65cb39f859470a8ffe8973818868)
Reviewed-on: http://review.gluster.org/13032

bitrot: getting correct value of scrub stat's

2015-12-18T03:29:00+00:00

This patch is backport of: http://review.gluster.org/#/c/12776/

When user execute bitrot scrub status command then gluster
is not giving correct value of Number of Scrubbed files,
Number of Unsigned files, Last completed scrub time,
Duration of last scrub.

With this patch scrub status will give correct value for
all the above fields.

  >> Change-Id: Ic966f76d22db5b0c889e6386a1c2219afbda1f49
  >> BUG: 1285989
  >> Signed-off-by: Gaurav Kumar Garg 
  >> Signed-off-by: Kotresh HR 
  >> Reviewed-on: http://review.gluster.org/12776
  >> Tested-by: NetBSD Build System 
  >> Tested-by: Gluster Build System 
  >> Reviewed-by: Venky Shankar 

Change-Id: Ic966f76d22db5b0c889e6386a1c2219afbda1f49
BUG: 1291546
Signed-off-by: Gaurav Kumar Garg 


(cherry picked from commit 22827d51c232c44a8f5ac003529d907d93baf7b0)

Change-Id: Icef24cce35c8d54ffdfa5282491338318e78780b
Reviewed-on: http://review.gluster.org/12966
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Venky Shankar

features/bit-rot: Fix NULL dereference

2015-11-27T04:04:47+00:00

        Backport of http://review.gluster.org/12754

Problem:
By the time br_stub_worker is accessing this->private in it's
thread, 'init' may not have set 'this->private = priv'. This
leads to NULL dereference leading to brick crash.

Fix:
Set this->private before launching these threads.

BUG: 1285758
Change-Id: I8a9234c4f96b0e5ea78f5b336369ec41f5a120ef
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12764
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar