glusterfs.git/xlators/features/bit-rot/src/bitd/bit-rot-scrub.c, branch v3.12.9

libglusterfs: Name threads on creation

2017-07-19T14:16:19+00:00

Set names to threads on creation for easier
debugging.

Output of top -H -p 
Before:
19773 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19774 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19775 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19776 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19777 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19778 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19779 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19780 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19781 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19782 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19783 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19784 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19785 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.01 glusterfsd
19786 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.01 glusterfsd
19787 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.01 glusterfsd
19789 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19790 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
25178 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
 5398 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
 7881 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd

After:
19773 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19774 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glustertimer
19775 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterfsd
19776 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glustermemsweep
19777 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glustersproc0
19778 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glustersproc1
19779 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterepoll0
19780 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusteridxwrker
19781 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusteriotwr0
19782 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterbrssign
19783 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterbrswrker
19784 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterclogecon
19785 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.01 glusterclogd0
19786 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.01 glusterclogd1
19787 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.01 glusterclogd2
19789 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterposixjan
19790 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterposixfsy
25178 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterepoll1
 5398 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterepoll2
 7881 root      20   0 1301.3m  12.6m   8.4m S  0.0  0.1   0:00.00 glusterposixhc

Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703
BUG: 1254002
Updates: #271
Signed-off-by: Raghavendra Talur 
Reviewed-on: https://review.gluster.org/11926
Reviewed-by: Prashanth Pai 
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System

feature/bitrot: Fix ondemand scrub

2017-06-16T06:01:49+00:00

The flag which keeps tracks of whether the scrub
frequency is changed from previous value should
not be considered for on-demand scrubbing. It
should be considered only for 'scrub-frequency'
where it should not be re-scheduled if it is
set to same value again. But in case ondemand
scrub, it should start the scrub immediately
no matter what the scrub-frequency.

Reproducer:
1. Enable bitrot
2. Set scrub-throttle
3. Set ondemand scrub
Make sure glusterd is not restarted while doing
below steps

Change-Id: Ice5feaece7fff1579fb009d1a59d2b8292e23e0b
BUG: 1461845
Signed-off-by: Kotresh HR 
Reviewed-on: https://review.gluster.org/17552
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
NetBSD-regression: NetBSD Build System

feature/bitrot: Ondemand scrub option for bitrot

2016-08-25T21:39:38+00:00

The bitrot scrubber takes 'hourly/daily/biweekly/monthly'
as the values for 'scrub-frequency'. There is no way
to schedule the scrubbing when the admin wants it.

Ondemand scrubbing brings in the new option 'ondemand'
with which the admin can start scrubbing ondemand.
It starts the scrubbing immediately.

Ondemand scrubbing is successful only if the scrubber
is in 'Active (Idle)' (waiting for it's next frequency
cycle to start scrubbing). It is not entertained when
the scrubber is in 'Paused' or already running.

Here is the command line syntax.

gluster volume bitrot  scrub ondemand

Change-Id: I84c28904367eed827a7dae8d6a535c14b28e9f4d
BUG: 1366195
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15111
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Venky Shankar

eventsapi: Fix disable-events issue

2016-08-24T08:23:06+00:00

Events related sources are not loaded in libglusterfs when
configure is run with --disable-events option. Due to this
every call of gf_event should be guarded with USE_EVENTS macro.

To prevent this, USE_EVENTS macro was included in events.c
itself(Patch #15054)

Instead of disabling building entire directory "events", selectively
disabled the code. So that constants and empty function gf_event is
exposed. Code will not fail even if gf_event is called when events is
disabled.

BUG: 1368042
Change-Id: Ia6abfe9c1e46a7640c4d8ff5ccf0e9c30c87f928
Signed-off-by: Aravinda VK 
Reviewed-on: http://review.gluster.org/15198
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System

eventsapi: Bitrot events

2016-08-19T11:14:43+00:00

Following Bitrot Events are added

BITROT_ENABLE/BITROT_DISABLE
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": EVENT_TYPE,
    "message": {
       "name": VOLUME_NAME,
    }
}

BITROT_SCRUB_THROTTLE/BITROT_SCRUB_FREQ/BITROT_SCRUB
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": EVENT_TYPE,
    "message": {
       "name": VOLUME_NAME,
       "value": OPTION_VALUE
    }
}

EVENT_BITROT_BAD_FILE
{
    "nodeid": NODEID,
    "ts": TIMESTAMP,
    "event": EVENT_TYPE,
    "message": {
       "gfid": GFID_OF_FILE,
       "brick": BRICK_ROOT,
       "path":  FILE_PATH_FROM_BRICK_ROOT (if available)
    }
}

Change-Id: I8c37b0e9db9f4f0f3d05d8f78b5521c7db0e2237
BUG: 1367815
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/15190
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
Reviewed-by: Aravinda VK

feature/bitrot: Ignore files with sticky bit set

2016-07-21T13:59:49+00:00

Scrubber scrubs entries in backend. It is scrubbing files with sticky
bit as well. This might include linkfiles which should be skipped.  This
patch adds the check to ignore linkfiles during scrub.

Change-Id: Ic21367b37770d391326c55c659491a1e5a82335b
BUG: 1355706
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14903
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
CentOS-regression: NetBSD Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

feature/bitrot: Fix scrub status with sharded volume

2016-07-19T14:41:09+00:00

Bitrot scrubs each shard entries separately. Scrub
statistics was counting each shard entry which is
incorrect. This patch skips the statistics count
for sharded entries.

Change-Id: I184c315a4bc7f2cccabc506eef083ee926ec26d3
BUG: 1356851
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14927
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Vijay Bellur

features/bitrot: Move throttling code to libglusterfs

2016-07-18T12:03:17+00:00

Since throttling is a separate feature by itself,
move throttling code to libglusterfs.

Change-Id: If9b99885ceb46e5b1865a4af18b2a2caecf59972
BUG: 1352019
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14846
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Jeff Darcy

feature/bitrot: Show whether scrub is in progress/idle

2016-07-11T13:32:04+00:00

Bitrot scrub status shows whether the scrub is paused
or active. It doesn't show whether the scrubber is
actually scrubbing or waiting in the timer wheel
for the next schedule. This patch shows this status
with "In Progress" and "Idle" respectively.

Change-Id: I995d8553d1ff166503ae1e7b46282fc3ba961f0b
BUG: 1352871
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14864
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

features/bitrot: Option to set scrub interval to a minute

2016-07-05T11:46:14+00:00

Bitrot scrub-frequency supports "hourly|daily|weekly|biweekly|monthly".
But it is painful for testing as minimum  scrub-interval is an hour
Hence introducing a scrub interval of minute to ease testing.
It is intentionally not exposed in bitrot command help as it is
only for testing.

e.g.,
gluster vol bitrot  scrub-frequency minute

Change-Id: I155a65298d3fad5ae9e529d9c7d4b0d25fa297c0
BUG: 1351537
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14836
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy