Since this xlator measures time intervals in seconds, gettimeofday() may
be replaced with simpler gf_time(). Adjust comments and style as well.

Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Change-Id: I2e5900ccb5da2478656d80e5f570487e3ba70966
Updates: #1002

Logs and other output carrying timestamps
will have now timezone offsets indicated, eg.:

[2020-03-12 07:01:05.584482 +0000] I [MSGID: 106143] [glusterd-pmap.c:388:pmap_registry_remove] 0-pmap: removing brick (null) on port 49153

To this end,

- gf_time_fmt() now inserts timezone offset via %z strftime(3) template.
- A new utility function has been added, gf_time_fmt_tv(), that
  takes a struct timeval pointer (*tv) instead of a time_t value to
  specify the time. If tv->tv_usec is negative,

  gf_time_fmt_tv(... tv ...)

  is equivalent to

  gf_time_fmt(... tv->tv_sec ...)

  Otherwise it also inserts tv->tv_usec to the formatted string.
- Building timestamps of usec precision has been converted to
  gf_time_fmt_tv, which is necessary because the method of appending
  a period and the usec value to the end of the timestamp does not work
  if the timestamp has zone offset, but it's also beneficial in terms of
  eliminating repetition.
- The buffer passed to gf_time_fmt/gf_time_fmt_tv has been unified to
  be of GF_TIMESTR_SIZE size (256). We need slightly larger buffer space
  to accommodate the zone offset and it's preferable to use a buffer
  which is undisputedly large enough.

This change does *not* do the following:

- Retaining a method of timestamp creation without timezone offset.
  As to my understanding we don't need such backward compatibility
  as the code just emits timestamps to logs and other diagnostic
  texts, and doesn't do any later processing on them that would rely
  on their format. An exception to this, ie. a case where timestamp
  is built for internal use, is graph.c:fill_uuid(). As far as I can
  see, what matters in that case is the uniqueness of the produced
  string, not the format.
- Implementing a single-token (space free) timestamp format.
  While some timestamp formats used to be single-token, now all of
  them will include a space preceding the offset indicator. Again,
  I did not see a use case where this could be significant in terms
  of representation.
- Moving the codebase to a single unified timestamp format and
  dropping the fmt argument of gf_time_fmt/gf_time_fmt_tv.
  While the gf_timefmt_FT format is almost ubiquitous, there are
  a few cases where different formats are used. I'm not convinced
  there is any reason to not use gf_timefmt_FT in those cases too,
  but I did not want to make a decision in this regard.

Change-Id: I0af73ab5d490cca7ed8d07a2ce7ac22a6df2920a
Updates: #837
Signed-off-by: Csaba Henk <csaba@redhat.com>

Found with clang-10 -Wfortify-source:

bit-rot-scrub.c:1802:15: warning: 'snprintf' size argument is too large;
destination buffer has size 32, but size argument is 4096 [-Wfortify-source]
        len = snprintf(key, PATH_MAX, "quarantine-%d", j);
              ^
bit-rot-scrub.c:1813:9: warning: 'snprintf' size argument is too large;
destination buffer has size 32, but size argument is 4096 [-Wfortify-source]
        snprintf(main_key, PATH_MAX, "quarantine-%d", tmp_count);

Change-Id: I9b9c09ef2223ed181d81215154345de976b82f13
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Fixes: #1221

The number of signing process threads (glfs_brpobj)
is set to 4 by default. The recommendation is to set
it to number of cores available. This patch makes it
configurable as follows

gluster vol bitrot <volname> signer-threads <count>

fixes: bz#1797869
Change-Id: Ia883b3e5e34e0bc8d095243508d320c9c9c58adc
Signed-off-by: Kotresh HR <khiremat@redhat.com>

convert all gf_msg() to gf_smsg()

Change-Id: Ifa45b5089f83ddfcf69132bb8d9c0dc6d012464b
Updates: #657
Signed-off-by: yatipadia <ypadia@redhat.com>

In many cases, we were freely allocating long keys with no need.
Smaller char arrays are just fine almost anywhere, so just went ahead
and looked where they we can use smaller ones.

In some cases, annotated the functions as static and the prefixes
passed as const as it was easier to read and understand.

Where relevant, converted the dict functions to use known key length.

Change-Id: I882ab33ea20d90b63278336cd1370c09ffdab7f2
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

There are many include statements that are not needed.
A previous more ambitious attempt failed because of *BSD plafrom
(see https://review.gluster.org/#/c/glusterfs/+/21929/ )

Now trying a more conservative reduction.
It does not solve all circular deps that we have, but it
does reduce some of them. There is just too much to handle
reasonably (dht-common.h includes dht-lock.h which includes
dht-common.h ...), but it does reduce the overall number of lines
of include we need to look at in the future to understand and fix
the mess later one.

Change-Id: I550cd001bdefb8be0fe67632f783c0ef6bee3f9f
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Currently bit-rot feature has an issue with disabling and reenabling it
on the same volume. Consider enabling bit-rot detection which goes on to
crawl and sign all the files present in the volume. Then some files are
modified and the bit-rot daemon goes on to sign the modified files with
the correct signature. Now, disable bit-rot feature. While, signing and
scrubbing are not happening, previous checksums of the files continue to
exist as extended attributes. Now, if some files with checksum xattrs get
modified, they are not signed with new signature as the feature is off.

At this point, if the feature is enabled again, the bit rot daemon will
go and sign those files which does not have any bit-rot specific xattrs
(i.e. those files which were created after bit-rot was disabled). Whereas
the files with bit-rot xattrs wont get signed with proper new checksum.
At this point if scrubber runs, it finds the on disk checksum and the actual
checksum of the file to be different (because the file got modified) and
marks the file as corrupted.

FIX:

The fix is to unconditionally sign the files when the bit-rot daemon
comes up (instead of skipping the files with bit-rot xattrs).

Change-Id: Iadfb47dd39f7e2e77f22d549a4a07a385284f4f5
fixes: bz#1700078
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>

Mostly, unlock before logging.
In some cases, moved different code that was not needed
to be under lock (for example, taking time, or malloc'ing)
to be executed before taking the lock.

Note: logging might be slightly less accurate in order, since it may
not be done now under the lock, so order of logs is racy. I think
it's a reasonable compromise.

Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I2438710016afc9f4f62a176ef1a0d3ed793b4f89

updates: bz#1622665
Change-Id: I9f3a75ed9be3d90f37843a140563c356830ef945
Signed-off-by: Amar Tumballi <amarts@redhat.com>