glusterfs.git/libglusterfs/src/common-utils.h, branch v5.5

core: make compute_cksum function op_version compatible

2019-03-11T00:31:01+00:00

Problem: commit 5a152a changed the mechansim of computing the
checksum. In heterogeneous cluster, peers are running into
rejected state because we have different cksum computation
mechansims in upgraded and non-upgraded nodes.

Solution: add a check for op-version so that all the nodes
in the cluster follow the same mechanism for computing the
cksum.

fixes: bz#1684569

> Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
> BUG: bz#1685120
> Signed-off-by: Sanju Rakonde 

Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
Signed-off-by: Sanju Rakonde

core: libuuid-devel breakage

2018-10-18T13:25:23+00:00

The #include "uuid.h" left over from using .../contrib/uuid is debatably
incorrect now that we use the "system header" file /usr/include/uuid/uuid.h
from libuuid-devel.

Unfortunately this is complicated by things like FreeBSD having its own
/usr/include/uuid.h, and the e2fsprogs-libuuid uuid.h in installed - as
most third-party packages in FreeBSD are - in /usr/local as
/usr/local/include/uuid/uuid.h

With a system header file it should at least be #include , and
even better as #include , much like the way 
and  are included. Using #include  guarantees
not getting the /usr/include/uuid.h on FreeBSD, but clang/cc knows to
find "system" header files like this in /usr/local/include; with or
without the -I/... from uuid.pc. Also using #include "uuid.h" leaves
the compiler free to find a uuid.h from any -I option it might be passed.
(Fortunately we don't have any at this time.)

As we now require libuuid-devel or e2fsprogs-libuuid and configure will
exit with an error if the uuid.pc file doesn't exist, the HAVE_LIBUUID
(including the #elif FreeBSD) tests in compat-uuid.h are redundant. We
are guaranteed to have it, so testing for it is a bit silly IMO. It may
also break building third party configure scripts if they omit defining
it. (Just how hard do we want to make things for third party developers?)

Change-Id: I7317f63c806281a5d27de7d3b2208d86965545e1
updates: bz#1639688
Signed-off-by: Kaleb S. KEITHLEY

Land clang-format changes

2018-09-12T11:52:48+00:00

Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b

features/uss: Use xxh64 to generate gfid instead of md5sum

2018-09-05T13:13:21+00:00

* This is to ensure FIPS support
* Also changed the signature of svs_uuid_generate to
  get xlator argument
* Added xxh64 wrapper functions in common-utils to
  generate gfid using xxh64
  - Those wrapper functions can be used by other xlators
    as well to generate gfids using xxh64. But as of now
    snapview-server is going to be the only consumer.

Change-Id: Ide66573125dd74122430cccc4c4dc2a376d642a2
Updates: #230
Signed-off-by: Raghavendra Manjunath 
Signed-off-by: Raghavendra Bhat

libglusterfs/src/common-utils.c: strncpy -> sprintf, remove dead code.

2018-08-20T03:12:21+00:00

strncpy may not be very efficient for sort strings copied into
a large buffer: If the length of src is less than n,
strncpy() writes additional null bytes to dest to ensure
that a total of n bytes are written.

Instead, do a quick calc to see how much we really need and use
snprintf() to copy as much.

Also, move from CALLOC to MALLOC, as we are writing to this newly
allocated memory right away and add terminating null.

Lastly, removed some dead code. I did the same optimization as above
to it, only to find out no one is using it.

Compile-tested only!

Change-Id: Ib91b9a73c3d74c511fd067446b1bf6c2e1802687
updates: bz#1193929
Signed-off-by: Yaniv Kaul

contrib/xxhash: update to latest xxHash (0.6.5)

2018-08-16T04:06:13+00:00

Update to latest xxHash, which is supposed to faster with small keys.

Specifically, updated to 3064d42e7d74b0921bdd1818395d9cb37bb8976a,
which is a bit higher than 0.6.5.

Compiled hopefully with namespace (XXH_NAMESPACE=GF_),
which allows to use XXH() funcs with no fear they'll 'leak'
from our library.

Only compile tested!

xxhsum is modified to display messages which was conflicting
with regression tests (TAP harness). So modified the
gfid2path_fuse.t and gfid2path_nfs.t to adhere to that.

updates: bz#1193929
Signed-off-by: Yaniv Kaul 

Change-Id: I35cea5cc93f338c1023ac2c9bc6d7d13225a967b

core (named threads): flood of -Wformat-truncation warnings with gcc-7.1

2018-07-23T03:01:10+00:00

Starting in Fedora 26 which has gcc-7.1.x, -Wformat-trunction is enabled
with -Wformat, resulting in a flood of new warnings. This many warnings
is a concern because it makes it hard(er) to see other warnings that
should be addressed.

An example is at
https://kojipkgs.fedoraproject.org//packages/glusterfs/3.12.0/1.fc28/data/logs/x86_64/build.log

For more info see https://review.gluster.org/#/c/18267/

I can't find much (or good) documentation on the heuristics the
compiler uses for this warning. In the case of printing integer types
it appears it looks at the available space in the destination and the
range of values for the variable and/or its type.

To address the specific question about why 0x3ff versus 0xfff to mask
the value, either would suffice to hint to the compiler that the
printed value will fit in three characters. But the loop is from
0...1023 (or 0...0x3ff if you prefer) so I chose that as a more
"accurate" mask to use as it exactly matches the range of values of
the loop.

Fixes: bz#1492847
Change-Id: I6e309ba42159841131d8241bfc0566ef09e00aa9

logging: check for fmts not being NULL

2018-07-16T03:36:27+00:00

this fix is just a review of possible SIGSEGV issues in line 714
as per crash report at the bug:

```
08:35:25 Program terminated with signal 11, Segmentation fault.
08:35:25 #0  0x00007f4ebb491c5c in gf_time_fmt (dst=0x7f4eb1ff9a90 "", sz_dst=256, utime=1531470915, fmt=0)
at /home/jenkins/root/workspace/centos7-regression/libglusterfs/src/common-utils.h:714
```

fixes: bz#1600878
Change-Id: I160c391f8ac1a3456e59103d293b24e0e3fae718
Signed-off-by: Amar Tumballi

features/shard: Introducing ".shard/.remove_me" for atomic shard deletion (part 1)

2018-06-13T09:57:14+00:00

PROBLEM:
Shards are deleted synchronously when a sharded file is unlinked or
when a sharded file participating as the dst in a rename() is going to
be replaced. The problem with this approach is it makes the operation
really slow, sometimes causing the application to time out, especially
with large files.

SOLUTION:
To make this operation atomic, we introduce a ".remove_me" directory.
Now renames and unlinks will simply involve two steps:
1. creating an empty file under .remove_me named after the gfid of the file
participating in unlink/rename
2. carrying out the actual rename/unlink
A synctask is created (more on that in part 2) to scan this directory
after every unlink/rename operation (or upon a volume mount) and clean
up all shards associated with it. All of this happens in the background.
The task takes care to delete the shards associated with the gfid in
.remove_me only if this gfid doesn't exist in backend, ensuring that the
file was successfully renamed/unlinked and its shards can be discarded now
safely.

Change-Id: Ia1d238b721a3e99f951a73abbe199e4245f51a3a
updates: bz#1568521
Signed-off-by: Krutika Dhananjay

core: make glfs_iobuf_copy() consumable for general purpose.

2018-05-24T09:32:09+00:00

Currently plugins for cloudsync will be using it to write back data
downloaded from remote store/cloud.

Change-Id: I59f10bebed21b19568c94cbf29e3d536d5570749
Updates: #387
Signed-off-by: Susant Palai