glusterfs.git/libglusterfs/src/dict.c, branch v3.8.10

libglusterfs: coverity fix

2016-04-14T12:55:50+00:00

fix missing varargs cleanup

CID 1124856: string overflow

CID 1124656: NULL return

CID 1124374: constant expression

Change-Id: Iead530c599bdfef05a40c68b892215f4e4f02247
BUG: 789278
Signed-off-by: Sakshi Bansal 
Reviewed-on: http://review.gluster.org/9630
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

tier/libgfdb: Replacing ASCII query file with binary

2015-11-06T18:30:45+00:00

Earlier, when the database was queried we used to save
all the queried records in an ASCII format in the query file.
This caused issues like filename having ASCII delimiter and used
to take a lot of space. The tier.c file also had a lot of parsing code.

Here we changed the format of the query file to binary.
All the logic of serialization and formating of query record is done
by libgfdb. Libgfdb provides API,
gfdb_write_query_record() and gfdb_read_query_record(),
which the user i.e tier migrator and CTR xlator can use to
write to and read from query file.
With this binary format we save on disk space i.e reduce to 50% atleast
as we are saving GFID's in binary format 16 bytes and not the string format
which takes 36 bytes + We are not saving path of the file + we are also saving on
ASCII delimiters.

The on disk format of query record is as follows,

+---------------------------------------------------------------------------+
| Length of serialized query record |       Serialized Query Record         |
+---------------------------------------------------------------------------+
             4 bytes                     Length of serialized query record
                                                      |
                                                      |
     -------------------------------------------------|
     |
     |
     V
   Serialized Query Record Format:
   +---------------------------------------------------------------------------+
   | GFID |  Link count   |    |.....                      | FOOTER |
   +---------------------------------------------------------------------------+
     16 B        4 B         Link Length                                  4 B
                                |                                          |
                                |                                          |
   -----------------------------|                                          |
   |                                                                       |
   |                                                                       |
   V                                                                       |
   Each  will be serialized as                                  |
   +-----------------------------------------------+                       |
   | PGID | BASE_NAME_LENGTH |      BASE_NAME      |                       |
   +-----------------------------------------------+                       |
     16 B       4 B             BASE_NAME_LENGTH                           |
                                                                           |
                                                                           |
   ------------------------------------------------------------------------|
   |
   |
   V
   FOOTER is a magic number 0xBAADF00D indicating the end of the record.
   This also serves as a serialized schema validator.

Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a
BUG: 1272207
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12354
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

dict: dict_set_bin() should never free the pointer on error

2015-07-24T15:09:08+00:00

dict_set_bin() is handling the pointer that it passed inconsistently.
Depending on the errors that can occur, the pointer passed to the dict
can be free'd, but there is no guarantee.

It is cleaner to have the caller free the pointer that allocated it and
dict_set_bin() returned an error. When dict_set_bin() returned success,
the given pointer will be free'd when dict_unref() calls data_destroy().

Many callers of dict_set_bin() already take care of free'ing the pointer
on error. The ones that did not, are corrected with this change too.

Change-Id: I39a4f7ebc0cae6d403baba99307d7ce408f25966
BUG: 1242280
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11638
Tested-by: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Raghavendra G 
Tested-by: NetBSD Build System

dict: merge static/non-static dict_set_* variations

2015-07-24T15:08:24+00:00

Copy/pasting complete functions for setting one boolean is rather ugly,
this can easily be solved with a helper function that gets the boolean
as an extra parameter.

Also removing dict_set_dynmstr() because there are no callers, and using
GF_MALLOC() or GF_CALLOC() to allocate memory for strings is much more
preferred.

And, as Kaleb pointed out, prefixing functions with an underscore '_' is
reserved for libc internals. Renaming the _dict_*() functions for that
reason.

Change-Id: I8e681954502f17e59c8f5e8a294d3db238d3e5f3
BUG: 1232304
Reported-by: Kaleb KEITHLEY 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11716
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Jeff Darcy

libglusterfs: delete duplicate code

2015-07-12T21:28:44+00:00

This patch saves cpu cyles by deleting duplicated code in most frequently
used functions 'dict_set_static_bin' & 'dict_set_bin'  which is already
repeated/executed in function 'bin_to_data'

Change-Id: I8e18b3f2b321e06bb6bd5a67e1782cb159182dd2
BUG: 1232304
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/11255
Reviewed-by: Niels de Vos 
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System

call-stub,circ-buff,client_t,compat,dict/libglusterfs : Porting to a new logging framework

2015-06-25T08:50:53+00:00

Change-Id: Ie7d180e0ab2fed1270d66504606d1b2522884020
BUG: 1194640
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/10828
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Fix dictionary compare function

2015-05-05T02:46:32+00:00

If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not.  value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.

geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.

Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1207712
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10078
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

features/bit-rot: Implementation of bit-rot xlator

2015-03-24T17:55:32+00:00

This is the "Signer" -- responsible for signing files with their
checksums upon last file descriptor close (last release()).
The event notification facility provided by the changelog xlator
is made use of.

Moreover, checksums are as of now SHA256 hash of the object data
and is the only available hash at this point of time. Therefore,
there is no special "what hash to use" type check, although it's
does not take much to add various hashing algorithms to sign
objects with. Signatures are stored in extended attributes of the
objects along with the the type of hashing used to calculate the
signature. This makes thing future proof when other hash types
are added. The signature  infrastructure is provided by bitrot
stub: a little piece of code that sits over the POSIX xlator
providing interfaces to "get or set" objects signature and it's
staleness.

Since objects are signed upon receiving release() notification,
pre-existing data which are "never" modified would never be
signed. To counter this, an initial crawler thread is spawned
The crawler scans the entire brick for objects that are unsigned
or "missed" signing due to the server going offline (node reboots,
crashes, etc..) and triggers an explicit sign. This would also
sign objects when bit-rot is enabled for a volume and/or after
upgrade.

Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b
BUG: 1170075
Original-Author: Raghavendra Bhat 
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9711
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Add tier translator.

2015-03-21T16:50:29+00:00

The tier translator shares most of DHT's code. It differs in how
subvolumes are chosen for I/Os, and how file migration (cache promotion
and demotion) is managed. That different functionality is split to either
DHT or tier logic according to the "tier_methods" structure.

A cache promotion and demotion thread is created in a manner
similar to the rebalance daemon. The thread operates a timing
wheel which periodically checks for promotion and demotion candidates
(files). Candidates are queued and then migrated. Candidates must exist on
the same node as the daemon and meet other critera per caching policies.

This patch has two authors (Dan Lambright and Joseph Fernandes). Dan
did the DHT changes and Joe wrote the cache policies. The fix depends on
DHT readidr changes and the database library which have been submitted
separately.  Header files in libglusterfs/src/gfdb should be reviewed in
patch 9683.

For more background and design see the feature page [1].

[1]
http://www.gluster.org/community/documentation/index.php/Features/data-classification

Change-Id: Icc26c517ccecf5c42aef039f5b9c6f7afe83e46c
BUG: 1194753
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9724
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur