glusterfs.git/libglusterfs/src/dict.h, branch v3.7.17

call-stub,circ-buff,client_t,compat,dict/libglusterfs : Porting to a new logging framework

2015-07-09T07:11:04+00:00

        Backport of http://review.gluster.org/10828

Cherry picked from a1e32fbcfbfaf6e4c63e140b3b90a80dc748a269
>Change-Id: Ie7d180e0ab2fed1270d66504606d1b2522884020
>BUG: 1194640
>Signed-off-by: Mohamed Ashiq 
>Reviewed-on: http://review.gluster.org/10828
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 

Change-Id: Ie7d180e0ab2fed1270d66504606d1b2522884020
BUG: 1217722
Signed-off-by: Mohamed Ashiq 
Reviewed-on: http://review.gluster.org/11406
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Fix dictionary compare function

2015-05-08T22:03:16+00:00

If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not.  value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.

geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.

Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1216303
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10078
Reviewed-on: http://review.gluster.org/10690
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

features/bit-rot: Implementation of bit-rot xlator

2015-03-24T17:55:32+00:00

This is the "Signer" -- responsible for signing files with their
checksums upon last file descriptor close (last release()).
The event notification facility provided by the changelog xlator
is made use of.

Moreover, checksums are as of now SHA256 hash of the object data
and is the only available hash at this point of time. Therefore,
there is no special "what hash to use" type check, although it's
does not take much to add various hashing algorithms to sign
objects with. Signatures are stored in extended attributes of the
objects along with the the type of hashing used to calculate the
signature. This makes thing future proof when other hash types
are added. The signature  infrastructure is provided by bitrot
stub: a little piece of code that sits over the POSIX xlator
providing interfaces to "get or set" objects signature and it's
staleness.

Since objects are signed upon receiving release() notification,
pre-existing data which are "never" modified would never be
signed. To counter this, an initial crawler thread is spawned
The crawler scans the entire brick for objects that are unsigned
or "missed" signing due to the server going offline (node reboots,
crashes, etc..) and triggers an explicit sign. This would also
sign objects when bit-rot is enabled for a volume and/or after
upgrade.

Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b
BUG: 1170075
Original-Author: Raghavendra Bhat 
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9711
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Add tier translator.

2015-03-21T16:50:29+00:00

The tier translator shares most of DHT's code. It differs in how
subvolumes are chosen for I/Os, and how file migration (cache promotion
and demotion) is managed. That different functionality is split to either
DHT or tier logic according to the "tier_methods" structure.

A cache promotion and demotion thread is created in a manner
similar to the rebalance daemon. The thread operates a timing
wheel which periodically checks for promotion and demotion candidates
(files). Candidates are queued and then migrated. Candidates must exist on
the same node as the daemon and meet other critera per caching policies.

This patch has two authors (Dan Lambright and Joseph Fernandes). Dan
did the DHT changes and Joe wrote the cache policies. The fix depends on
DHT readidr changes and the database library which have been submitted
separately.  Header files in libglusterfs/src/gfdb should be reviewed in
patch 9683.

For more background and design see the feature page [1].

[1]
http://www.gluster.org/community/documentation/index.php/Features/data-classification

Change-Id: Icc26c517ccecf5c42aef039f5b9c6f7afe83e46c
BUG: 1194753
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9724
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

features/index: Add option to track specific xattrs for xattrop64

2015-02-25T05:57:53+00:00

This enables trusted.ec.dirty to be tracked in index

Change-Id: Ief1619110859f6f9ccee3da229f0688b73e2124b
BUG: 1177601
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9602
Reviewed-by: Krutika Dhananjay 
Tested-by: Gluster Build System

features/marker: Filter internal xattrs in lookup

2014-11-11T11:33:41+00:00

Afr should ignore quota-size-key as part of self-heal
but should heal quota-limit key.

Change-Id: Ic0b06bd20a563a00d6bfdc2dc5a76c661e533ecb
BUG: 1161106
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9061
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

glusterd: statedump support

2014-10-15T10:02:40+00:00

Although glusterd currently has statedump support but it doesn't dump its
context information. Implementing glusterd_dump_priv function to export per-node
glusterd information would be useful for debugging bugs. Once implemented, we
could enhance sos-report to fetch this information. This would potentially
reduce our time to root cause and data needed for debugability can be dumped
gradually.

Following is the main items of the dump list targeted in this patch :

    * Supported max/min op-version and current op-version
    * Information about peer list
    * Information about peer list involved while a transaction is going on
      (xaction_peers)
    * option dictionary in glusterd_conf_t
    * mgmt_v3_lock in glusterd_conf_t
    * List of connected clients
    * uuid of glusterd
    * A section of rpc related information like live connections and their
      statistics

There are couple of issues which were found during implementation and testing
phase:
 - xaction_peers of glusterd_conf_t was not initialized in init because of which
   traversing through this list head was crashing when there was no active
   transaction
 - gf_free was not setting the typestr to NULL if the the alloc count becomes 0
   for a mem-type earlier allocated.

Change-Id: Ic9bce2d57682fc1771cd2bc6af0b7316ecbc761f
BUG: 1139682
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/8665
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

libglusterfs: wrong function definition of synclock_destory().

2014-08-06T08:29:28+00:00

synclock_destory() has been prototyped in syncop.h,
how-ever synclock_destroy() is the actual function used in syncop.c.
Correcting this function definition along with few typos.


Change-Id: I35a818190c1d37c303279ca7a820f01895751bd9
BUG: 1075417
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/8266
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: soumya k 
Reviewed-by: Poornima G 
Reviewed-by: Vijay Bellur

dict: add dict_set_dynstr_with_alloc

2014-04-14T16:15:14+00:00

There is an overwhelming no. of instances of the following pattern in
glusterd module.

    ...

    char *dynstr = gf_strdup (str);
    if (!dynstr)
       goto err;
    ret = dict_set_dynstr (dict, key, dynstr);
    if (ret)
       goto err;

    ...

With this changes it would look as below,

   ret = dict_set_dynstr_with_alloc (dict, key, str);
   if (ret)
       goto err;

Change-Id: I6a47b1cbab4834badadc48c56d0b5c8c06c6dd4d
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/7379
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

features/quota: Improvements to quota

2013-11-26T18:24:02+00:00

* Two stages of quota enforcement is done:

  Soft and hard quota Upon reaching soft quota limit on the directory
  it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log)
  and no more writes allowed after hard
  quota limit. After reaching the soft-limit the daemon alerts the
  user/admin repeatively for every 'alert-time', which is
  configurable.

* Quota enforcer is moved to server-side.

  It  takes care of enforcing quota. Since enforcer doesn't have the
  cluster view, it relies on another service called
  quota-aggregator. Aggregator, on query can return the size of a
  directory based on the cluster view.

  Enforcer is always loaded in the server graph and is by passed if
  the feature is not enabled.

  Options specific to enforcer:

  server-quota - Specifies whether the feature is on/off. It is used
  to by pass the quota if turned off.

  deem-statfs - If set to on, it takes quota limits into consideration
  while estimating fs size. (df command). The algorithm followed is,
  i.   Adjust statvfs based on limit configured on root.
  ii.  If limit is set on the inode passed, use size/limits on that inode to
       populate statvfs. Otherwise, use size/limits configured on root.
  iii. Upon statvfs, update the ctx->size on the inode.
  iv.  Don't let DHT aggregate, instead take the maximum of the usages from the
       subvols of the DHT, since each of it contains the complete information.

  Enforcer also makes use of gfid-to-path conversion functionality to
  work correctly when a client like nfs predominently relies on
  nameless lookups.

* Quota Aggregator acts as a thin client to provide cluster view

  Its a lightweight *gluster client* process with no mount point,
  started upon enabling quota or restarting the volume. This is a
  single process run on each brick, which can answer queries on all
  volumes in the cluster. Its volfile stored in
  GLUSTERD_DEFAULT_WORKING_DIR/quotad/quotad.vol.

Credits:
Raghavendra Bhat        
Varun Shastry           
Shishir Gowda           
Kruthika Dhananjay      
Brian Foster            
Krishnan Parthasarathi  

Change-Id: Id1cb25b414951da34c665a55f77385d482e0f9de
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5952
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati