glusterfs.git/libglusterfs/src/dict.h, branch v3.6.2beta1

features/marker: Filter internal xattrs in lookup

2014-11-17T07:35:25+00:00

        Backport of http://review.gluster.org/9061

Afr should ignore quota-size-key as part of self-heal
but should heal quota-limit key.

BUG: 1163569
Change-Id: I93d203002eac4fe20b70730c27c852d783c16d7f
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9110
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

dict: add dict_set_dynstr_with_alloc

2014-04-14T16:15:14+00:00

There is an overwhelming no. of instances of the following pattern in
glusterd module.

    ...

    char *dynstr = gf_strdup (str);
    if (!dynstr)
       goto err;
    ret = dict_set_dynstr (dict, key, dynstr);
    if (ret)
       goto err;

    ...

With this changes it would look as below,

   ret = dict_set_dynstr_with_alloc (dict, key, str);
   if (ret)
       goto err;

Change-Id: I6a47b1cbab4834badadc48c56d0b5c8c06c6dd4d
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/7379
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

features/quota: Improvements to quota

2013-11-26T18:24:02+00:00

* Two stages of quota enforcement is done:

  Soft and hard quota Upon reaching soft quota limit on the directory
  it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log)
  and no more writes allowed after hard
  quota limit. After reaching the soft-limit the daemon alerts the
  user/admin repeatively for every 'alert-time', which is
  configurable.

* Quota enforcer is moved to server-side.

  It  takes care of enforcing quota. Since enforcer doesn't have the
  cluster view, it relies on another service called
  quota-aggregator. Aggregator, on query can return the size of a
  directory based on the cluster view.

  Enforcer is always loaded in the server graph and is by passed if
  the feature is not enabled.

  Options specific to enforcer:

  server-quota - Specifies whether the feature is on/off. It is used
  to by pass the quota if turned off.

  deem-statfs - If set to on, it takes quota limits into consideration
  while estimating fs size. (df command). The algorithm followed is,
  i.   Adjust statvfs based on limit configured on root.
  ii.  If limit is set on the inode passed, use size/limits on that inode to
       populate statvfs. Otherwise, use size/limits configured on root.
  iii. Upon statvfs, update the ctx->size on the inode.
  iv.  Don't let DHT aggregate, instead take the maximum of the usages from the
       subvols of the DHT, since each of it contains the complete information.

  Enforcer also makes use of gfid-to-path conversion functionality to
  work correctly when a client like nfs predominently relies on
  nameless lookups.

* Quota Aggregator acts as a thin client to provide cluster view

  Its a lightweight *gluster client* process with no mount point,
  started upon enabling quota or restarting the volume. This is a
  single process run on each brick, which can answer queries on all
  volumes in the cluster. Its volfile stored in
  GLUSTERD_DEFAULT_WORKING_DIR/quotad/quotad.vol.

Credits:
Raghavendra Bhat        
Varun Shastry           
Shishir Gowda           
Kruthika Dhananjay      
Brian Foster            
Krishnan Parthasarathi  

Change-Id: Id1cb25b414951da34c665a55f77385d482e0f9de
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5952
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd: Introduce volume op-versions

2013-04-26T08:45:06+00:00

Each volume is now associated with two op-versions,
 * op_version - the op-version of the highest op-versioned feature enabled
 * client_op_version - the op-version of the highest op-versioned feature
   enabled which affects the clients only.

These two op-versions are generated dynamically and kept updated during
runtime. Glusterd now uses the respective volumes' client-op-version during
getspec requests.

To achieve the above a new field in the vme table is introduced,
client_option, this boolean field tells if the option is a client side
option.

Change-Id: I12c83b1dd29ab506026efd50d448cebbcee53c27
BUG: 907311
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/4584
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

core: remove all the 'inner' functions in codebase

2012-12-20T07:35:29+00:00

* move 'dict_keys_join()' from api/glfs_fops.c to libglusterfs/dict.c
  - also added an argument which is treated as a filter function if
    required, currently useful for fuse.

* now 'make CFLAGS="-std=gnu99 -pedantic" 2>&1 | grep nested' gives
  no output.

Change-Id: I4e18496fbd93ae1d3942026ef4931889cba015e8
Signed-off-by: Amar Tumballi 
BUG: 875913
Reviewed-on: http://review.gluster.org/4187
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

dict: add new API 'dict_add()'

2012-10-03T03:07:16+00:00

* this new API is used only when we are sure that there is no
  replacing keys, ie. in dict_unserialize().

Change-Id: I383dffc65056ebdaf0ab19727f7dc14ec7017fc1
Signed-off-by: Amar Tumballi 
BUG: 850917
Reviewed-on: http://review.gluster.org/3844
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

dict: no need to memdup() the XDR buffer for dict_unserialize

2012-10-01T19:51:54+00:00

* current memdup() of full buffer is not required as
  'dict_unserialize()' does its own memdup() of the value.

Change-Id: Iba866b68b753d060ee35bb300ee687e7d293ac82
Signed-off-by: Amar Tumballi 
BUG: 850917
Reviewed-on: http://review.gluster.org/3845
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Anand Avati

libglusterfs/dict: make 'dict_t' a opaque object

2012-09-06T07:34:15+00:00

* ie, don't dereference dict_t pointer, instead use APIs everywhere
* other than dict_t only 'data_t' should be the valid export from dict.h

* added 'dict_foreach_fnmatch()' API
* changed dict_lookup() to use data_t, instead of data_pair_t

Change-Id: I400bb0dd55519a7c5d2a107e67c8e7a7207228dc
Signed-off-by: Amar Tumballi 
BUG: 850917
Reviewed-on: http://review.gluster.org/3829
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

calls to dict_allocate_and_serialize() are not 64-bit clean

2012-07-12T07:31:47+00:00

All calls to dict_allocate_and_serialize() pass the address of a 32-bit
type, but must cast it to the 64-bit pointer type (size_t *).

This happens to work on LE machines, but even if it's apparently benign,
it's still a bug. On BE machines it is not benign.

GF_PROTOCOL_DICT_SERIALIZE() hacks around it by creating a size_t temp
var, but that's, well, a hack, IMO when you consider that all the callers
are actually passing &; the param should just be a u_int * and
eliminate the buggy casts and the temp var in the macro.

Nobody apparently uses the Fedora/EPEL PPC RPMs, but they might. People
are trying to build gluster.org bits on SPARC and tripping over this.

Change-Id: I92ea139f9e3e91ddbbb32a51b96fa582a9515626
Signed-off-by: Kaleb S. KEITHLEY 
BUG: 838928
Reviewed-on: http://review.gluster.com/3642
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

Optimize for small dicts, and avoid an overrun.

2012-06-01T19:15:07+00:00

As dicts get used more and more in the I/O path (especially for xattrs and
the new xdata feature), removing some of their inherent inefficiency
becomes more important.  This patch addresses some of the issues around
allocating data_pair_t structures separately.  Along the way, I found that
the way we're allocating the "members" hash table was subtly wrong, and
could lead to a memory overrun.  This is a latent bug because nobody uses
dict_get_new_full that way, but I added an assert to guard against that
possibility.  One beneficial side effect is that we now save four pointers'
worth of space per dict, offsetting the extra space used for the new
members.

Change-Id: Ie8c3e49f1e584daec4b0d2d8ce9dafbc76fb57b2
BUG: 827448
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.com/3040
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati