glusterfs.git/xlators/cluster/ec/src/ec.h, branch v3.7.0alpha0

cluster/ec: Fix readdir de-itransform

2015-04-11T09:11:15+00:00

Problem:
gf_deitransform returns the glbal client-id in the complete graph. So except
for the first disperse subvolume under dht, all the other disperse subvolumes
will return a client-id greater than ec->nodes, so readdir will always error
out in those subvolumes.

Fix:
Get the client subvolume whose client-id matches the client-id returned by
gf_deitransform of offset.

Change-Id: I26aa17504352d48d7ff14b390b62f49d7ab2d699
BUG: 1209113
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10165
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

libxlator: Change marker xattr handling interface

2015-03-25T18:10:40+00:00

- Changed the implementation of marker xattr handling to take just a
  function which populates important data that is different from
  default 'gauge' values and subvolumes where the call needs to be
  wound.
- Removed duplicate code I found while reading the code and moved it to
  cluster_marker_unwind. Removed unused structure members.
- Changed dht/afr/stripe implementations to follow the new implementation
- Implemented marker xattr handling for ec.

Change-Id: Ib0c3626fe31eb7c8aae841eabb694945bf23abd4
BUG: 1200372
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9892
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

cluster/ec: Add self-heal-daemon command handlers

2015-03-09T22:36:31+00:00

This patch introduces the changes required in ec xlator to handle
index/full heal.

Index healer threads:
Ec xlator start an index healer thread per local brick. This thread keeps
waking up every minute to check if there are any files to be healed based on
the indices kept in index directory. Whenever child_up event comes, then also
this index healer thread wakes up and crawls the indices and triggers heal.
When self-heal-daemon is disabled on this particular volume then the healer
thread keeps waiting until it is enabled again to perform heals.

Full healer threads:
Ec xlator starts a full healer thread for the local subvolume provided by
glusterd to perform full crawl on the directory hierarchy to perform heals.
Once the crawl completes the thread exits if no more full heals are issued.

Changed xl-op prefix GF_AFR_OP to GF_SHD_OP to make it more generic.

Change-Id: Idf9b2735d779a6253717be064173dfde6f8f824b
BUG: 1177601
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9787
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

ec: Add trusted.ec.dirty xattr

2015-02-24T07:58:11+00:00

This xattr will be incremented before each data modifying operation and
decremented after it. This will add the possibility to detect partially
updated writes and refuse them on reads.

It will also be useful for interacting with index xlator and have a way
to heal dispersed files from the self-heal daemon.

Change-Id: Ie644a8dd074ae0f254c809c5863bdb030be5486a
BUG: 1190581
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9607
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Vijay Bellur

cluster/ec: Handle CHILD UP/DOWN in all cases

2015-01-29T03:49:53+00:00

Problem:
When all the bricks are down at the time of mounting the volume, then mount
command hangs.

Fix:
1. Ignore all CHILD_CONNECTING events comming from subvolumes.
2. On timer expiration (without enough up or down childs) send
   CHILD_DOWN.
3. Once enough up or down subvolumes are detected, send the appropriate event.
   When rest of the subvols go up/down without changing the overall
   ec-up/ec-down send CHILD_MODIFIED to parent subvols.

Change-Id: Ie0194dbadef2dce36ab5eb7beece84a6bf3c631c
BUG: 1179180
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9396
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System

mgmt/glusterd: Implement Volume heal enable/disable

2015-01-20T10:24:24+00:00

For volumes with replicate, disperse xlators, self-heal daemon should do
healing. This patch provides enable/disable functionality for the xlators to be
part of self-heal-daemon. Replicate already had this functionality with
'gluster volume set cluster.self-heal-daemon on/off'. But this patch makes it
uniform for both types of volumes. Internally it still does 'volume set' based
on the volume type.

Change-Id: Ie0f3799b74c2afef9ac658ef3d50dce3e8072b29
BUG: 1177601
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9358
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Tested-by: Krishnan Parthasarathi

cluster/ec: Handle internal xattr get/set

2015-01-09T05:55:37+00:00

Problem:
Internal xattrs of EC like trusted.ec.size/config/version
can be modified by users and that can lead to misbehavior
in EC.

Fix:
Don't let the user modify the xattrs. Hide these xattrs
in getfattr outputs.

Change-Id: I39cec96ae12826b506b496fda7da74201015fd75
BUG: 1178688
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9385
Tested-by: Gluster Build System 
Reviewed-by: Emmanuel Dreyfus 
Tested-by: Emmanuel Dreyfus 
Reviewed-by: Xavier Hernandez

ec: Fix self-healing issues.

2014-12-04T19:34:38+00:00

Three problems have been detected:

1. Self healing is executed in background, allowing the fop that
   detected the problem to continue without blocks nor delays.

   While this is quite interesting to avoid unnecessary delays,
   it can cause spurious failures of self-heal because it may
   try to recover a file inside a directory that a previous
   self-heal has not recovered yet, causing the file self-heal
   to fail.

2. When a partial self-heal is being executed on a directory,
   if a full self-heal is attempted, it won't be executed
   because another self-heal is already in process, so the
   directory won't be fully repaired.

3. Information contained in loc's of some fop's is not enough
   to do a complete self-heal.

To solve these problems, I've made some changes:

* Improved ec_loc_from_loc() to add all available information
  to a loc.

* Before healing an entry, it's parent is checked and partially
  healed if necessary to avoid failures.

* All heal requests received for the same inode while another
  self-heal is being processed are queued. When the first heal
  completes, all pending requests are answered using the results
  of the first heal (without full execution), unless the first
  heal was a partial heal. In this case all partial heals are
  answered, and the first full heal is processed normally.

* An special virtual xattr (not physically stored on bricks)
  named 'trusted.ec.heal' has been created to allow synchronous
  self-heal of files.

  Now, the recommended way to heal an entire volume is this:

    find  -d -exec getfattr -h -n trusted.ec.heal {} \;

Some minor changes:

* ec_loc_prepare() has been renamed to ec_loc_update().

* All loc management functions return 0 on success and -1 on
  error.

* Do not delay fop unlocks if heal is needed.

* Added basic ec xattrs initially on create, mkdir and mknod
  fops.

* Some coding style changes

Change-Id: I2a5fd9c57349a153710880d6ac4b1fa0c1475985
BUG: 1161588
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9072
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

ec: Change license

2014-12-03T18:45:26+00:00

Change-Id: Iae90ade2421898417b53dec0417a610cf306c44b
BUG: 1168167
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9201
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

ec: Add config information in an xattr

2014-09-23T16:12:32+00:00

To simplify backward compatibility of the ec xlator when some
parameter or the implementation itself is changed, a new xattr
is added to each file with the configuration needed to recover
it.

The new attribute is called 'trusted.ec.config', and it's a 64-bit
value containing the following information:

    8 bits: version of the config information (currently always 0)
    8 bits: algorithm used to encode the file (currently always 0)
    8 bits: size of the galois field (currently always 8)
    8 bits: number of bricks
    8 bits: redundancy
   24 bits: chunk size (currently 512)

This new xattr could allow, in a future version, to have different
configurations per file.

Change-Id: I8c12d40ff546cc201fc66caa367484be3d48aeb4
BUG: 1140861
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8770
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Vijay Bellur