glusterfs.git/xlators/cluster/ec/src/ec-helpers.c, branch v3.6.4beta1

ec: Special handling of anonymous fd

2015-03-30T07:22:32+00:00

Anonymous file descriptors need to be handled specially because
they can be used in some non standard ways (i.e. an anonymous fd
can be used without having been opened).

This caused NFS to fail on some operations because ec always
expected to have a previous successful opendir call (from patch
http://review.gluster.org/9098/).

This patch treats all anonymous fd as opened on all subvolumes.

This is a backport of http://review.gluster.org/9513/

Change-Id: I09dbbce2ffc1ae3a5bcbb328bed55b84f4f0b9f8
BUG: 1187526
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9596
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

ec: Fix failures with missing files

2015-02-11T09:20:24+00:00

When a file does not exist on a brick but it does on others, there
could be problems trying to access it because there was some loc_t
structures with null 'pargfid' but 'name' was set. This forced
inode resolution based on /name instead of  which
would be the correct one. To solve this problem, 'name' is always
set to NULL when 'pargfid' is not present.

Another problem was caused by an incorrect management of errors
while doing incremental locking. The only allowed error during an
incremental locking was ENOTCONN, but missing files on a brick can
be returned as ESTALE. This caused an EIO on the operation.

This patch doesn't care of errors during an incremental locking. At
the end of the operation it will check if there are enough successfully
locked bricks to continue or not.

This is a backport of http://review.gluster.org/9407/

Change-Id: I9360ebf8d819d219cea2d173c09bd37679a6f15a
BUG: 1183716
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9560
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Raghavendra Bhat

cluster/ec: Handle internal xattr get/set

2015-02-04T11:53:33+00:00

        Backport of http://review.gluster.org/9385

Problem:
Internal xattrs of EC like trusted.ec.size/config/version
can be modified by users and that can lead to misbehavior
in EC.

Fix:
Don't let the user modify the xattrs. Hide these xattrs
in getfattr outputs.

BUG: 1182490
Change-Id: Ie32ebb95ee67cabbb9488951097a517172b45bcf
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9455
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

ec: Fix self-healing issues.

2014-12-21T16:04:21+00:00

Three problems have been detected:

1. Self healing is executed in background, allowing the fop that
   detected the problem to continue without blocks nor delays.

   While this is quite interesting to avoid unnecessary delays,
   it can cause spurious failures of self-heal because it may
   try to recover a file inside a directory that a previous
   self-heal has not recovered yet, causing the file self-heal
   to fail.

2. When a partial self-heal is being executed on a directory,
   if a full self-heal is attempted, it won't be executed
   because another self-heal is already in process, so the
   directory won't be fully repaired.

3. Information contained in loc's of some fop's is not enough
   to do a complete self-heal.

To solve these problems, I've made some changes:

* Improved ec_loc_from_loc() to add all available information
  to a loc.

* Before healing an entry, it's parent is checked and partially
  healed if necessary to avoid failures.

* All heal requests received for the same inode while another
  self-heal is being processed are queued. When the first heal
  completes, all pending requests are answered using the results
  of the first heal (without full execution), unless the first
  heal was a partial heal. In this case all partial heals are
  answered, and the first full heal is processed normally.

* An special virtual xattr (not physically stored on bricks)
  named 'trusted.ec.heal' has been created to allow synchronous
  self-heal of files.

  Now, the recommended way to heal an entire volume is this:

    find  -d -exec getfattr -h -n trusted.ec.heal {} \;

Some minor changes:

* ec_loc_prepare() has been renamed to ec_loc_update().

* All loc management functions return 0 on success and -1 on
  error.

* Do not delay fop unlocks if heal is needed.

* Added basic ec xattrs initially on create, mkdir and mknod
  fops.

* Some coding style changes

This is a backport of http://review.gluster.org/9072/

Change-Id: I2a5fd9c57349a153710880d6ac4b1fa0c1475985
BUG: 1159484
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9073
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat

ec: Change license

2014-12-13T06:06:13+00:00

This is a backport of http://review.gluster.org/9201/

Change-Id: Iae90ade2421898417b53dec0417a610cf306c44b
BUG: 1170515
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9233
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-by: Raghavendra Bhat 
Tested-by: Raghavendra Bhat

ec: Fix self-heal issues

2014-10-22T08:05:19+00:00

Problem: Doing an 'ls' of a directory that has been modified while one
         of the bricks was down, sometimes returns the old directory
         contents.

Cause: Directories are not marked when they are modified as files are.
       The ec xlator balances requests amongst available and healthy
       bricks. Since there is no way to detect that a directory is
       out of date in one of the bricks, it is used from time to time
       to return the directory contents.

Solution: Basically the solution consists in use versioning information
          also for directories, however some additional changes have
          been necessary.

Changes:

 * Use directory versioning:

     This required to lock full directory instead of a single entry for
     all requests that add or remove entries from it. This is needed to
     allow atomic version update. This affects the following fops:

         create, mkdir, mknod, link, symlink, rename, unlink, rmdir

     Another side effect is that opendir requires to do a previous
     lookup to get versioning information and discard out of date
     bricks for subsequent readdir(p) calls.

 * Restrict directory self-heal:

     Till now, when one discrepancy was found in lookup, a self-heal
     was automatically started. This caused the versioning information
     of a bad directory to be healed instantly, making the original
     problem to reapear again.

     To solve this, when a missing directory is detected in one or more
     bricks on lookup or opendir fops, only a partial self-heal is
     performed on it. A partial self-heal basically creates the
     directory but does not restore any additional information.

     This avoids that an 'ls' could repair the directory and cause the
     problem to happen again. With this change, output of 'ls' is
     always consistent. However, since the directory has been created
     in the brick, this allows any other operation on it (create new
     files, for example) to succeed on all bricks and not add additional
     work to the self-heal process.

     To force a self-heal of a directory, any other operation must be
     done on it. For example a getxattr.

     With these changes, the correct healing procedure that would avoid
     inconsistent directory browsing consists on a post-order traversal
     of directoriesi being healed. This way, the directory contents will
     be healed before healing the directory itslef.

 * Additional changes to fix self-heal errors

     - Don't use fop->fd to decide between fd/loc.

         open, opendir and create have an fd, but the correct data is in
         loc.

     - Fix incorrect management of bad bricks per inode/fd.

     - Fix incorrect selection of fop's target bricks when there are bad
       bricks involved.

     - Improved ec_loc_parent() to always return a parent loc as
       complete as possible.

This is a backport of http://review.gluster.org/8916/

Change-Id: Iaf3df174d7857da57d4a87b4a8740a7048b366ad
BUG: 1149727
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8946
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System

ec: Add config information in an xattr

2014-09-24T05:46:58+00:00

To simplify backward compatibility of the ec xlator when some
parameter or the implementation itself is changed, a new xattr
is added to each file with the configuration needed to recover
it.

The new attribute is called 'trusted.ec.config', and it's a 64-bit
value containing the following information:

    8 bits: version of the config information (currently always 0)
    8 bits: algorithm used to encode the file (currently always 0)
    8 bits: size of the galois field (currently always 8)
    8 bits: number of bricks
    8 bits: redundancy
   24 bits: chunk size (currently 512)

This new xattr could allow, in a future version, to have different
configurations per file.

This is a backport of http://review.gluster.org/8770/

Change-Id: I8c12d40ff546cc201fc66caa367484be3d48aeb4
BUG: 1140862
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8825
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

ec: Fix some size_t vars to 64 bits even on 32 bits machines

2014-09-19T19:34:59+00:00

The 64 bits 'trusted.ec.size' extended attribute was incorrectly
computed on 32 bits machines due to an overflow on negative
numbers.

Also changed some potentially dangerous uses of size_t in other
places.

This is a backport of http://review.gluster.org/8738/

Change-Id: Id76cfe49a2f350e564b5c71d8c8644fb9ce86662
BUG: 1144407
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8779
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Vijay Bellur

ec: Optimize read/write performance

2014-09-16T17:14:28+00:00

This patch significantly improves performance of read/write
operations on a dispersed volume by reusing previous inodelk/
entrylk operations on the same inode/entry. This reduces the
latency of each individual operation considerably.

Inode version and size are also updated when needed instead
of on each request. This gives an additional boost.

This is a backport of http://review.gluster.org/8369/

Change-Id: I4b98d5508c86b53032e16e295f72a3f83fd8fcac
BUG: 1140844
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8746
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Dan Lambright

cluster/ec: Fix incorrect management of NFS requests

2014-08-11T11:34:13+00:00

Some operations, specially those comming from NFS, do not use a
regular fd and use an anonymous fd (i.e. a previous open call has
not been sent). Any context information created during open or
create will not be present on these fd's, so we simply return NULL
for contexts of those fd.

Also it seems that NFS can send write requests with a very big
buffer (higher that the default value of 128 KB). Some changes
have been made to correctly handle these large buffers.

This is a backport of http://review.gluster.org/8367.

Change-Id: I281476bd0d2cbaad231822248d6a616fcf5d4003
BUG: 1126734
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/8367
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8416