glusterfs.git/xlators/cluster/ec/src/ec.c, branch v3.7.1

cluster/ec: Fix all EIO errors in EC

2015-05-28T11:12:06+00:00

        Backport of http://review.gluster.org/10770
        Backport of http://review.gluster.org/10806
        Backport of http://review.gluster.org/10787
        Backport of http://review.gluster.org/10868
        Backport of http://review.gluster.com/10852

 - When a blocking lock is requested, lock request is succeeded even when
ec->fragment number of locks are acquired successfully in non-blocking locking
phase. This will lead to fop succeeding only on the bricks where the locks are
acquired, leading to the necessity of self-heals. To prevent these un-necessary
self-heals, if the remaining locks fail with EAGAIN in non-blocking lock phase
try blocking locking phase instead.

 -  Handle lookup failures while op in progress

 - cluster/ec: Correctly cleanup delayed locks
When a delayed lock is pending, a graph switch doesn't correctly
terminate it. This means that the update of version and size xattrs
is lost, causing EIO errors. This patch handles GF_EVENT_PARENT_DOWN
event to correctly finish pending udpdates before completing the
graph switch.

 - Fix use after free crash
ec_heal creates ec_fop_data but doesn't run ec_manager. ec_fop_data_allocate
adds this fop to ec->pending_fops, because ec_manager is not run on this heal
fop it is never removed from ec->pending_fops. When it is accessed after free
it leads to crash. It is better to not to add HEAL fops to ec->pending_fops
because we don't want graph switch to hang the mount because of a BIG
file/directory heal.

- Forced unlock when lock contention is detected
EC uses an eager lock mechanism to optimize multiple read/write
requests on the same entry or inode. This increases performance
but can have adverse results when other clients try to access the
same entry/inode. To solve this, this patch adds a functionality
to detect when this happens and force an earlier release to not
block other clients.

The method consists on requesting GF_GLUSTERFS_INODELK_COUNT and
GF_GLUSTERFS_ENTRYLK_COUNT for all fops that take a lock. When this
count is greater than one, the lock is marked to be released. All
fops already waiting for this lock will be executed normally before
releasing the lock, but new requests that also require it will be
blocked and restarted after the lock has been released and reacquired
again.

Another problem was that some operations did correctly lock the
parent of an entry when needed, but got the size and version xattrs
from the entry instead of the parent.

This patch solves this problem by binding all queries of size and
version to each lock and replacing all entrylk calls by inodelk ones
to remove concurrent updates on directory metadata.  This also allows
rename to correctly update source and destination directories.

BUG: 1225279
Change-Id: I02a6084b138dd38e018a462347cd9ce38610c7ef
Reviewed-on: http://review.gluster.org/10926
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Fix dictionary compare function

2015-05-08T22:03:16+00:00

If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not.  value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.

geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.

Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1216303
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10078
Reviewed-on: http://review.gluster.org/10690
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

cluster/ec: Fix readdir de-itransform

2015-04-11T09:11:15+00:00

Problem:
gf_deitransform returns the glbal client-id in the complete graph. So except
for the first disperse subvolume under dht, all the other disperse subvolumes
will return a client-id greater than ec->nodes, so readdir will always error
out in those subvolumes.

Fix:
Get the client subvolume whose client-id matches the client-id returned by
gf_deitransform of offset.

Change-Id: I26aa17504352d48d7ff14b390b62f49d7ab2d699
BUG: 1209113
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10165
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Implement heal info for ec

2015-03-31T06:38:49+00:00

This also lists the files that are on-going I/O, which
will be fixed later.

Change-Id: Ib3f60a8b7e8798d068658cf38eaef2a904f9e327
BUG: 1203581
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10020
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

libxlator: Change marker xattr handling interface

2015-03-25T18:10:40+00:00

- Changed the implementation of marker xattr handling to take just a
  function which populates important data that is different from
  default 'gauge' values and subvolumes where the call needs to be
  wound.
- Removed duplicate code I found while reading the code and moved it to
  cluster_marker_unwind. Removed unused structure members.
- Changed dht/afr/stripe implementations to follow the new implementation
- Implemented marker xattr handling for ec.

Change-Id: Ib0c3626fe31eb7c8aae841eabb694945bf23abd4
BUG: 1200372
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9892
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

cluster/ec: Add self-heal-daemon command handlers

2015-03-09T22:36:31+00:00

This patch introduces the changes required in ec xlator to handle
index/full heal.

Index healer threads:
Ec xlator start an index healer thread per local brick. This thread keeps
waking up every minute to check if there are any files to be healed based on
the indices kept in index directory. Whenever child_up event comes, then also
this index healer thread wakes up and crawls the indices and triggers heal.
When self-heal-daemon is disabled on this particular volume then the healer
thread keeps waiting until it is enabled again to perform heals.

Full healer threads:
Ec xlator starts a full healer thread for the local subvolume provided by
glusterd to perform full crawl on the directory hierarchy to perform heals.
Once the crawl completes the thread exits if no more full heals are issued.

Changed xl-op prefix GF_AFR_OP to GF_SHD_OP to make it more generic.

Change-Id: Idf9b2735d779a6253717be064173dfde6f8f824b
BUG: 1177601
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9787
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

cluster/ec: Wait for all bricks to notify before notifying parent

2015-02-02T20:22:02+00:00

This is to prevent spurious heals that can result in self-heal.

Change-Id: I0b27c1c1fc7a58e2683cb1ca135117a85efcc6c9
BUG: 1179180
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9523
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

cluster/ec: Handle CHILD UP/DOWN in all cases

2015-01-29T03:49:53+00:00

Problem:
When all the bricks are down at the time of mounting the volume, then mount
command hangs.

Fix:
1. Ignore all CHILD_CONNECTING events comming from subvolumes.
2. On timer expiration (without enough up or down childs) send
   CHILD_DOWN.
3. Once enough up or down subvolumes are detected, send the appropriate event.
   When rest of the subvols go up/down without changing the overall
   ec-up/ec-down send CHILD_MODIFIED to parent subvols.

Change-Id: Ie0194dbadef2dce36ab5eb7beece84a6bf3c631c
BUG: 1179180
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9396
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System

mgmt/glusterd: Implement Volume heal enable/disable

2015-01-20T10:24:24+00:00

For volumes with replicate, disperse xlators, self-heal daemon should do
healing. This patch provides enable/disable functionality for the xlators to be
part of self-heal-daemon. Replicate already had this functionality with
'gluster volume set cluster.self-heal-daemon on/off'. But this patch makes it
uniform for both types of volumes. Internally it still does 'volume set' based
on the volume type.

Change-Id: Ie0f3799b74c2afef9ac658ef3d50dce3e8072b29
BUG: 1177601
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9358
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Tested-by: Krishnan Parthasarathi

cluster/ec: Handle internal xattr get/set

2015-01-09T05:55:37+00:00

Problem:
Internal xattrs of EC like trusted.ec.size/config/version
can be modified by users and that can lead to misbehavior
in EC.

Fix:
Don't let the user modify the xattrs. Hide these xattrs
in getfattr outputs.

Change-Id: I39cec96ae12826b506b496fda7da74201015fd75
BUG: 1178688
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/9385
Tested-by: Gluster Build System 
Reviewed-by: Emmanuel Dreyfus 
Tested-by: Emmanuel Dreyfus 
Reviewed-by: Xavier Hernandez