glusterfs.git/xlators/cluster/ec/src/ec-data.h, branch v3.7.6

cluster/ec : Mark new entry changelog in entry self-heal

2015-10-06T16:38:42+00:00

Problem :
When a new entry is created dirty mark xattrs are not
created this will need full heal to be performed, even
when there are partial failures.

Solution :
Marks new entry changelog in self-heal.

PS: Also fixed erasing of dirty markers when no data heal
is required.

BUG: 1258313
Signed-off-by: Ashish Pandey 
Change-Id: I156e3d3201afa77efe118e1aaace1d91c90a9613
Reviewed-on: http://review.gluster.org/12306
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Fix write size in self-heal

2015-08-14T09:11:13+00:00

Self-heal was always using a fixed block size to heal a file. This
was incorrect for dispersed volumes with a number of data bricks not
being a power of 2.

This patch adjusts the block size to a multiple of the stripe size
of the volume. It also propagates errors detected during the data
heal to stop healing the file and not mark it as healed.

This is a backport if http//review.gluster.org/11862

Change-Id: I5104ae4bfed8585ca40cb45831ca20582566370c
BUG: 1236050
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11869
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Fix tracking of good bricks

2015-08-14T09:02:21+00:00

The bitmask of good and bad bricks was kept in the context of the
corresponding inode or fd. This was problematic when an external
process (another client or the self-heal process) did heal the
bricks but no one changed the bitmaks of other clients.

This patch removes the bitmask stored in the context and calculates
which bricks are healthy after locking them and doing the initial
xattrop. After that, it's updated using the result of each fop.

> Change-Id: I225e31cd219a12af4ca58871d8a4bb6f742b223c
> BUG: 1236065
> Signed-off-by: Xavier Hernandez 
> Reviewed-on: http://review.gluster.org/11844
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Pranith Kumar Karampuri 

Change-Id: Idbe68b28b865c4b28366703ad1e96ae16ba44b66
BUG: 1235964
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11867
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Prevent data corruptions

2015-07-22T06:27:42+00:00

- On lock reuse preserve 'healing' bits
- Don't set ctx->size outside locks in healing code
- Allow xattrop internal fops also on the fop->mask.

 >Change-Id: I6b76da5d7ebe367d8f3552cbf9fd18e556f2a171
 >BUG: 1232678
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/11640
 >Tested-by: NetBSD Build System 
 >Tested-by: Gluster Build System 
 >Reviewed-by: Xavier Hernandez 

BUG: 1243647
Change-Id: I1b3828e4d4a863b84b2c4e732e7965d1302cea47
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11686
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System

cluster/ec: wind readlink on good subvol(s)

2015-07-22T06:27:12+00:00

 >BUG: 1232172
 >Change-Id: I3a56e487840d86147dd85bf5fbe79b165eae289f
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/11589
 >Tested-by: NetBSD Build System 
 >Reviewed-by: Xavier Hernandez 
 >Tested-by: Gluster Build System 

BUG: 1234679
Change-Id: I08560eee095a3921e9c24f16dc2a242a76018a42
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11687
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Add throttling in background healing

2015-07-21T21:35:37+00:00

- 8 parallel heals can happen.
- 128 heals will wait for their turn
- Heals will be rejected if 128 heals are already waiting.

 >Change-Id: I2e99bf064db7bce71838ed9901a59ffd565ac390
 >BUG: 1237381
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/11471
 >Tested-by: Gluster Build System 
 >Reviewed-by: Xavier Hernandez 
 >Tested-by: NetBSD Build System 

BUG: 1238476
Change-Id: Id9625536197c1f5d6c3a9ed53f80bd9e6d14c507
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11679
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: wind fops on good subvols for access/readdir[p]

2015-07-02T06:42:17+00:00

        Backport of http://review.gluster.com/11246

BUG: 1234679
Change-Id: I2d774f62740c82e922efab50fc78fa74050ede93
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11357
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System

cluster/ec: EC_XATTR_DIRTY doesn't come in response

2015-06-07T05:36:07+00:00

         Backport of http://review.gluster.com/11078

Problem:
ec_update_size_version expects all the keys it did xattrop with to come in
response so that it can set the values again in ec_update_size_version_done.
But EC_XATTR_DIRTY is not combined so the value won't be present in the
response. So ctx->post/pre_dirty are not updated in
ec_update_size_version_done. So these values are still non-zero. When
ec_unlock_now is called as part of flush's unlock phase it again tries to
perform same xattrop for EC_XATTR_DIRTY. But ec_update_size_version is not
expected to be called in unlock phase of flush because ec_flush_size_version
should have reset everything to zero and unlock is never invoked from
ec_update_size_version_done for flush/fsync/fsyncdir. This leads to stale lock
which leads to hang.

Fix:
EC_XATTR_DIRTY is removed in ex_xattrop_cbk and is never combined with other
answers. So remove handling of this in the response.

BUG: 1228160
Change-Id: I657efca6e706e7acb541f98f526943f67562da9f
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11084
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

cluster/ec: Fix all EIO errors in EC

2015-05-28T11:12:06+00:00

        Backport of http://review.gluster.org/10770
        Backport of http://review.gluster.org/10806
        Backport of http://review.gluster.org/10787
        Backport of http://review.gluster.org/10868
        Backport of http://review.gluster.com/10852

 - When a blocking lock is requested, lock request is succeeded even when
ec->fragment number of locks are acquired successfully in non-blocking locking
phase. This will lead to fop succeeding only on the bricks where the locks are
acquired, leading to the necessity of self-heals. To prevent these un-necessary
self-heals, if the remaining locks fail with EAGAIN in non-blocking lock phase
try blocking locking phase instead.

 -  Handle lookup failures while op in progress

 - cluster/ec: Correctly cleanup delayed locks
When a delayed lock is pending, a graph switch doesn't correctly
terminate it. This means that the update of version and size xattrs
is lost, causing EIO errors. This patch handles GF_EVENT_PARENT_DOWN
event to correctly finish pending udpdates before completing the
graph switch.

 - Fix use after free crash
ec_heal creates ec_fop_data but doesn't run ec_manager. ec_fop_data_allocate
adds this fop to ec->pending_fops, because ec_manager is not run on this heal
fop it is never removed from ec->pending_fops. When it is accessed after free
it leads to crash. It is better to not to add HEAL fops to ec->pending_fops
because we don't want graph switch to hang the mount because of a BIG
file/directory heal.

- Forced unlock when lock contention is detected
EC uses an eager lock mechanism to optimize multiple read/write
requests on the same entry or inode. This increases performance
but can have adverse results when other clients try to access the
same entry/inode. To solve this, this patch adds a functionality
to detect when this happens and force an earlier release to not
block other clients.

The method consists on requesting GF_GLUSTERFS_INODELK_COUNT and
GF_GLUSTERFS_ENTRYLK_COUNT for all fops that take a lock. When this
count is greater than one, the lock is marked to be released. All
fops already waiting for this lock will be executed normally before
releasing the lock, but new requests that also require it will be
blocked and restarted after the lock has been released and reacquired
again.

Another problem was that some operations did correctly lock the
parent of an entry when needed, but got the size and version xattrs
from the entry instead of the parent.

This patch solves this problem by binding all queries of size and
version to each lock and replacing all entrylk calls by inodelk ones
to remove concurrent updates on directory metadata.  This also allows
rename to correctly update source and destination directories.

BUG: 1225279
Change-Id: I02a6084b138dd38e018a462347cd9ce38610c7ef
Reviewed-on: http://review.gluster.org/10926
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Change meaning of trusted.ec.dirty

2015-05-09T03:12:31+00:00

- With this change, the xattr will represent if the file needs to be healed or
  not. It will have different values for data/entry and metadata changes.
- inode ref leaks and dict_set_dynstr related leaks fixed
- Added support for trylock/lock based on heal-cmd execution or not
  in data heal.
- Made fixes to pass regression runs

Change-Id: I9d8def4c2badde18a76b7898816fecfac113737a
BUG: 1216303
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10385
Reviewed-on: http://review.gluster.org/10693
Tested-by: NetBSD Build System
Tested-by: Gluster Build System