glusterfs.git/xlators/cluster/ec/src/ec-inode-read.c, branch v3.7.2

cluster/ec: Prevent double unwind

2015-06-18T12:47:15+00:00

        Backport of http://review.gluster.com/11111

Problem:
1) ec_access/ec_readlink_/ec_readdir[p] _cbks are trying to recover only from
ENOTCONN.
2) When the fop succeeds it unwinds right away. But when its
ec_fop_manager resumes, if the number of bricks that are up is less than
ec->fragments, the the state machine will resume with -EC_STATE_REPORT which
unwinds again. This will lead to crashes.

Fix:
- If fop fails retry on other subvols, as ESTALE/ENOENT/EBADFD etc are also
recoverable.
- unwind success/failure in _cbks

BUG: 1229331
Change-Id: I7510984a237761efba65e872313a8ede8b7543e5
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11128
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Xavier Hernandez

cluster/ec: Ignore differences in non locked inodes

2015-05-30T12:35:44+00:00

        Backport of http://review.gluster.org/10974

When ec combines iatt structures from multiple bricks, it checks
for equality in important fields. This is ok for iatt related to
inodes involved in the operation that have been locked before
starting execution. However some fops return iatt information
from other inodes. For example a rename locks source and destination
parent directories, but it also returns an iatt from the entry
itself.

In these cases we ignore differences in some fields to avoid false
detection of inconsistencies and trigger unnecessary self-heals.

Another issue is solved in this patch that caused that the real
size of the file stored into the inode context was lost during
self-heal.

BUG: 1225796
Change-Id: I29f328a7b4895368ded859f3bae0359436c3588f
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/10983
Tested-by: Gluster Build System

cluster/ec: Fix all EIO errors in EC

2015-05-28T11:12:06+00:00

        Backport of http://review.gluster.org/10770
        Backport of http://review.gluster.org/10806
        Backport of http://review.gluster.org/10787
        Backport of http://review.gluster.org/10868
        Backport of http://review.gluster.com/10852

 - When a blocking lock is requested, lock request is succeeded even when
ec->fragment number of locks are acquired successfully in non-blocking locking
phase. This will lead to fop succeeding only on the bricks where the locks are
acquired, leading to the necessity of self-heals. To prevent these un-necessary
self-heals, if the remaining locks fail with EAGAIN in non-blocking lock phase
try blocking locking phase instead.

 -  Handle lookup failures while op in progress

 - cluster/ec: Correctly cleanup delayed locks
When a delayed lock is pending, a graph switch doesn't correctly
terminate it. This means that the update of version and size xattrs
is lost, causing EIO errors. This patch handles GF_EVENT_PARENT_DOWN
event to correctly finish pending udpdates before completing the
graph switch.

 - Fix use after free crash
ec_heal creates ec_fop_data but doesn't run ec_manager. ec_fop_data_allocate
adds this fop to ec->pending_fops, because ec_manager is not run on this heal
fop it is never removed from ec->pending_fops. When it is accessed after free
it leads to crash. It is better to not to add HEAL fops to ec->pending_fops
because we don't want graph switch to hang the mount because of a BIG
file/directory heal.

- Forced unlock when lock contention is detected
EC uses an eager lock mechanism to optimize multiple read/write
requests on the same entry or inode. This increases performance
but can have adverse results when other clients try to access the
same entry/inode. To solve this, this patch adds a functionality
to detect when this happens and force an earlier release to not
block other clients.

The method consists on requesting GF_GLUSTERFS_INODELK_COUNT and
GF_GLUSTERFS_ENTRYLK_COUNT for all fops that take a lock. When this
count is greater than one, the lock is marked to be released. All
fops already waiting for this lock will be executed normally before
releasing the lock, but new requests that also require it will be
blocked and restarted after the lock has been released and reacquired
again.

Another problem was that some operations did correctly lock the
parent of an entry when needed, but got the size and version xattrs
from the entry instead of the parent.

This patch solves this problem by binding all queries of size and
version to each lock and replacing all entrylk calls by inodelk ones
to remove concurrent updates on directory metadata.  This also allows
rename to correctly update source and destination directories.

BUG: 1225279
Change-Id: I02a6084b138dd38e018a462347cd9ce38610c7ef
Reviewed-on: http://review.gluster.org/10926
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Change meaning of trusted.ec.dirty

2015-05-09T03:12:31+00:00

- With this change, the xattr will represent if the file needs to be healed or
  not. It will have different values for data/entry and metadata changes.
- inode ref leaks and dict_set_dynstr related leaks fixed
- Added support for trylock/lock based on heal-cmd execution or not
  in data heal.
- Made fixes to pass regression runs

Change-Id: I9d8def4c2badde18a76b7898816fecfac113737a
BUG: 1216303
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10385
Reviewed-on: http://review.gluster.org/10693
Tested-by: NetBSD Build System
Tested-by: Gluster Build System

cluster/ec: Fix dictionary compare function

2015-05-08T22:03:16+00:00

If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not.  value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.

geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.

Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1216303
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10078
Reviewed-on: http://review.gluster.org/10690
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

cluster/ec: Use fd instead of loc for get_size_version

2015-05-08T17:26:33+00:00

Change-Id: Ia7d43cb3b222db34ecb0e35424f1766715ed8e6a
BUG: 1219358
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/10176
Reviewed-on: http://review.gluster.org/10625
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

ec: Add trusted.ec.dirty xattr

2015-02-24T07:58:11+00:00

This xattr will be incremented before each data modifying operation and
decremented after it. This will add the possibility to detect partially
updated writes and refuse them on reads.

It will also be useful for interacting with index xlator and have a way
to heal dispersed files from the self-heal daemon.

Change-Id: Ie644a8dd074ae0f254c809c5863bdb030be5486a
BUG: 1190581
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9607
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Vijay Bellur

ec: Fix posix compliance failures

2015-01-29T03:49:29+00:00

This patch solves some problems that caused dispersed volumes to not
pass posix smoke tests:

* Problems in open/create with O_WRONLY
    Opening files with -w- permissions using O_WRONLY returned an EACCES
    error because internally O_WRONLY was replaced with O_RDWR.

* Problems with entrylk on renames.
    When source and destination were the same, ec tried to acquire
    the same entrylk twice, causing a deadlock.

* Overwrite of a variable when reordering locks.
    On a rename, if the second lock needed to be placed at the beggining
    of the list, the 'lock' variable was overwritten and later its timer
    was cancelled, cancelling the incorrect one.

* Handle O_TRUNC in open.
    When O_TRUNC was received in an open call, it was blindly propagated
    to child subvolumes. This caused a discrepancy between real file
    size and the size stored into trusted.ec.size xattr. This has been
    solved by removing O_TRUNC from open and later calling ftruncate.

Change-Id: I20c3d6e1c11be314be86879be54b728e01013798
BUG: 1161886
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9420
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri

ec: Don't use inodelk on getxattr when clearing locks

2015-01-19T06:04:40+00:00

When command 'clear-locks' from cli is executed, a getxattr request
is received by ec. This request was handled as usual, first locking
the inode. Once this request was processed by the bricks, all locks
were removed, including the lock used by ec.

When ec tried to unlock the previously acquired lock (which was
already released), caused a crash in glusterfsd.

This fix executes the getxattr request without any lock acquired
for the clear-locks command.

Change-Id: I77e550d13c4673d2468a1e13fe6e2fed20e233c6
BUG: 1179050
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9440
Reviewed-by: Dan Lambright 
Tested-by: Gluster Build System

ec: Remove O_APPEND from flags on create and open.

2015-01-09T17:46:08+00:00

Allowing O_APPEND flag to pass through to the brick files
corrupts fragment contents because writes are not stored on
the desired place.

Write fop has been modified so that it uses current file
size as its write offset. This guarantees that all writes,
even those comming from different file descriptors and
clients, will write to the end of the file.

Change-Id: I9f721f12217a98231fe52e344166d1c94172c272
BUG: 1161621
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/9079
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Reviewed-by: Vijay Bellur