glusterfs.git/xlators/cluster/ec/src/ec-combine.c, branch v3.7.15

cluster/ec: Handle absence of keys in some callback dict

2016-07-27T07:00:04+00:00

Problem: This issue arises when we do a rolling update
from 3.7.5 to 3.7.9.
For 4+2 volume running 3.7.5, if we update 2 nodes
and after heal completion  kill 2 older nodes, this
problem can be seen. After update and killing of
bricks, 2 nodes will return inodelk count key in dict
while other 2 nodes will not have inodelk count in dict.
This is also true for get-link-count.
During dictionary match , ec_dict_compare, this will
lead to mismatch of answers and the file operation
on mount point will fail with IO error.

Solution:
Don't match inode, entry and link count keys while
comparing two dictionaries. However, while combining the
data in ec_dict_combine, go through all the dictionaries
and select the maximum values received in different dicts
for these keys.

master-
http://review.gluster.org/#/c/14761/

Change-Id: I33546e3619fe8f909286ee48fb0df2009cd3d22f
BUG: 1360152
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/14761
Reviewed-by: Xavier Hernandez 
Smoke: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Signed-off-by: Ashish Pandey 
Reviewed-on: http://review.gluster.org/15012

cluster/ec: Minimize usage of EIO error

2015-08-08T15:36:57+00:00

>Change-Id: I82e245615419c2006a2d1b5e94ff0908d2f5e891
>BUG: 1245276
>Signed-off-by: Xavier Hernandez 
>Reviewed-on: http://review.gluster.org/11741
>Tested-by: Gluster Build System 
>Reviewed-by: Pranith Kumar Karampuri 
>Tested-by: NetBSD Build System 

Change-Id: Ifd3d63f88a686a2963c5ba2e62110249f84f338d
BUG: 1250864
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11852
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System

cluster/ec: wind fops on good subvols for access/readdir[p]

2015-07-02T06:42:17+00:00

        Backport of http://review.gluster.com/11246

BUG: 1234679
Change-Id: I2d774f62740c82e922efab50fc78fa74050ede93
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/11357
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System

ec: Porting messages to new logging framework

2015-06-27T09:19:38+00:00

This is a backport of http://review.gluster.org/#/c/10465/

cherry-picked from commit b0b9eaea9dbb4e9a535f5e969defc4556a9e2204
>Change-Id: Ia05ae750a245a37d48978e5f37b52f4fb0507a8c
>BUG: 1194640
>Signed-off-by: Nandaja Varma 

Change-Id: Ia05ae750a245a37d48978e5f37b52f4fb0507a8c
BUG: 1217722
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/11429
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/dht: Fix dht_setxattr to follow files under migration

2015-06-04T12:03:59+00:00

If a file is under migration, then any xattrs created on it
are lost post migration of the file. This is because
the xattrs are set only on the cached subvol of the source
and as the source is under migration, it becomes a linkto file
post migration.

Change-Id: Ib8e233b519cf954e7723c6e26b38fa8f9b8c85c0
BUG: 1225839
Signed-off-by: Nithya Balachandran 
Reviewed-on: http://review.gluster.org/10968
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

cluster/ec: Fix incorrect check for iatt differences

2015-06-03T10:35:15+00:00

        Backport of http://review.gluster.org/11018

A previous patch (http://review.gluster.org/10974) introduced a
bug that caused that some metadata differences could not be
detected in some circumstances. This could cause that self-heal
is not triggered and the file not repaired.

We also need to consider all differences for lookup requests, even
if there isn't any lock. Special handling of differences in lookup
is already done in lookup specific code.

Change-Id: I88e3678634d848a89f428a37209c31464e42cf8b
BUG: 1225796
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/11027
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Ignore differences in non locked inodes

2015-05-30T12:35:44+00:00

        Backport of http://review.gluster.org/10974

When ec combines iatt structures from multiple bricks, it checks
for equality in important fields. This is ok for iatt related to
inodes involved in the operation that have been locked before
starting execution. However some fops return iatt information
from other inodes. For example a rename locks source and destination
parent directories, but it also returns an iatt from the entry
itself.

In these cases we ignore differences in some fields to avoid false
detection of inconsistencies and trigger unnecessary self-heals.

Another issue is solved in this patch that caused that the real
size of the file stored into the inode context was lost during
self-heal.

BUG: 1225796
Change-Id: I29f328a7b4895368ded859f3bae0359436c3588f
Signed-off-by: Xavier Hernandez 
Reviewed-on: http://review.gluster.org/10983
Tested-by: Gluster Build System

cluster/ec: Fix all EIO errors in EC

2015-05-28T11:12:06+00:00

        Backport of http://review.gluster.org/10770
        Backport of http://review.gluster.org/10806
        Backport of http://review.gluster.org/10787
        Backport of http://review.gluster.org/10868
        Backport of http://review.gluster.com/10852

 - When a blocking lock is requested, lock request is succeeded even when
ec->fragment number of locks are acquired successfully in non-blocking locking
phase. This will lead to fop succeeding only on the bricks where the locks are
acquired, leading to the necessity of self-heals. To prevent these un-necessary
self-heals, if the remaining locks fail with EAGAIN in non-blocking lock phase
try blocking locking phase instead.

 -  Handle lookup failures while op in progress

 - cluster/ec: Correctly cleanup delayed locks
When a delayed lock is pending, a graph switch doesn't correctly
terminate it. This means that the update of version and size xattrs
is lost, causing EIO errors. This patch handles GF_EVENT_PARENT_DOWN
event to correctly finish pending udpdates before completing the
graph switch.

 - Fix use after free crash
ec_heal creates ec_fop_data but doesn't run ec_manager. ec_fop_data_allocate
adds this fop to ec->pending_fops, because ec_manager is not run on this heal
fop it is never removed from ec->pending_fops. When it is accessed after free
it leads to crash. It is better to not to add HEAL fops to ec->pending_fops
because we don't want graph switch to hang the mount because of a BIG
file/directory heal.

- Forced unlock when lock contention is detected
EC uses an eager lock mechanism to optimize multiple read/write
requests on the same entry or inode. This increases performance
but can have adverse results when other clients try to access the
same entry/inode. To solve this, this patch adds a functionality
to detect when this happens and force an earlier release to not
block other clients.

The method consists on requesting GF_GLUSTERFS_INODELK_COUNT and
GF_GLUSTERFS_ENTRYLK_COUNT for all fops that take a lock. When this
count is greater than one, the lock is marked to be released. All
fops already waiting for this lock will be executed normally before
releasing the lock, but new requests that also require it will be
blocked and restarted after the lock has been released and reacquired
again.

Another problem was that some operations did correctly lock the
parent of an entry when needed, but got the size and version xattrs
from the entry instead of the parent.

This patch solves this problem by binding all queries of size and
version to each lock and replacing all entrylk calls by inodelk ones
to remove concurrent updates on directory metadata.  This also allows
rename to correctly update source and destination directories.

BUG: 1225279
Change-Id: I02a6084b138dd38e018a462347cd9ce38610c7ef
Reviewed-on: http://review.gluster.org/10926
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

cluster/ec: Fix dictionary compare function

2015-05-08T22:03:16+00:00

If both dicts are NULL then equal. If one of the dicts is NULL but the other
has only ignorable keys then also they are equal. If both dicts are non-null
then check if for each non-ignorable key, values are same or not.  value_ignore
function is used to skip comparing values for the keys which must be present in
both the dictionaries but the value could be different.

geo-rep's stime xattr doesn't need to be present in list xattr but when
getxattr comes on stime xattr even if there aren't enough responses with the
xattr we should still give out an answer which is maximum of the stimes
available.

Change-Id: I8de2ceaa2db785b797f302f585d88e73b154167d
BUG: 1216303
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/10078
Reviewed-on: http://review.gluster.org/10690
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System

quota/disperse: handle inode quotas in xattr aggregate

2015-04-14T05:01:18+00:00

with the inode quota feature, quota size is now
increased from 64bit to 192bits which contains
values of 'file size', 'file count' and 'dir count'

This change in quota size xattr needs to be handled
in disperse xattr aggregation

Signed-off-by: vmallika 

Change-Id: I5fd28aa9f5b8b6cba83a98360236417a97ac16ee
BUG: 1207967
Reviewed-on: http://review.gluster.org/10112
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Sachin Pandit 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G