glusterfs.git/tests/basic, branch v3.5.2beta1

features/gfid-access: Fix entry operations

2014-07-08T08:13:15+00:00

        Backport of http://review.gluster.org/8204

Problem:
When more than one aux-mounts are performing rmdir .gfid//dir
simultaneously, then sometimes a hang is observed.  In gfid-access xlator When
virtual parent/inode are replaced with real parent/inode in loc, virtual
pargfid/gfid are not replaced with real pargfid/gfid respectively. Afr is using
parent_loc->gfids to order the entry locks. But parent_loc->gfid contains
random/virtual gfid generated by gfid-access xlator. Entrylk in client xlator
is using loc->inod->gfid for sending entrylk which has 'real' gfid. Because the
ordering is happening based on random gfids, One mount orders the locks as (L1,
L2) where as the other orders them as (L2, L1) leading to a dead-lock thus
a hang.

Fix:
Replace virtual pargfid/gfid with real pargfid/gfid when virtual-inodes are
replaced with real-inodes in loc.

BUG: 1114501
Change-Id: I13016de1da11762e0697792d76e6e946d991c0a4
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8251
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

features/gfid-access: Fix inode leaks and loc path corruption

2014-07-08T08:11:50+00:00

        Backport of http://review.gluster.org/8009
        Backport of http://review.gluster.org/8163

BUG: 1112659
Change-Id: Ic70a3ddfcfef88909c12ca6791a8e20c3fee2eae
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8250
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

cluster/afr: Fix resolution issues with afr

2014-06-24T16:26:13+00:00

Problem with afr:
Lets say there is a directory hierarchy a/b/c/d on the mount and the
user is cd'ed into the directory. Bring down one of the bricks of replica and
remove all directories/files to simulate disk replacement on that brick. Now
this brick is brought back up. Creates on the cd'ed directory fail with ESTALE.
Basically before sending a create of 'f' inside 'd', fuse sends a lookup to
make sure the file is not present.  On one of the bricks  'd' is present and
'f' is not so it sends ENOENT as response. On the new brick 'd' itself is not
present. So it sends ESTALE. In afr ESTALE is considered to be special errno on
witnessing which lookup has to fail. And ESTALE is given more priority than
ENOENT. Due to these reasons lookup fails with ESTALE rather than ENOENT. Since
lookup didn't fail with ENOENT, 'create' can't be issued so the command is
failed with ESTALE.

Solution:
Afr needs to consider ESTALE errno normally and ENOENT needs to
be given more priority so that operations like create can proceed even when
only one of the brick is up and running. Whenever client xlator identifies
that gfid-changed, it sets that information in lookup xdata. Afr uses this
information to fail the lookup with ESTALE so that top xlator can send
fresh lookup.

Change-Id: Ie8e0e327542fd644409eb5dadf451679afa1c0e5
BUG: 1112348
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8154
Tested-by: Justin Clift 
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Niels de Vos

tests: Remove spurious failure test

2014-06-23T09:54:40+00:00

Backport of http://review.gluster.org/#/c/8031/

BUG: 1107937
Change-Id: I21646b69b99963117cdef969d44b96f5cdfe2aa2
Reviewed-on: http://review.gluster.org/8095
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

tests/rpm: always run ./autogen.sh to create missing files

2014-05-23T18:20:56+00:00

In some occasions 'install-sh' seems to be missing in the 'make dist'
tarball when ./autogen.sh has not been run (skipped when 'configure'
exists). With this changes, 'autogen.sh' is always run so that missing
files should get added to the tarball.

Also write the logs from mock to a known location, and copy them to the
'/var/log/' directory that Jenkins archives after a regression test
failure. This makes it easier to find build issues that mock detects.

Cherry picked from commit 79de5b63775f1ab8e2e498fd51b55509a30cd896:
> Change-Id: I8d0cf1afef61ebab0137aa0d20521e75a35ddbdd
> BUG: 1038391
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/7786
> Tested-by: Gluster Build System 
> Reviewed-by: Kaleb KEITHLEY 
> Reviewed-by: Anand Avati 

Change-Id: I8d0cf1afef61ebab0137aa0d20521e75a35ddbdd
BUG: 1089054
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/7850
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System

cluster/afr: trigger self-heals even when they are set to off.

2014-04-28T16:43:24+00:00

When attempt-self-heal is set to true, trigger data/metadata/entry
self-heals even when they are disabled. This is useful for
gluster volume heal info to report them even when metadata-self-heal
entry-self-heal, data-self-heal are set to off.

Change-Id: Idc3f0d5d049c875b4f975248fef56ea2238da47c
BUG: 1039544
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/7480
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

Tests: Changes to tests for glfs-heal implementation.

2014-04-28T16:40:16+00:00

volume heal  info does not depend on self-heald
anymore so removed one complete test tests/bugs/bug-880898.t
and also removed specific tests related to that in
tests/basic/self-heald.t.

Change-Id: I2e5cd3641b86cad63af41c9564d57ab6ebb60742
BUG: 1039544
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/6515
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

mgmt/glusterd: make sure quota enforcer has established connection with quotad before marking quota as enabled.

2014-01-29T06:46:35+00:00

without this patch there is a window of time when quota is marked as
enabled in quota-enforcer, but connection to quotad wouldn't have been
established. Any checklimit done during this period can result in a
failed fop because of unavailability of quotad.

Change-Id: I0d509fabc434dd55ce9ec59157123524197fcc80
Signed-off-by: Raghavendra G 
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6572
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/6820

features/compress: rename "compress" option to "network.compression"

2014-01-27T05:40:57+00:00

Prevent mistaking the "compress" options for storage (at rest)
compression. The cdc-xlator is implemented to support compressing of
network traffic (READ and WRITE FOPs).

URL: http://www.gluster.org/community/documentation/index.php/Features/On-Wire_Compression_+_Decompression
Master-Change-Id: I9fedf4106dcb226d135ab92e4b533aff284881d7
Master-Reviewed-on: http://review.gluster.org/6765
Change-Id: Ib882af855b36df93fac46236c349c33dd4c3ced4
BUG: 1053670
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/6773
Tested-by: Gluster Build System 
Reviewed-by: Prashanth Pai

features/quota: Improvements to quota

2013-11-26T18:24:02+00:00

* Two stages of quota enforcement is done:

  Soft and hard quota Upon reaching soft quota limit on the directory
  it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log)
  and no more writes allowed after hard
  quota limit. After reaching the soft-limit the daemon alerts the
  user/admin repeatively for every 'alert-time', which is
  configurable.

* Quota enforcer is moved to server-side.

  It  takes care of enforcing quota. Since enforcer doesn't have the
  cluster view, it relies on another service called
  quota-aggregator. Aggregator, on query can return the size of a
  directory based on the cluster view.

  Enforcer is always loaded in the server graph and is by passed if
  the feature is not enabled.

  Options specific to enforcer:

  server-quota - Specifies whether the feature is on/off. It is used
  to by pass the quota if turned off.

  deem-statfs - If set to on, it takes quota limits into consideration
  while estimating fs size. (df command). The algorithm followed is,
  i.   Adjust statvfs based on limit configured on root.
  ii.  If limit is set on the inode passed, use size/limits on that inode to
       populate statvfs. Otherwise, use size/limits configured on root.
  iii. Upon statvfs, update the ctx->size on the inode.
  iv.  Don't let DHT aggregate, instead take the maximum of the usages from the
       subvols of the DHT, since each of it contains the complete information.

  Enforcer also makes use of gfid-to-path conversion functionality to
  work correctly when a client like nfs predominently relies on
  nameless lookups.

* Quota Aggregator acts as a thin client to provide cluster view

  Its a lightweight *gluster client* process with no mount point,
  started upon enabling quota or restarting the volume. This is a
  single process run on each brick, which can answer queries on all
  volumes in the cluster. Its volfile stored in
  GLUSTERD_DEFAULT_WORKING_DIR/quotad/quotad.vol.

Credits:
Raghavendra Bhat        
Varun Shastry           
Shishir Gowda           
Kruthika Dhananjay      
Brian Foster            
Krishnan Parthasarathi  

Change-Id: Id1cb25b414951da34c665a55f77385d482e0f9de
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5952
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati