glusterfs.git/libglusterfs, branch release-3.4

libglusterfs/syncop: implement inodelk

2014-10-20T14:55:22+00:00

Change-Id: Iea489157490b70cb2bb03576b0d4943c6d8f052d
BUG: 1139998
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/8522
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8680
Reviewed-by: N Balachandran 
Reviewed-by: Kaleb KEITHLEY

cluster/dht: Fix races to avoid deletion of linkto

2014-10-20T14:53:56+00:00

 file

Explanation of Race between rebalance processes:
https://bugzilla.redhat.com/show_bug.cgi?id=1110694#c4

scenario-1:
===========

STATE 1:                          BRICK-1
only one brick                   Cached File
in the system

STATE 2:
Add brick-2                       BRICK-1                BRICK-2

STATE 3:                                       Lookup of File on brick-2
                                               by this node's rebalance
                                               will fail because hashed
                                               file is not created yet.
                                               So dht_lookup_everywhere is
                                               about to get called.

STATE 4:                         As part of lookup
                                 link file at brick-2
                                 will be created.

STATE 5:                         getxattr to check that
                                 cached file belongs to
                                 this node is done

STATE 6:

                                            dht_lookup_everywhere_cbk detects
                                            the link created by rebalance-1.
                                            It will unlink it.

STATE 7:                        getxattr at the link
                                file with "pathinfo" key
                                will be called will fail
                                as the link file is deleted
                                by rebalance on node-2

Fix:
So in the STATE 6, we should avoid the deletion of link file. Every time
dht_lookup_everywhere gets called, lookup will be performed on all the nodes.
So to avoid STATE 6, if linkto file is found, it is not deleted until valid
case is found in dht_lookup_everywhere_done.

Case 1: if linkto file points to cached node, and cached file exists,
        uwind with success.

Case 2: if linkto does not point to current cached node, and cached file
        exists:
        a) Unlink stale link file
        b) Create new link file

Case 3: Only linkto file exists:
        Delete linkto file

Case 4: Only cached file
        Create link file (Handled event without patch)

Case 5: Neither cached nor hashed file is present
        Return with ENOENT (handled even without patch)

Reviewed-on: http://review.gluster.org/8231

******************************************************************************

scenario-2:
===========
cluster/dht: Modified logic of linkto file deletion on non-hashed

Currently whenever dht_lookup_everywhere gets called, if in
dht_lookup_everywhere_cbk, a linkto file is found on non-hashed
subvolume, file is unlinked. But there are cases when this file
is under migration. Under such condition, we should avoid deletion
of file.

When  some other rebalance process changes the layout of parent
such that dst_file (w.r.t. migration) falls on non-hashed node,
then may be lookup could have found it as linkto file but just
before unlink, file  is under migration or already migrated
In such cased unlink can be avoided.

Race:
-------
If we have two bricks (brick-1 and brick-2) with initial file "a"
under BaseDir which is hashed as well as cached on (brick-1).

Assume "a"  hashing gives 44.

                              Brick-1              Brick-2

Initial Setup:               BaseDir/a             BaseDir
                             [1-50]                [51-100]

Now add new-brick Brick-3.

1. Rebalance-1 on node Node-1 (Brick-1 node) will reset
the BaseDir Layout.

2. After that it will perform
a)  Create linkto file on  new-hashed (brick-2)
b)  Perform file migration.

1.Rebalance-1 Fixes the base-layout:
                 Brick-1             Brick-2           Brick-3
                 ---------         ----------         ------------
                 BaseDir/a            BaseDir           BaseDir
                  [1-33]              [34-66]           [67-100]

2. Only a) is     BaseDir/a          BaseDir/a(linkto)   BaseDir
   performed                         Create linktofile

Now rebalance 2 on node-2 jumped in and it will perform
step 1 and 2-a.

After (rebal-2, step-1), it changes the layout of the BaseDir.
                    BaseDir/a     BaseDir/a(link)    BaseDir
                    [67-100]           [1-33]        [34-66]

For  (rebale-2, step-2), It will perform lookup at Brick-3 as w.r.t new
layout 44 falls for brick-3. But lookup will fail.
So  dht_lookup_everywhere gets called.

NOTE: On brick-2 by rebalance-1, a linkto file was created.

Currently that linkto files gets deleted by rebalance-2 lookup as it
is considered as stale linkto file.  But  with patch if rebalance is
already in progress or rebalance is over,  linkto file will not be
unlinked. If rebalance is in progress fd will be  open and if rebalance
is over then linkto file wont be set.

Reviewed-on: http://review.gluster.org/8345

*******************************************************************************

scenario-3:
===========

cluster/dht: Added keys in dht_lookup_everywhere_done

Case where both cached  (C1)  and hashed file are found,
but hash does not point to above cached node (C1), then
dont unlink if either fd-is-open on hashed or
linkto-xattr is not found.

Reviewed-on: http://review.gluster.org/8429
BUG: 1139995
Signed-off-by: Venkatesh Somyajulu 
Change-Id: I86d0a21d4c0501c45d837101ced4f96d6fedc5b9
Signed-off-by: Venkatesh Somyajulu 
Tested-by: Gluster Build System 
Reviewed-by: susant palai 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8674
Reviewed-by: Kaleb KEITHLEY

cluster/afr: Fix memory leak of file-path in self-heal-daemon

2014-08-27T13:46:38+00:00

        Backport of http://review.gluster.org/4790

Note: Only the part which fixes the memory leak is backported

shd event has path which needs to be freed as part of circular buffer cleanup.
This patch introduces the functionality so that self-heal-daemon can use it.

Change-Id: I3f3823d5587eda2fcb278f0fdb89123a31c9d786
BUG: 1119894
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8541
Tested-by: Gluster Build System 
Reviewed-by: Ravishankar N 
Reviewed-by: Kaleb KEITHLEY

gfapi: backport handles to 3.4 for nfs-ganesha

2013-12-04T11:34:13+00:00

nfs-ganesha-2.0 will ship shortly. It depends on the gfapi handle support,
which won't otherwise be released until 3.5 sometime in 2014.

Change-Id: I104d6fb275bb2c710790340fdc7d998446403026
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/6394
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

rpcsvc: allocate large auxgid list on demand

2013-09-19T21:06:33+00:00

For rpc requests having large aux group list, allocate large list
on demand. Else use small static array by default.

Without this patch, glusterfsd allocates 140+MB of resident memory
just to get started and initialized.

Change-Id: I3a07212b0076079cff67cdde18926e8f3b196258
Signed-off-by: Anand Avati 
BUG: 953694
Reviewed-on: http://review.gluster.org/5927
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5928

parser: make the parser thread safe.

2013-09-10T08:14:02+00:00

The volfile parser thread safe by guarding the parsing phase
in a mutex. Thread safety becomes a problem when there are multiple
glfs_t objects created by gfapi and all of them potentially parse
the respective volfiles at the same time.

Change-Id: I4376019c4956994b72397ab36e6ac3ce849797ec
BUG: 1004519
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5872
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

core: increase the auxillary group limit to 65536

2013-09-10T00:25:38+00:00

Make the allocation of groups dynamic and increase the limit
to 65536.

Change-Id: I702364ff460e3a982e44ccbcb3e337cac9c2df51
BUG: 953694
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5172
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System

mount/fuse: perform lookup() on inodes linked through readdirplus

2013-09-10T00:19:09+00:00

Some xlators still require lookup() fop to be sent for proper working.
This patch remembers inodes which have been linked through readdiprlus
and makes the resolver send lookups on them.

Also, introduce and use context count for inode table.

Change-Id: Ibe8a04a659539d90dfc794521b51bf2bda017a0b
BUG: 979910
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5267
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-on: http://review.gluster.org/5806

call-stub: remove fall-through in ftruncate

2013-09-06T02:25:43+00:00

Original-author: Pranith Kumar K 

Change-Id: I7c8ed3e3e4ddc558ff061f2e1407fb43095e67f4
BUG: 913544
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/5827
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System

call-stub: internal refactor

2013-09-05T13:27:05+00:00

- re-structure members of call_stub_t with new simpler layout
- easier to inspect call_stub_t contents in gdb now
- fix a bunch of double unrefs and double frees in cbk stub
- change all STACK_UNWIND to STACK_UNWIND_STRICT and thereby fixed
  a lot of bad params
- implement new API call_unwind_error() which can even be called on
  fop_XXX_stub(), and not necessarily fop_XXX_cbk_stub()

Change-Id: Idf979f14d46256af0afb9658915cc79de157b2d7
BUG: 846240
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/4520
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/5820
Reviewed-by: Raghavendra Bhat