glusterfs.git/libglusterfs/src, branch v3.5.0qa3

gNFS: Inconsistent behaviour of setfacl/getfacl

2013-12-05T18:36:11+00:00

The permissions returned by NFS ACL are wrong, which are rejected
by NFS client as "Invalid argument". Refactor the NFS ACL code
to return the proper permissions which would match with the
requested permissions.

Upstream master review: http://review.gluster.org/6368

Change-Id: Ieb079b5da98b061291b44655e18a1dee92a8e463
BUG: 1035218
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/6418
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cli, glusterd: More quota fixes ...

2013-12-03T09:48:31+00:00

... which may be grouped under the following categories:

1. Fix incorrect cli exit status for 'quota list' cmd
2. Print appropriate error message on quota parse errors in cli

        Authored by: Anuradha Talur 

3. glusterd: Improve quota validation during stage-op
4. Fix peer probe issues resulting from quota conf checksum mismatches
5. Enhancements to CLI output in the event of quota command failures

        Authored by: Kaushal Madappa 

7. Move aux mount location from /tmp to /var/run/gluster

        Authored by: Krishnan Parthasarathi 

8. Fix performance issues in quota limit-usage

        Authored by: Krutika Dhananjay 

Note: Some functions that were used in earlier version of quota,
      that aren't called anymore have been removed.

Change-Id: I963d4145f3ecdfe30c61bfa8920baccb33d2d4bd
BUG: 969461
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/6386
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

iobufs: make iobref container size dynamic

2013-11-26T18:31:17+00:00

With gfapi we can receive read/write size beyond the natural limits
of FUSE and NFS server. iobref was hardcoded to hold iobuf refs up
to 16 in count, which imposes a natural limit of 2MB with 128KB page
sizes of read-ahead and io-cache. Fix this by making iobref's iobuf
ref container size dynamic.

Change-Id: I93d88104d6c5e7af96cc9f1bfcc870d80fa81dad
BUG: 1034398
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6348
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

socket: limit vector count to IOV_MAX

2013-11-26T18:31:04+00:00

IOV_MAX is the maximum supported vector count on a given platform.
Limit the count to IOV_MAX if higher. As we are performing non-blocking
IO getting a smaller return value is handled naturally.

Change-Id: I94ef67a03ed0e10da67a776af2b55506bf721611
BUG: 1034398
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6354
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

cluster/dht: instruct marker whenever it shouldn't do accounting

2013-11-26T18:27:21+00:00

This is needed for two reasons:

* since dht-linkfiles are internal, they shouldn't be accounted.
* hardlink handling in marker is broken. link/unlink of hardlinks
  present in same directory can break marker accounting. Hence, if src
  and dst are in same directory in case of rename, dht - if it breaks
  rename into link/unlink operations - should instruct marker to not to
  do accounting.

Change-Id: I9c9f7384569f75a2792f6450ee7a5279bf751ae7
BUG: 1022995
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6203
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core: add dht_is_linkfile helper procedure.

2013-11-26T18:26:44+00:00

components other than distribute (like marker to exclude linkfiles
from being accounted) also need awareness of what constitutes a
linkfile. Hence its good to separate out this functionality into
core.

Change-Id: Ib944eeacc991bb1de464c9e73ee409fc7a689ff1
BUG: 1022995
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6152
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cli/glusterd: Changes to quota command Quota feature

2013-11-26T18:25:27+00:00

 re-work.

Following are the cli commands that are new/re-worked:
======================================================

volume quota  {enable|disable|list [ ...]|remove | default-soft-limit } |
volume quota  {limit-usage   []} |
volume quota  {alert-time|soft-timeout|hard-timeout} {}
volume status [all |  [nfs|shd||quotad]] [detail|clients|mem|inode|fd|callpool]
volume statedump  [nfs|quotad] [all|mem|iobuf|callpool|priv|fd|inode|history]

glusterd changes:
=================
* Quota limits are now set as extended attributes by glusterd from
  the aux mount created by the cli.
* The gfids of the directories on which quota limits are set
  for a given volume are stored in
  /var/lib/glusterd/vols//quota.conf file in binary format,
  and whose cksum and version is stored in
  /var/lib/glusterd/vols//quota.cksum.

Original-author: Krutika Dhananjay 
Original-author: Krishnan Parthasarathi 

BUG: 969461
Change-Id: If32bba36c67f9c2a30417af9c6389045b2b7c13b
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6003
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

features/quota: Improvements to quota

2013-11-26T18:24:02+00:00

* Two stages of quota enforcement is done:

  Soft and hard quota Upon reaching soft quota limit on the directory
  it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log)
  and no more writes allowed after hard
  quota limit. After reaching the soft-limit the daemon alerts the
  user/admin repeatively for every 'alert-time', which is
  configurable.

* Quota enforcer is moved to server-side.

  It  takes care of enforcing quota. Since enforcer doesn't have the
  cluster view, it relies on another service called
  quota-aggregator. Aggregator, on query can return the size of a
  directory based on the cluster view.

  Enforcer is always loaded in the server graph and is by passed if
  the feature is not enabled.

  Options specific to enforcer:

  server-quota - Specifies whether the feature is on/off. It is used
  to by pass the quota if turned off.

  deem-statfs - If set to on, it takes quota limits into consideration
  while estimating fs size. (df command). The algorithm followed is,
  i.   Adjust statvfs based on limit configured on root.
  ii.  If limit is set on the inode passed, use size/limits on that inode to
       populate statvfs. Otherwise, use size/limits configured on root.
  iii. Upon statvfs, update the ctx->size on the inode.
  iv.  Don't let DHT aggregate, instead take the maximum of the usages from the
       subvols of the DHT, since each of it contains the complete information.

  Enforcer also makes use of gfid-to-path conversion functionality to
  work correctly when a client like nfs predominently relies on
  nameless lookups.

* Quota Aggregator acts as a thin client to provide cluster view

  Its a lightweight *gluster client* process with no mount point,
  started upon enabling quota or restarting the volume. This is a
  single process run on each brick, which can answer queries on all
  volumes in the cluster. Its volfile stored in
  GLUSTERD_DEFAULT_WORKING_DIR/quotad/quotad.vol.

Credits:
Raghavendra Bhat        
Varun Shastry           
Shishir Gowda           
Kruthika Dhananjay      
Brian Foster            
Krishnan Parthasarathi  

Change-Id: Id1cb25b414951da34c665a55f77385d482e0f9de
BUG: 969461
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5952
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

posix: placeholders for GFID to path conversion

2013-11-26T18:22:40+00:00

what?
=====
    The following is an attempt to generate the paths of a file when
    only its gfid is known.

    To find the path of a directory, the symlink handle to the
    directory maintained in  the ".glusterfs" backend directory is
    read. The symlink handle is generated using the gfid of the
    directory. It (handle) contains the directory's name and parent
    gfid, which are used to recursively construct the absolute path as
    seen by the user from the mount point.

    A similar approach cannot be used for a regular file or a symbolic
    link since its hardlink handle, generated using its gfid, doesn't
    contain its parent gfid and basename. So xattrs are set to store
    the parent gfids and the number of hardlinks to a file or a
    symlink having the same parent gfid.  When an user/application
    requests for the paths of a regular file or a symlink with
    multiple hardlinks, using the parent gfids stored in the xattrs,
    the paths of the parent directories are generated as mentioned
    earlier. The base names of the hardlinks (with the same parent
    gfid) are determined by matching the actual backend inode numbers
    of each entry in the parent directory with that of the hardlink
    handle.

    Xattr is set on a regular file, link, and symbolic link as
    follows, Xattr name : trusted.pgfid. Xattr value :
    

    If a regular file, hard link, symbolic link is created then an
    xattr in the above format is set in the backend.

how to use?
===========
    This functionality can be used through getxattr interface. Two
    keys - glusterfs.ancestry.dentry and glusterfs.ancestry.path - enable
    usage of this functionality. A successful getxattr will have the
    result stored under same keys. Values will be,

    glusterfs.ancestry.dentry:
    --------------------------
    A linked list of gf-dirent structures for all possible paths from
    root to this gfid. If there are multiple paths, the linked-list
    will be a series of paths one after another. Each path will be a
    series of dentries representing all components of the path. This
    key is primarily for internal usage within glusterfs.

    glusterfs.ancestry.path:
    ------------------------
    A string containing all possible paths from root to this gfid.
    Multiple hardlinks of a file or a symlink are displayed as a colon
    seperated list (this could interfere with path components
    containing ':').

    e.g. If there is a file "file1" in root directory with two hardlinks,
         "/dir2/link2tofile1" and "/dir1/link1tofile1", then

         [root@alpha gfsmntpt]# getfattr -n glusterfs.ancestry.path -e text
          file1
          glusterfs.ancestry.path="/file1:/dir2/link2tofile1:/dir1/link1tofile1"

    Thanks Amar, Avati and Venky for the inputs.

Original Author: Ramana Raja 
BUG: 990028
Signed-off-by: Raghavendra G 
Change-Id: I0eaa9101e333e0c1f66ccefd9e95944dd4a27497
Reviewed-on: http://review.gluster.org/5951
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

fuse: revalidate group id cache on uid/gid change detection

2013-11-21T21:11:18+00:00

- Remember the uid and gid of the pid at the time of caching
  the group id list.

- Next time when referring to the cache confirm that uid and gid
  of that pid has not changed since. If it has, treat it like
  a timeout/cache miss.

- Solves group id caching issue caused when Samba runs on gluster
  FUSE mount and changes the uid/gid on a per syscall basis.

Change-Id: I3382b037ff0b6d5eaaa36d9c898232543475aeda
BUG: 1032438
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6320
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi