glusterfs.git/libglusterfs, branch v3.3.0qa21

features/marker,quota: Incorporate changes to support nameless lookup

2012-02-03T03:33:11+00:00

Change-Id: Ic5f00a9891bd835ebee5a3e103ef0f75d0b7fc25
BUG: 783925
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.com/2702
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Jeff Darcy

cli: Extend "volume status" with statedump info

2012-01-27T12:20:04+00:00

This patch enhances and extends the "volume status" command with information
obtained from the statedump of the bricks of volumes.

Adds new status types : clients, inode, fd, mem, callpool
The new syntax of "volume status" is,
 #gluster volume status [all|{ []
                         [misc-details|clients|inode|fd|mem|callpool]}]

Change-Id: I8d019718465bbc3de727653a839de7238f45da5c
BUG: 765495
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.com/2637
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi

common-utils: Add support for solaris, bsd and mac os/x

2012-01-26T10:32:08+00:00

get_mem_size() is more standardized now.

Change-Id: I8e3dc29df0a64a5eb8eea4fd3965b268cb1a85c2
BUG: 772808
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.com/2618
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

Add format/printf/attribute to log decls

2012-01-26T10:10:33+00:00

This enables compile-time checking of printf-style format checking

Reason for doing it this way: N/A

Description of test cases: N/A

Change-Id: I9e26a5dceef5b545b9434b1d418c3d1193b4ef9a
Signed-off-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.com/2693
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

complete the implementation of missing 'f**xattr()' fops

2012-01-25T10:26:51+00:00

in debug/* and cluster/* translators and a syncop_fsetxattr()

added a test case for testing the working of 'f-fop()' on
fuse mount.

Change-Id: I0c2aeeb30a0fb382ef2495cca1e66b00abaffd35
Signed-off-by: Amar Tumballi 
BUG: 766571
Reviewed-on: http://review.gluster.com/802
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core: add 'fremovexattr()' fop

2012-01-25T10:24:20+00:00

so operations can be done on fd for extended attribute removal

Change-Id: Ie026f1b53793aeb4ae33e96ea5408c7a97f34bf6
Signed-off-by: Amar Tumballi 
BUG: 766571
Reviewed-on: http://review.gluster.com/778
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core: get xattrs also as part of readdirp

2012-01-25T10:03:44+00:00

readdirp_req() call sends a dict_t * as an argument, which
contains all the xattr keys for which the entries got in
readdirp_rsp() are having xattr value filled dictionary.

Change-Id: I8b7e1290740ea3e884e67d19156ce849227167c0
Signed-off-by: Amar Tumballi 
BUG: 765785
Reviewed-on: http://review.gluster.com/771
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core: change lk-owner as a 1k buffer

2012-01-25T04:14:17+00:00

so, NLM can send the lk-owner field directly to the locks translators,
while doing the same effort, also enabled sending maximum of 500 aux gid
over protocol.

Change-Id: I87c2514392748416f7ffe21d5154faad2e413969
Signed-off-by: Amar Tumballi 
BUG: 767229
Reviewed-on: http://review.gluster.com/779
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

pump: move internal pump xattrs out of trusted domain

2012-01-23T17:46:43+00:00

* the trusted.glusterfs.pump.{start|pause|commit|status|abort} xattrs
  have been moved out of trusted domain. This enables separation of
  xattrs used as gluster-internal commands (handled by pump) for
  replace-brick, which are not set in the back-end, from xattrs set
  on the replace-brick source and destinations bricks.

* macros definitions from pump.h and glusterd.h, #defining these xattrs
  have been merged and put into libglusterfs/src/glusterfs.h

Change-Id: I87b8bfbf045aa140f5d3f0c9baa9b2e79f87b67b
BUG: 783049
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/2663
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

core: GFID filehandle based backend and anonymous FDs

2012-01-20T13:03:42+00:00

1. What
--------
This change introduces an infrastructure change in the filesystem
which lets filesystem operation address objects (inodes) just by its
GFID. Thus far GFID has been a unique identifier of a user-visible
inode. But in terms of addressability the only mechanism thus far has
been the backend filesystem path, which could be derived from the
GFID only if it was cached in the inode table along with the entire set
of dentry ancestry leading up to the root.

This change essentially decouples addressability from the namespace. It
is no more necessary to be aware of the parent directory to address a
file or directory.

2. Why
-------
The biggest use case for such a feature is NFS for generating
persistent filehandles. So far the technique for generating filehandles
in NFS has been to encode path components so that the appropriate
inode_t can be repopulated into the inode table by means of a recursive
lookup of each component top-down.

Another use case is the ability to perform more intelligent self-healing
and rebalancing of inodes with hardlinks and also to detect renames.

A derived feature from GFID filehandles is anonymous FDs. An anonymous FD
is an internal USABLE "fd_t" which does not map to a user opened file
descriptor or to an internal ->open()'d fd. The ability to address a file
by the GFID eliminates the need to have a persistent ->open()'d fd for the
purpose of avoiding the namespace. This improves NFS read/write performance
significantly eliminating open/close calls and also fixes some of today's
limitations (like keeping an FD open longer than necessary resulting
in disk space leakage)

3. How
-------

At each storage/posix translator level, every file is hardlinked inside
a hidden .glusterfs directory (under the top level export) with the name
as the ascii-encoded standard UUID format string. For reasons of performance
and scalability there is a two-tier classification of those hardlinks
under directories with the initial parts of the UUID string as the directory
names.

For directories (which cannot be hardlinked), the approach is to use a symlink
which dereferences the parent GFID path along with basename of the directory.
The parent GFID dereference will in turn be a dereference of the grandparent
with the parent's basename, and so on recursively up to the root export.

4. Development
---------------

4a. To leverage the ability to address an inode by its GFID, the technique is
to perform a "nameless lookup". This means, to populate a loc_t structure as:

loc_t {
   pargfid: NULL
   parent: NULL
   name: NULL
   path: NULL
   gfid: GFID to be looked up [out parameter]
   inode: inode_new () result [in parameter]
}

and performing such lookup will return in its callback an inode_t
populated with the right contexts and a struct iatt which can be
used to perform an inode_link () on the inode (without a parent and
basename). The inode will now be hashed and linked in the inode table
and findable via inode_find().

A fundamental change moving forward is that the primary fields in a
loc_t structure are now going to be (pargfid, name) and (gfid) depending
on the kind of FOP. So far path had been the primary field for operations.
The remaining fields only serve as hints/helpers.

4b. If read/write is to be performed on an inode_t, the approach so far
has been to: fd_create(), STACK_WIND(open, fd), fd_bind (in callback) and
then perform STACK_WIND(read, fd) etc. With anonymous fds now you can do
fd_anonymous (inode), STACK_WIND (read, fd). This results in great boost
in performance in the inbuilt NFS server.

5. Misc
-------
The inode_ctx_put[2] has been renamed to inode_ctx_set[2] to be consistent
with the rest of the codebase.

Change-Id: Ie4629edf6bd32a595f4d7f01e90c0a01f16fb12f
BUG: 781318
Reviewed-on: http://review.gluster.com/669
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati