glusterfs.git/xlators/features, branch v3.7dev

make snapview-server more compatible with NFS server

2014-07-16T09:27:50+00:00

* There was no handle based API for listxattr. With this change, glfs_h_getxattrs
  also handles the listxattr functionality by checking whether the name is NULL
  or not (like posix). But all the gfapi functions for listxattr
  (glfs_h_getxattrs AND glfs_listxattr AND glfs_flistxattr) returns the names of
  the xattrs in a buffer provided by the caller. But snapview-server has to
  return the list of xattrs in a dict itself (similar to posix xlator). But
  the buffer just contains the names of the xattrs. So for each xattr, a zero
  byte value is set (i.e. "") into the dict and sent back. Translators which
  do xattr caching (as of now md-cache which caches selinux and acl related
  xattrs) should not cache those xattrs whose value is a zero byte data ("").
  So made changes in md-cache to ignore zero byte values.

* NFS server was not linking the inodes to inode table in readdirp. This was
  leading to applications getting errors. The below set of operations would
  lead to applications getting error
  1) ls -l in one of the snaopshots (snapview-server would generate gfids for
     each entry on the fly and link the inodes associated with those entries)
  2) NFS server upon getting readdirp reply would not link the inodes of the
     entries. But it used to generate filehandles for each entry and associate
     the gfid of that entry with the filehandle and send it as part of the
     reply to nfs client.
  3) NFS client would send the filehandle of one of those entries when some
     activity is done on it.
  4) NFS server would not be able to find the inode for the gfid present in the
     filehandle (as the inode was not linked) and would go for hard resolution
     by sending a lookup on the gfid by creating a new inode.
  5) snapview-client will not able to identify whether the inode is a real inode
     existing in the main volume or a virtual inode existing in the snapshots
     as there would not be any inode context.
  6) Since the gfid upon which lookup is sent is a virtual gfid which is not
     present in the disk, lookup would fail and the application would get an
     error.

  To handle above situation, now nfs server also does inode linking in readdirp.

Change-Id: Ibb191408347b6b5f21cff72319ccee619ea77bcd
BUG: 1115949
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/8230
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Raghavendra G 
Reviewed-by: Vijay Bellur

compress: Change variable name

2014-07-15T11:39:38+00:00

'writen' and 'total_writen' variables has been defined in compress
xlator. This patch replaced those with proper name.

Change-Id: If8e38fae8f997a92686a891d64cbaf6b0b69129a
BUG: 1075417
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/8268
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

gfapi : remove gfapi.py to avoid confusion

2014-07-14T16:47:36+00:00

python binding of libgfapi is evolving via libgfapi-python
project. The example available here confuses end users,
so removing it from this path.

Signed-off-by: Humble Chirammal 
Bug Id: 1119328
> Reviewed-on: http://review.gluster.org/7920
> Reviewed-by: Humble Devassy Chirammal 
> Tested-by: Humble Devassy Chirammal 
> Reviewed-by: Thiago da Silva 
> Reviewed-by: Prashanth Pai 
> Reviewed-by: Vijay Bellur 

Change-Id: I17b3aa0f0505342496019ce012cca21d84184027
Reviewed-on: http://review.gluster.org/8237
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

changelog: variable 'writen' is changed

2014-07-14T14:02:43+00:00

This patch change the variable name to 'written' in changelog, along
with few other typo fixing.

Change-Id: I1d3cbea5c89fc0a15233bb6e61b4fbd4bfdd3a67
BUG: 1075417
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/8267
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/gfid-access: error handling for entry creation

2014-07-14T13:26:55+00:00

Proceed with setattr() only on a successfull entry creation.
Winding a setattr() using a freshlyOC initiated inode would
most likely fail in one translator or the other (e.g. DHT
expecting the layout information to be set in the inode
context), which is the case if the inode was not looked up.

Therefore, gfid-access handles failure entry creations and
passes the _correct_ errno back to the client instead of
continuing with setattr() call and probably returning back
incorrect errno. Also, filling up inode->gfid is required
as the new inode is not looked up and ->gfid would be
certainely required for inode operations.

Change-Id: Ie92f5647a89bf558c07710ab0400bce69d59fc31
BUG: 1111490
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8260
Tested-by: Gluster Build System 
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur

features/changelog: prevent deadlock on thread cancellation

2014-07-11T07:08:57+00:00

helper threads (fsync, rollover) wake up periodically and perform
their respective operation under a lock (crt->lock). These threads
are also subjected to cancellation under some circumstance such as
disabling changelog. This is inherently dangerous when funtions
which are cancellation points for pthread_cancel(3) are used
in the locked region.

Consider this

         pthread_mutex_lock(&mutex);
         {
                /* ... */
                ret = fsync (fd);   <-- cancellation point
                /* ... */
         }
         pthread_mutex_unlock(&mutex);

A pthread_cancel(3) by another thread just before fsync(3) but
after pthread_mutex_lock(3) would result in the thread getting
cancelled when fsync(3) is invoked, thereby never unlocking the
mutex. Moreover, in case of changelog translator, the locked
region (under crt->lock in changelog-rt.c) is also the code
path for fop changelog updation. Therefore, unlocking the
mutex in thread cleanup handler (pthread_cleanup_pop(3)) might
prematurely release the mutex during fop updation path.

This patch fixes such problems existing in fsync and rollover
threads. Fix is to enter the locked region with cancellation
disabled and enable it after mutex unlock. Also, test for a
cancellation request early on in case none of the functions
are cancellation points.

Change-Id: I1795627a12827609c1da659d07fc1457ffa033de
BUG: 1110917
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8106
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System

Changes in statedump for stack, frame, locks

2014-07-07T17:27:00+00:00

Internal call-stacks don't have lk-owner so it is a bit
difficult to confirm if a stack hung by comparing two
statedump files. This change prints call-stack, frame's
address. This should solve the comparison problem.

Lock times and log times don't have same timezone because
of which one has to manually convert the times for debugging
the issues. This change prints blocked, granted times also
in UTC.

Also fixed line truncation issue when client-unique-string
is big.

Change-Id: I116372c0d63476823a36ca6dbfba91648f9234cc
BUG: 1114188
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8197
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Bhat 
Reviewed-by: Ravishankar N 
Reviewed-by: Vijay Bellur

features/gfid-access: Fix entry operations

2014-07-07T04:01:20+00:00

Problem:
When more than one aux-mounts are performing rmdir .gfid//dir
simultaneously, then sometimes a hang is observed.  In gfid-access xlator When
virtual parent/inode are replaced with real parent/inode in loc, virtual
pargfid/gfid are not replaced with real pargfid/gfid respectively. Afr is using
parent_loc->gfids to order the entry locks. But parent_loc->gfid contains
random/virtual gfid generated by gfid-access xlator. Entrylk in client xlator
is using loc->inod->gfid for sending entrylk which has 'real' gfid. Because the
ordering is happening based on random gfids, One mount orders the locks as (L1,
L2) where as the other orders them as (L2, L1) leading to a dead-lock thus
a hang.

Fix:
Replace virtual pargfid/gfid with real pargfid/gfid when virtual-inodes are
replaced with real-inodes in loc.

BUG: 1114501
Change-Id: Ie94e816122ef9e7aad51605adbf49291de60827e
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/8204
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

porting: Port for FreeBSD rebased from Mike Ma's efforts

2014-07-03T00:20:34+00:00

- Provides a working Gluster Management Daemon, CLI
- Provides a working GlusterFS server, GlusterNFS server
- Provides a working GlusterFS client
- execinfo port from FreeBSD is moved into ./contrib/libexecinfo
  for ease of portability on NetBSD. (FreeBSD 10 and OSX provide
  execinfo natively)
- More portability cleanups for Darwin, FreeBSD and NetBSD
- Provides a new rc script for FreeBSD

Change-Id: I8dff336f97479ca5a7f9b8c6b730051c0f8ac46f
BUG: 1111774
Original-Author: Mike Ma 
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8141
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

feature/changelog: Fix for missing changelogs at backend.

2014-06-30T11:25:48+00:00

Problem:
       A few changelog files are missing at the backend
       during snapshot with changelog enabled.

Cause:
       Race between actual rollover and explicit rollover.

       Changelog rollover can happen either due to actual
       or the explict rollover due to snapshot. Actual
       rollover is controlled by tuneable called rollover-time.
       The minimum granularity for rollover-time is 1 second
       Explicit rollover is asynchronous in nature and happens
       during snapshot.

       Basically, rollover renames the current CHANGELOG file
       to CHANGELOG.TIMESTAMP after rollover-time. Let's assume,
       at time 't1', actual and explicit rollover raced against
       each other and actual rollover won the race renaming the
       CHANGELOG file to CHANGELOG.t1 and opens a new
       CHANGELOG file. An immediate explicit rollover at time
       't1' happened with in the same second to rename
       CHANGELOG file to CHANGELOG.t1 resulting in purging the
       earlier CHANGELOG.t1 file created by actual rollover.

Solution:
       Adding a delay of 1 sec guarantees unique CHANGELOG.TIMESTAMP
       during explicit rollover.

Thanks Venky, for the all the help in root causing the issue.

Change-Id: I8958824e107e16f61be9f09a11d95f8645ecf34d
BUG: 1109692
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8196
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar