glusterfs-snapshot.git/xlators/cluster/dht, branch development

cluster/dht - rebalance: handle the rebalance @ inode level (!fd level)

2013-11-13T19:45:18+00:00

* migrate all the fd's on an inode to newer subvol after rebalance
* use the migration in progress flag in inode, so all the operations
  on the inode can make use of it

Change-Id: Ib807a46e927a1062688fc15119c916797c52a350
BUG: 1013456
Signed-off-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5891
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

bd: Add BD support to other xlators

2013-11-13T19:38:55+00:00

Make changes to distributed xlator to work with BD xlator. Unlike files,
a block device can't be removed when its opened. So some part of the
code were moved down to avoid this situation. Also before truncating a
BD file its BD_XATTR should be set otherwise truncate will result in
truncating posix file. So file is created with needed BD_XATTR and
truncate is invoked. Also enables BD xlator in stripe volume type.

Change-Id: If127516e261fac5fc5b137e7fe33e100bc92acc0
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5235
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterfs: zerofill support

2013-11-11T05:25:49+00:00

Add support for a new ZEROFILL fop. Zerofill writes zeroes to a file in
the specified range. This fop will be useful when a whole file needs to
be initialized with zero (could be useful for zero filled VM disk image
provisioning or  during scrubbing of VM disk images).

Client/application can issue this FOP for zeroing out. Gluster server
will zero out required range of bytes ie server offloaded zeroing. In
the absence of this fop,  client/application has to repetitively issue
write (zero) fop to the server, which is very inefficient method because
of the overheads involved in RPC calls  and acknowledgements.

WRITESAME is a  SCSI T10 command that takes a block of data as input and
writes the same data to other blocks and this write is handled
completely within the storage and hence is known as offload . Linux ,now
has support for SCSI WRITESAME command which is exposed to the user in
the form of BLKZEROOUT ioctl.  BD Xlator can exploit BLKZEROOUT ioctl to
implement this fop. Thus zeroing out operations can be completely
offloaded to the storage device , making it highly efficient.

The fop takes two arguments offset and size. It zeroes out 'size' number
of bytes in an opened file starting from 'offset' position.

This patch adds zerofill support to the following areas:
	- libglusterfs
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

Client applications can exloit this fop by using glfs_zerofill introduced in
libgfapi.FUSE support to this fop has not been added as there is no system call
for this fop.

Changes from previous version 3:
* Removed redundant memory failure log messages

Changes from previous version 2:
* Rebased and fixed build error

Changes from previous version 1:
* Rebased for latest master

TODO :
     * Add zerofill support to trace xlator
     * Expose zerofill capability as part of gluster volume info

Here is a performance comparison of server offloaded zeofill vs zeroing
out using repeated writes.

[root@llmvm02 remote]# time ./offloaded aakash-test log 20

real	3m34.155s
user	0m0.018s
sys	0m0.040s
[root@llmvm02 remote]# time ./manually aakash-test log 20

real	4m23.043s
user	0m2.197s
sys	0m14.457s
[root@llmvm02 remote]# time ./offloaded aakash-test log 25;

real	4m28.363s
user	0m0.021s
sys	0m0.025s
[root@llmvm02 remote]# time ./manually aakash-test log 25

real	5m34.278s
user	0m2.957s
sys	0m18.808s

The argument log is a file which we want to set for logging purpose and
the third argument is size in GB .

As we can see there is a performance improvement of around 20% with this
fop.

Change-Id: I081159f5f7edde0ddb78169fb4c21c776ec91a18
BUG: 1028673
Signed-off-by: Aakash Lal Das 
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5327
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

dht: dht_lookup_dir_cbk should set op_errno as local->op_errno

2013-10-15T07:38:07+00:00

Two glusterfs clients return inconsistent errnos when the bricks of the volume
were down. Consider two gluster mounts. Mount 1 was done when the bricks were
online. Mount 2 was done after the bricks were killed, (using the 'glusterfs'
command instead of the mount script).

For any request, mount 1 will return ENOTCONN, where as mount 2 will return
ENOENT.

This happens because for the 2nd mount, a fuse would send a lookup on '/' for
any request, as it hadn't been done yet. The client xlator returns ENOTCONN,
but the dht_lookup_dir_cbk changed this to ENOENT unconditionally when
aggregating. So, fuse returned ENOENT, even though the errno should have been
ENOTCONN.

Change-Id: I4b7a6d84ce5153045a807fccc01485afe0377117
BUG: 1019095
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/6072
Reviewed-by: Anand Avati 
Tested-by: Anand Avati

gNFS: Incorrect NFS ACL encoding for XFS

2013-09-29T23:54:37+00:00

Problem:
Incorrect NFS ACL encoding causes "system.posix_acl_default"
setxattr failure on bricks on XFS file system. XFS (potentially
others?) doesn't understand when the 0x10 prefix is added to the
ACL type field for default ACLs (which the Linux NFS client adds)
which causes setfacl()->setxattr() to fail silently. NFS client
adds NFS_ACL_DEFAULT(0x1000) for default ACL.

FIX:
Mask the prefix (added by NFS client) OFF, so the setfacl is not
rejected when it hits the FS.

Original patch by: "Richard Wareing"

Change-Id: I17ad27d84f030cdea8396eb667ee031f0d41b396
BUG: 1009210
Signed-off-by: Santosh Kumar Pradhan 
Reviewed-on: http://review.gluster.org/5980
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core: block unused signals in created threads

2013-09-25T08:33:16+00:00

Block all signal except those which are set for explicit handling
in glusterfs_signals_setup(). Since thread spawning code in
libglusterfs and xlators can get called from application threads
when used through libgfapi, it is necessary to do this blocking.

Change-Id: Ia320f80521a83d2edcda50b9ad414583a0175281
BUG: 1011662
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5995
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

Revert "cluster/dht: Return success in dht_discover if layout issues"

2013-09-25T06:48:04+00:00

This reverts commit a3e593f9f17cb1e68db97bb5a0d8074793a33964 which
was bought into fix dht_layout_anomalies error handling.

We still see applications error'ing out due to graph switches.

To fix the above issue -

We cannot heal in dht_discover, as it is a gfid based lookup, and not
path based. So, returning error here would lead to app's to see failure.

Also, update the layout in inode_ctx even if it has anomalies. Let
subsequent heals fix the issue.

Conflicts:
	xlators/cluster/dht/src/dht-common.c

Signed-off-by: shishir gowda 
Change-Id: I68c1056c3587e04a02344548546ddd06034489c5
BUG: 960348
Reviewed-on: http://review.gluster.org/5443
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Fix anomaly check

2013-09-23T18:40:33+00:00

We were wrongly detecting holes/overlaps for already accounted
errors. Additionally, sort should also handle zero'ed out layout

Change-Id: Ic3d13e1d735b914f9acc01fe919bc90656baea48
BUG: 1003851
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/5762
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Anand Avati

distribute: Rebalance should provide even disk space distribution

2013-09-19T16:25:28+00:00

Earlier disk space check had an issue which didn't
provide the needed functionality to avoid migration
when the destination had lesser available space,
scenario we need to avoid is stated below :

During rebalance `migrate-data` - Destination subvol experiences
a `reduction` in 'blocks' of free space, at the same time source
subvol gains certain 'blocks' of free space. A valid check is
necessary here to avoid errorneous move to destination where
the space could be scantily available.

This patch provides a proper fix in place by subtracting
necessary file blocks from destination and adding those blocks
to source.

Change-Id: I9c7840716a4256ef614ffc0fbfd9f2b456ac28c8
BUG: 982919
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/5961
Tested-by: Gluster Build System 
Reviewed-by: Shishir Gowda 
Reviewed-by: Anand Avati

cli/glusterd: improve rebalance fix-layout status reporting

2013-09-19T16:22:36+00:00

Problem:
Currenly the CLI rebalance status command output does not indicate the
'type' of rebalance, i.e. whether a full rebalance or only a fix-layout
was carried out.

Fix: After the rebalance status of all peers is received by the
originator glusterd, alter it to reflect the type of rebalance
before passing it on to the CLI process.

Change-Id: I1940ffda0d36e25e5b33c84a0ea210394cc9e1d3
BUG: 1004744
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/5826
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati