glusterfs.git/xlators/performance/write-behind, branch v3.7.3

Logging: Porting the performance translator

2015-06-29T09:21:53+00:00

         logs to new logging framework

> Change-Id: Ie6aaf8d30bd4457bb73c48e23e6b1dea27598644
> BUG: 1194640
> Signed-off-by: arao 

BUG: 1217722
Change-Id: I0856c43dbf8c0a1aa084d4478c9bdf3f41dfc0b8
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/11442
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

build: MacOSX Porting fixes

2014-04-24T21:41:48+00:00

git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs

Working functionality on MacOSX

 - GlusterD (management daemon)
 - GlusterCLI (management cli)
 - GlusterFS FUSE (using OSXFUSE)
 - GlusterNFS (without NLM - issues with rpc.statd)

Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac
BUG: 1089172
Signed-off-by: Harshavardhana 
Signed-off-by: Dennis Schafroth 
Tested-by: Harshavardhana 
Tested-by: Dennis Schafroth 
Reviewed-on: http://review.gluster.org/7503
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

write-behind: track filesize when doing extending writes

2014-02-28T05:56:48+00:00

A program that calls mmap() on a newly created sparse file, may receive
a SIGBUS signal. If SIGBUS is not handled, a segmentation fault will
occur and the program will exit.

A bug in the write-behind translator can cause the creation of a sparse
file created with open(), seek(), write() to be cached. The last write()
may not be sent to the server, until write-behind deems this necessary.

* open(.., O_TRUNC, ...)/creat() the file, it is 0 bytes big
* seek() into the file, use offset 31
* write() 1 byte to the file
* the range from byte 0-30 are unwritten so called 'sparse'

The following illustration tries to capture this:

    Legend:
    [ = start of file
    _ = unallocated/unwritten bytes
    # = allocated bytes in the file
    ] = end of file

    [_______________#]
     |              |
     '- byte 0      '- byte 31

Without this change, reading from byte 0-30 will return an error, and
reading the same area through an mmap()'d pointer will trigger a SIGBUS.
Reading from this range did not trigger the outstanding write() to be
flushed. The brick that receives the read() (translated over the network
from mmap()) does not know that the file has been extended, and returns
-EINVAL. This error gets transported back from the brick to the
glusterfs-fuse client, and translated by the Linux kernel/VFS into
SIGBUS triggered by mmap().

In order to solve this, a new attribute to the wb_inode structure is
introduced; the current size of the file. All FOPs that can modify the
size, are expected to update wb_inode->size. This makes it possible for
extending writes with an offset bigger than EOF to mark the unwritten
area as modified/pending.

Change-Id: If5ba6646732e6be26568541ea9b12852a5d0b988
BUG: 1058663
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/6835
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Anand Avati

Fix for 'use after free' errors reported by coverity.

2014-02-06T06:09:13+00:00

Change-Id: I941fc89b2d696c7f227330321ed4bba3ed1deac4
BUG: 789278
Signed-off-by: Poornima 
Reviewed-on: http://review.gluster.org/6868
Reviewed-by: Raghavendra G 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

write-behind: handle iobref_merge() error gracefully

2013-11-26T18:32:00+00:00

.. by UNWINDing ENOMEM error, rather than crashing.

Change-Id: Ica2d6399eaf7e381e7ebc41155620559c139c4d3
BUG: 1034398
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/6349
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

performance/write-behind: invoke request queue processing if

2013-08-14T09:11:58+00:00

we find fd marked bad while trying to fulfill lies.

* flush was queued behind some unfulfilled write.
* A previously wound write returned an error and hence fd was marked
  bad with corresponding error.
* wb_fulfill_head (invocation probably rooted in wb_flush), before
  winding checks for failures of previous writes and since there was a
  failure, calls wb_head_done without even winding one request in head.
* wb_head_done unrefs all the requests in list "head".
* since flush was last operation on fd (and most likely last operation
  on inode itself), no one invokes wb_process_queue and flush is stuck
  in request queue for eternity.

Change-Id: I3b5b114a1c401d477dd7ff64fb6119b43fda2d18
BUG: 988642
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/5398
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

write-behind: preserve error returned as-is

2013-07-25T01:39:13+00:00

Change-Id: Ib766403774c1323e0bbddafedeaa47e7fa3a59fa
Signed-off-by: Amar Tumballi 
BUG: 987415
Reviewed-on: http://review.gluster.org/5296
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

performance/write-behind: Enable write-behind when strict_O_DIRECT is not set.

2013-05-28T17:05:28+00:00

When open() with O_DIRECT happens, write-behind was being disabled for the
fd irrespective of strict_O_DIRECT option. This commit disables write-behind
only when strict_O_DIRECT is enabled.

Change-Id: Ieef180e52910c3bf64d46b26b0e5dc3b8542f6d2
BUG: 923556
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/4697
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

mount/fuse: enable fuse real async dio when available

2013-05-16T00:19:06+00:00

fuse has support for optimized async. direct I/O handling via the
FUSE_ASYNC_DIO init flag. Enable FUSE_ASYNC_DIO when advertised
by fuse.

performance/write-behind: fix dio hang

Also fix a hang observed during aio-stress testing due to conflicting
request handling in write-behind. Overlapping requests are skipped
in pick_winds and may never continue when the conflicting write in
progress returns. Add a wb_process_queue() call after a non-wb request
completes to keep the queue moving.

BUG: 963258
Change-Id: Ifba6e8aba7a7790b288a32067706b75f263105d4
Signed-off-by: Brian Foster 
Reviewed-on: http://review.gluster.org/5014
Reviewed-by: Anand Avati 
Tested-by: Gluster Build System

performance/write-behind: guarantee non-overlapping concurrent writes

2013-02-20T19:03:14+00:00

Maintain a list of writes (either written behind or SYNC) which are
currently "in progress" (i.e, STACK_WIND'ed towards server) and hold
off any new STACK_WIND of write (either written behind or SYNC) which
overlaps with any of the "in progress" writes.

This is a guarantee which AFR's eager-lock depends upon (though not
strictly a write-behind requirement)

Change-Id: Icedd0b51b440366a906dc9223d62b7fd6ef2ca03
BUG: 857673
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/4551
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G