glusterfs.git/xlators/performance/read-ahead, branch v6dev

Land part 2 of clang-format changes

2018-09-12T12:22:45+00:00

Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu

Land clang-format changes

2018-09-12T11:52:48+00:00

Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b

performance/read-ahead: stricter adherence to force-atime-update

2018-07-19T06:56:20+00:00

Throwaway read-ahead cache in fstat only if force-atime-update is
set. Note that fstat flushes read-ahead cache only for atime
consistency. However if atime consistency is needed user is required
to set force-atime-update which updates atime on backend fs even
though application reads are served from read-ahead cache. So, if user
has not set force-atime-update, atime won't be accurate and there is
no point in flushing read-ahead cache in fstats. mounts
requiring atime consistency have to mandatorily set
force-atime-update.

Also note that normally kernel interspers reads with fstat. So,
read-ahead is not effective as fstats flush read-ahead-cache. Instead
it regresses performance due to wasted network reads. It is
recommended to turn off read-ahead if applications require atime
consistency.

This patch is aimed at applications which don't require atime
consistency. Without atime consistency required, read-ahead cache is
effective and increases performance of sequential reads.

Change-Id: I122bbc410cee96661823f9c4b934383495c18446
Signed-off-by: Raghavendra G 
Fixes: bz#1601166

performance/read-ahead: throwaway read-ahead cache of all fds on writes on any fd

2018-05-29T02:28:19+00:00

This is to make sure applications that read and write on different fds
of the same file work.

This patch also fixes two other issues:
1. while iterating over the list of open fds on an inode, initialize
tmp_file to 0 for each iteration before fd_ctx_get to make sure we
don't carry over the history from previous iterations.
2. remove flushing of cache in flush and fsync as by themselves, they
don't modify the data

Change-Id: Ib9959eb73702a3ebbf90badccaa16b2608050eff
Signed-off-by: Raghavendra G 
Updates: bz#1512691

xlators/performance: Add pass-through option

2018-04-11T10:53:48+00:00

Add pass-through option in performance traslators. Set the option in
GF_OPTION_INIT() and GF_OPTION_RECONF()

Updates: #304

Change-Id: If1537450147d154905831e36f7162a32866d7ad6
Signed-off-by: Varsha Rao

performance/read-ahead: volume option fixes for GD2

2018-01-22T15:59:42+00:00

Updates #302

Change-Id: I842ab746502bc4ff3b312ee19f397a6177edc6f7
Signed-off-by: Csaba Henk

all: Simplify component message id's definition

2017-12-14T02:33:51+00:00

This patch creates a new way of defining message id's that is easier
and less error prone because it doesn't require so many manual changes
each time a new component is defined or a new message created.

Change-Id: I71ba8af9ac068f5add7e74f316a2478bc991c67b
Signed-off-by: Xavier Hernandez

performance/read-ahead: prevent stale data being returned to application.

2017-05-09T10:06:39+00:00

Assume that fd is shared by two application threads/processes.

T0 read is triggered from app-thread t1 and read call passes through
   write-behind.
T1 app-thread t2 issues a write. The page on which read from t1 is
   waiting is marked stale
T2 write-behind caches write and indicates to application as write
   complete.
T3 app-thread t2 issues read to same region. Since, there is already a
   page for that region (created as part of read at T0), this read
   request waits on that page to be filled (though it is stale, which
   is a bug).
T4 read (triggered at T0) completes from brick (with write still
   pending). Now both read requests from t1 and t2 are served this data
   (though data is stale from app-thread t2's perspective - which is a
   bug)
T5 write is flushed to brick by write-behind.

Fix is to not to serve data from a stale page, but instead initiate a
fresh read to back-end.

Change-Id: Id6af733464fa41bb4e81fd29c7451c73d06453fb
BUG: 1414242
Signed-off-by: Raghavendra G 
Reviewed-on: https://review.gluster.org/7447
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Csaba Henk 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Zhou Zhengping 
Reviewed-by: Amar Tumballi

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

build: export minimum symbols from xlators for correct resolution

2015-12-22T17:15:01+00:00

Revisiting http://review.gluster.org/#/c/11814/, which unintentionally
introduced warnings from libtool about the xlator .so names.

According to [1], the -module option must appear in the Makefile.am
file(s); if -module is defined in a macro, e.g. in configure(.ac),
then libtool will not recognize that this is a module and will emit a
warning.

[1]
http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules

Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078
BUG: 1248669
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13003
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos