glusterfs.git/xlators/performance/md-cache, branch v3.7.16

readdir-ahead: Prefetch xattrs needed by md-cache

2016-05-11T11:02:57+00:00

Problem:
Negative cache feature implementation in md-cache requires xattrs
returned by posix to be intercepted for every call that can possibly
return xattrs. This includes readdirp(). This is crucial to treat
missing keys in cache as a case of negative entry (returns ENODATA)

md-cache puts names of xattrs that it wants to cache in xdata and
passes it down to posix which returns the specified xattrs in the
callback. This is done in lookup() and readdirp(). Hence, a xattr
that is cached can be invalidated during readdirp_cbk too.

This is based on the assumption that readdirp() will always return
all xattrs that md-cache is interested in. However, this is not the
case when readdirp() call is served from readdir-ahead's cache.
readdir-ahead xlator will pre-fetch dentries during opendir_cbk
and readdirp. These internal readdirp() calls made by readdir-ahead
xlator does not set xdata in it's requests. Hence, no xattrs are
fetched and stored in it's internal cache.

This causes metadata loss in gluster-swift. md-cache returns ENODATA
during getxattr() call even though the xattr for that object exists on
the brick. On receiving ENODATA, gluster-swift will create new metadata
and do setxattr(). This results in loss of information stored in
existing xattr.

Fix:
During opendir, md-cache will communicate to readdir-ahead asking it
to store the names of xattrs it's interested in so that readdir-ahead
can fetch those in all subsequent internal readdirp() calls issued by
it. This stored names of xattrs is invalidated/updated on the next
real readdirp() call issued by application. This readdirp() call will
have xdata set correctly by md-cache xlator.

> Reviewed-on: http://review.gluster.org/14214
> Tested-by: Prashanth Pai 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Tested-by: Gluster Build System 
> Smoke: Gluster Build System 
> Reviewed-by: Raghavendra G 
> Tested-by: Raghavendra G 

BUG: 1334700
Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9
(cherry picked from commit 0c73e7050c4d30ace0c39cc9b9634e9c1b448cfb)
Reviewed-on: http://review.gluster.org/14282
Tested-by: Prashanth Pai 
NetBSD-regression: NetBSD Build System 
Tested-by: Gluster Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

md-cache: Cache gluster-swift metadata

2016-03-31T07:48:53+00:00

> BUG: 1317785
> Change-Id: Ie02b8fc294802f8fdf49dee8bf97f1e6177d92bd
> Signed-off-by: Prashanth Pai 
> Reviewed-on: http://review.gluster.org/13735
> Smoke: Gluster Build System 
> Reviewed-by: Poornima G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Niels de Vos 
> Reviewed-by: Gaurav Kumar Garg 

BUG: 1317788
Change-Id: I86efca6a829cdda70ec7ed7fe0a16b73c25888c3
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/13753
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

md-cache: Remove readdirp fop for md-cache

2015-11-17T18:23:03+00:00

readdirp call will return inode for each entry and
will share this nodeid with kernal, also md-cache
will cache this gfid and base name. So when a lookup
operation is perfromed on such an inode, md-cache
will wind the call, that prevents populating inode
ctx for other lower layer xlators.

>Change-Id: I43c768703a3cc66d05b1c32909d1a2781001cb49
>BUG: 1236032
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/11894
>Tested-by: Gluster Build System 
>Reviewed-by: Raghavendra G 

(cherry picked from commit c8c9308134ae4ce24c630a1b0ccfcf4e8f9b0fe7)


Change-Id: Iaa5451f5ff25fb16119a6c3322b1787709d1aba4
BUG: 1266880
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12554
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

Logging: Porting the performance translator

2015-06-29T09:21:53+00:00

         logs to new logging framework

> Change-Id: Ie6aaf8d30bd4457bb73c48e23e6b1dea27598644
> BUG: 1194640
> Signed-off-by: arao 

BUG: 1217722
Change-Id: I0856c43dbf8c0a1aa084d4478c9bdf3f41dfc0b8
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/11442
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

performance/md-cache: set right error check in {f}getxattr_cbk()

2015-04-07T03:57:45+00:00

Currently mdc_{f}getxattr_cbk() check(s) for a non-zero value to
determine if any cache update has to be performed. Right from
posix xlator, op_ret has a positive value upon success and -1 upon
failure. This patch sets right the check in getxattr callbacks so
that xattr cache update happens for successful calls.

Change-Id: Ifa5ec38bdf7e3dc095de9a56d91559b13cd9e8b6
BUG: 1208784
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/10127
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

md-cache: cache virtual POSIX ACL xattrs

2015-03-27T17:45:12+00:00

http://review.gluster.org/9627 introduces two new virtual extended
attributes that are used similar to the existing POSIX ACL xattrs. These
new xattrs should get cached in the same way.

BUG: 1185654
Change-Id: I6294b4f9ade887e29f8bd8ae5a3642891df8a631
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9947
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

performance/md-cache: Initialise local->loc before winding nameless lookup

2015-01-24T05:37:37+00:00

That way, in the cbk, the latest values are updated in the cache.

Change-Id: Ia149e352e4763e8f5b910a3f4cb64d2dda4534b1
BUG: 1179169
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/9476
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

build: Remove files completely in make uninstall.

2015-01-01T11:10:29+00:00

If we create symlinks are part of make install,
should remove them as part of make uninstall.

Change-Id: Ie6e641caed60104cd256de4f020c1c6743d4ae60
BUG: 1177767
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9367
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

md-cache: check mtime_nsec and ctime_nsec fields for changes too

2014-11-20T07:01:27+00:00

When md-cache checks for an updated mtime/ctime in order to see if it
should request a cache invalidation, it only looks at the whole-second
"mtime" and "ctime" fields. If a file was modified a fraction of a
second after the time of the cached data, md-cache won't notice the
change.

BUG: 1164506
Change-Id: Ieb7baa46f84489d2d7ea8d60a7fb2595564eb955
Original-author: Philip Spencer 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9131
Tested-by: Gluster Build System 
Reviewed-by: Xavier Hernandez 
Reviewed-by: Vijay Bellur