glusterfs.git/xlators/features/upcall/src/upcall.h, branch v7.0

upcall: some modifications to reduce work under lock

2019-02-19T15:29:23+00:00

1. Reduced the number of times we call time(). This may affect accuracy
of access time and so on - please review carefully. I think the resolution is OK'ish.
2. Removed dead code.
3. Changed from CALLOC() to MALLOC() where it made sense.
4. Moved some bits of work outside of a lock.

Compile-tested only!
updates: bz#1193929
Signed-off-by: Yaniv Kaul 

Change-Id: I9fb8ca5d79b0e9126c1eb07e1a1ab5dbd8bf3f79

libglusterfs: Move devel headers under glusterfs directory

2018-12-05T21:47:04+00:00

libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR

Land clang-format changes

2018-09-12T11:52:48+00:00

Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b

multiple: fix struct/typedef inconsistencies

2017-06-30T10:31:02+00:00

The most common pattern, both in our code and elsewhere, is this:

   struct _xyz {
      ...
   };
   typedef struct _xyz xyz_t;

These exceptions - especially call_frame/call_stack - have been slowing
down code navigation for years.  By converging on a single pattern,
navigating from xyz_t in code to the actual definition of struct _xyz
(i.e. without having to visit the typedef first) might even be
automatable.

Change-Id: I0e5dd1f51f98e000173c62ef4ddc5b21d9ec44ed
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17650
Smoke: Gluster Build System 
Tested-by: Jeff Darcy 
CentOS-regression: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Niels de Vos

upcall: Update the access time in missing cases

2017-06-09T11:30:31+00:00

Issue: In fops like rename, link, unlink etc, the parent
dirrs' client access time was not being updated. And in fops like
create, link, symlink etc. the new file/dirs' client access time was
not updated.

Solution: Update the client access time for both parent and new entry.

Change-Id: Id9f63583216ae857f6251dca15797ac66fa85430
BUG: 1458127
Signed-off-by: Poornima G 
Reviewed-on: https://review.gluster.org/17450
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

md-cache, afr: Reduce the window of stale read

2016-10-20T07:07:55+00:00

Problem:
Consider a replica setup, where one mount writes data to a
file and the other mount reads the file. In afr, read operations
are not transaction based, a brick(read subvolume) is chosen as
a part of lookup or other operations, read is always wound only
to the read subvolume, even if there was write from a different client
that failed on this brick. This stale read continues until there is
a lookup or any write operation from the mount point. Currently, this
is not a major issue, as a lookup is issued before every read and it will
switch the read subvolume to a correct one. But with the plan of
increasing md-cache timeout to 600s, the stale read problem will be
more pronounced, i.e. stale read can continue for 600s(or more if cascaded
with readdirp), as there will be no lookups.

Solution:
Afr doesn't have any built-in solution for stale read(without affecting
the performance). The solution that came up, was to use upcall. When a file
on any brick is marked bad for the first time, upcall sends a notification
to all the clients that had recently accessed the file. The solution has
2 parts:
- Identifying when a file is marked bad, on any of the bricks,
  for the first time
- Client side actions on recieving the notifications

Identifying when a file is marked bad on any of the bricks for the first time:
-----------------------------------------------------------------------------
The idea is to track xattrop in upcall. xattrop currently comes with 2 afr
xattrs - afr dirty bit and afr pending xattrs.
   Dirty xattr is set to 1 before every write, and is unset if write succeeds.
In certain scenarios, dirty xattr can be 0 and still the file could be bad
copy. Hence do not track dirty xattr.
   Pending xattr is set on the good copy, indicating the other bricks that have
bad copy. It is still not as simple as, notifying when any of the pending xattrs
change. It could lead to flood of notifcations, in case the other brick is
completely down or consistantly failing. Hence it is important to notify only
once, the first time a good copy is marked bad.

Client side actions on recieving pending xattr change, notification:
--------------------------------------------------------------------
md-cache will invalidate the cache of that file, so that further lookup is
passed down to afr and hence update the read subvolume. Invalidating only in
md-cache is not enough, consider the folling oder of opertaions:
- pending xattr invalidation - invalidate md-cache
- readdirp on the bad read subvolume - fill md-cache
- lookup (served from md-cache)
- read - wound to the old read subvol.
Hence, along with invalidating md-cache, it is very important to reset the
read subvolume for that file, in afr.

Design Credit: Anuradha Talur, Ravishankar N

1. xattrop doesn't carry info saying post op/pre op.
2. Pre xattrop will have 0 value for all pending xattrs,
   the cbk of pre xattrop carries the on-disk xattr value.
   Non zero indicated healing is required.
3. Post xattrop will have non zero value for any of the
   pending xattrs, if the fop failed on any of the bricks.

Change-Id: I469cbc111714c433984fe1c922be2ef113c25804
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15398
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

md-cache: Register the list of xattrs with cache-invalidation

2016-08-31T06:07:01+00:00

Issue:
md-cache caches a specified list of xattrs, and when cache invalidation
is enabled, it makes sense to recieve invalidation only when those xattrs
are modified by other clients. But the current implementation of upcall
is that, it will send invalidation when any of the on-disk xattrs is modified.

Solution:
md-cache sends a list of xattrs that it is interested in, to upcall by
issuing an ipc(). The challenge here is to make sure everytime a brick
goes offline and comes back up, the ipc() needs to be issued to the
bricks. Hence ipc() is sent from md-cache every time there is a
CHILD_UP/CHILD_MODIFIED event.

TODO:
There will be patches following, in cluster xlators, to implement ipc fop.

Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15002
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra G

features/upcall: segment fault while join thread reaper_thr in fini()

2016-08-29T12:13:49+00:00

reaper_thr thread may not be started according to option
'cache-invalidation', if it's not started, join it in fini will cause a
segment fault.

Change-Id: I1c145a5feb137767880a08e79f810537283fb6b9
BUG: 1369524
Signed-off-by: Ryan Ding 
[ndevos: check .reaper_init_done and make it a boolean]
Reviewed-on: http://review.gluster.org/15298
Smoke: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

upcall: Invalidation for xattr should also carry a valid stat

2016-07-05T11:53:03+00:00

xattr modification also impacts the ctime of the file, hence
include the new stat along with the new xattrs, in the
invalidation request

Change-Id: Ieaa4382fa62f397f61a995b926013b9207c6e9da
BUG: 1352671
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/14828
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

upcall: pass dict with xattrs on xattr invalidation

2016-05-19T09:28:07+00:00

In case of xattr invalidation, return a dict containing
the updated xattrs.

[ndevos: move chunks to change 12995 and only address the xattrs-dict here]

Change-Id: I8733f06a519a9a0f24be1bb4b2c38c9c9dce0ce2
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/12996
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: soumya k 
Tested-by: soumya k