glusterfs.git/xlators, branch v3.10dev

io-stats: Add stats for upcall notifications

2016-09-01T03:18:42+00:00

With this patch, there will be additional entries seen in
the profile info:

UPCALL : Total number of upcall events that were sent from
         the brick(in brick profile), and number of upcall
         notifications recieved by client(in client profile)

Cache invalidation events:
-------------------------
CI_IATT : Number of upcalls that were cache invalidation and
         had one of the IATT_UPDATE_FLAGS set. This indicates
         that one of the iatt value was changed.

CI_XATTR : Number of upcalls that were cache invalidation, and
         had one of the UP_XATTR or UP_XATTR_RM set. This indicates
         that an xattr was updated or deleted.

CI_RENAME : Number of upcalls that were cache invalidation,
         resulted by the renaming of a file or directory

CI_UNLINK : Number of upcalls that were cache invalidation,
         resulted by the unlink of a file.

CI_FORGET : Number of upcalls that were cache invalidation,
         resulted by the forget of inode on the server side.

Lease events:
------------
LEASE_RECALL : Number of lease recalls sent by the brick (in
         brick profile), and number of lease recalls recieved
         by client(in client profile)

Note that the sum of CI_IATT, CI_XATTR, CI_RENAME, CI_UNLINK,
CI_FORGET, LEASE_RECALL may not be equal to UPCALL. This is
because, each cache invalidation can carry multiple flags.
Eg:
- Every CI_XATTR will have CI_IATT
- Every CI_UNLINK will also increment CI_IATT as link count is an
iatt attribute.

Also UP_PARENT_DENTRY_FLAGS is currently not accounted for,
as CI_RENAME and CI_UNLINK will always have the flag
UP_PARENT_DENTRY_FLAGS

Change-Id: Ieb8cd21dde2c4c7618f12d025a5e5156f9cc0fe9
BUG: 1371543
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15193
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Rajesh Joseph 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

event: fix gf_event messages for replace/reset brick op

2016-08-31T22:56:31+00:00

Change-Id: I80ebeeaffd2b228d7d0796c8d08bc2a051c4ccac
BUG: 1266876
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/15370
Smoke: Gluster Build System 
Reviewed-by: Atin Mukherjee 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri

upcall: Mark the clients as accessed on readdirp entries

2016-08-31T13:44:01+00:00

Currently when a client performs a readdirp it is not stored
in upcall, as one of the clients that have accessed the files.
Hence, when any other client modifies the file, the client that
had performed readdirp will not get any notifications.

Fix this by adding the clients to upcall database when they
perform readdirp.

Change-Id: I7767f1e26bf1bd1f67702a6d01f8aa64526ccc46
BUG: 1369430
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15313
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos

compound fops: Some fixes to compound fops framework

2016-08-31T06:50:40+00:00

Change-Id: I808fd5f9f002a35bff94d310c5d61a781e49570b
BUG: 1360169
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/15010
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

md-cache: Process all the cache invalidation flags

2016-08-31T06:40:52+00:00

Currently, md-cache only processes IATT_UPDATE_FLAGS, UP_XATTR and
UP_XATTR_RM. We also need to process UP_RENAME_FLAGS, UP_FORGET,
UP_PARENT_DENTRY_FLAGS and UP_NLINK_FLAGS. Otherwise the files
unlinked or renamed will not be reflected on other mounts.

Change-Id: Icb8b03da51482c3fc2e2a7292d16d56e11a341d9
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15324
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

dht, md-cache, upcall: Add invalidation of IATT when the layout changes

2016-08-31T06:08:54+00:00

Issue:
dht_layout is built as a part of lookup only. The layout can be
modified by rebalance process. Since every IO fop is preceded
by a lookup, there are very less issues of stale layout. But
with enhancements of aggressive caching of stats in md-cache,
the lookup will reduce and expose the stale layout issue often.

Solution:
Since stale layout is already an issue on dht, there is already
a plan to fix this at the dht layer, but this fix is not currently
planned for any release. Until this fix comes out, we can have
a workaround where, the upcall will send a notification to md-cache
when a layout xattr is changed. As a part of layout change notification
the existing cache is invalidated and the next lookup will fetch the
latest layout.

This is not a foolproof solution as the window between the layout change
and the next lookup(after invalidation of stat), where there will be stale
layout. But until the final fix comes in, this reduces the stale layout
window.

Change-Id: Iacf871a38b35880c1fc0bc68fe7ce291265e71d4
BUG: 1369638
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15300
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Raghavendra G

md-cache: Register the list of xattrs with cache-invalidation

2016-08-31T06:07:01+00:00

Issue:
md-cache caches a specified list of xattrs, and when cache invalidation
is enabled, it makes sense to recieve invalidation only when those xattrs
are modified by other clients. But the current implementation of upcall
is that, it will send invalidation when any of the on-disk xattrs is modified.

Solution:
md-cache sends a list of xattrs that it is interested in, to upcall by
issuing an ipc(). The challenge here is to make sure everytime a brick
goes offline and comes back up, the ipc() needs to be issued to the
bricks. Hence ipc() is sent from md-cache every time there is a
CHILD_UP/CHILD_MODIFIED event.

TODO:
There will be patches following, in cluster xlators, to implement ipc fop.

Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15002
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Rajesh Joseph 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Raghavendra G

glusterd (op-sm): fix unused variable warnings/errors

2016-08-31T02:27:40+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I6c1474547586b5e43f30545872ec512cdd32dac2
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15269
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai

dht/tiering: fix unused variable warnings/errors

2016-08-30T18:45:03+00:00

http://review.gluster.org/14085 fixes a/the "leak" - via the
generated rpc/xdr headers - of pragmas that mask these warnings.

However 14085 won't pass the smoke test until all the warnings are
fixed.

Change-Id: I367a737570dd7d2f6cc25f4bf4299d31bb6826aa
BUG: 1369124
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15242
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Dan Lambright

glusterd : Introduce reset brick

2016-08-30T02:55:53+00:00

The command basically allows replace brick with src and
dst bricks as same.

Usage:
gluster v reset-brick   start
This command kills the brick to be reset. Once this command is run,
admin can do other manual operations that they need to do,
like configuring some options for the brick. Once this is done,
resetting the brick can be continued with the following options.

gluster v reset-brick    commit {force}

Does the job of resetting the brick. 'force' option should be used
when the brick already contains volinfo id.

Problem: On doing a disk-replacement of a brick in a replicate volume
the following 2 scenarios may occur :

a) there is a chance that reads are served from this replaced-disk brick,
which leads to empty reads. b) potential data loss if next writes succeed
only on replaced brick, and heal is done to other bricks from this one.

Solution: After disk-replacement, make sure that reset-brick command is
run for that brick so that pending markers are set for the brick and it
is not chosen as source for reads and heal. But, as of now replace-brick
for the same brick-path is not allowed. In order to fix the above
mentioned problem, same brick-path replace-brick is needed.
With this patch reset-brick commit {force} will be allowed even when
source and destination  are identical as long as
1) destination brick is not alive
2) source and destination brick have the same brick uuid and path.
Also, the destination brick after replace-brick will use the same port
as the source brick.

Change-Id: I440b9e892ffb781ea4b8563688c3f85c7a7c89de
BUG: 1266876
Signed-off-by: Anuradha Talur 
Reviewed-on: http://review.gluster.org/12250
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ashish Pandey 
Reviewed-by: Pranith Kumar Karampuri