glusterfs.git/rpc/xdr, branch v3.5.3beta2

libgfapi: Added support to fetch volume info from glusterd and store in glfs object.

2014-05-12T15:14:52+00:00

Defined new APIs in the libgfapi module, given a glfs object,
 * to send handshake RPC call to glusterd process to fetch UUID of the volume
 * store it in the glusterfs_context linked to the glfs object.
 * to parse UUID from its cannonical string format into 16-byte array
   before sending it to the libgfapi users.

Defined a RPC call in glusterd which can be used to query volume related
info by other processes using 'clnt_handshake_procs'.

Note - Currently this RPC call to glusterd process is used only to fetch UUID.
But it can be extended to get other volume related structures as well.

In addition to the above, defined a new variable to keep track of such handshake
RPCs still in progress to make sure all the corresponding RPC callbacks have been
processed before libgfapi returns the glfs object initialized.

Also bumping up the GFAPI current version number since there is a new API
"glfs_get_volume_id" defined and exposed by libgfapi as part of these changes.

Cherry picked from commit 5adb10b9ac1c634334f29732e062b12d747ae8c5:
> Change-Id: I303f76d7177d32d25bdb301b1dbcf5cd73f42807
> BUG: 1095775
> Signed-off-by: Soumya Koduri 
> Reviewed-on: http://review.gluster.org/7218
> Reviewed-by: Anand Avati 
> Reviewed-by: Harshavardhana 
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

This change differs a little from the patch in the master branch:
- libgfapi in 3.5 does not have glfs_get_volfile(), so there were some
  merge conflicts resolved,
- libgfapi in 3.5 is not versioned, the configure.ac changes related to
  the versioning have been skipped,
- in the master branch only the XDR .x files are available, release-3.5
  requires a manual re-generation of the relates .h and .c files.

Change-Id: I52c32d0e69a52a7f4285f74164bca6fd83c4f3b3
BUG: 1095775
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/7741
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos

cli: More checks in rebalance status output

2013-12-23T14:56:17+00:00

Change-Id: Ibd2edc5608ae6d3370607bff1c626c8347c4deda
BUG: 1031887
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6561
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cli/glusterd: Changes to quota command Quota feature

2013-11-26T18:25:27+00:00

 re-work.

Following are the cli commands that are new/re-worked:
======================================================

volume quota  {enable|disable|list [ ...]|remove | default-soft-limit } |
volume quota  {limit-usage   []} |
volume quota  {alert-time|soft-timeout|hard-timeout} {}
volume status [all |  [nfs|shd||quotad]] [detail|clients|mem|inode|fd|callpool]
volume statedump  [nfs|quotad] [all|mem|iobuf|callpool|priv|fd|inode|history]

glusterd changes:
=================
* Quota limits are now set as extended attributes by glusterd from
  the aux mount created by the cli.
* The gfids of the directories on which quota limits are set
  for a given volume are stored in
  /var/lib/glusterd/vols//quota.conf file in binary format,
  and whose cksum and version is stored in
  /var/lib/glusterd/vols//quota.cksum.

Original-author: Krutika Dhananjay 
Original-author: Krishnan Parthasarathi 

BUG: 969461
Change-Id: If32bba36c67f9c2a30417af9c6389045b2b7c13b
Signed-off-by: Krutika Dhananjay 
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6003
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterfs: zerofill support

2013-11-11T05:25:49+00:00

Add support for a new ZEROFILL fop. Zerofill writes zeroes to a file in
the specified range. This fop will be useful when a whole file needs to
be initialized with zero (could be useful for zero filled VM disk image
provisioning or  during scrubbing of VM disk images).

Client/application can issue this FOP for zeroing out. Gluster server
will zero out required range of bytes ie server offloaded zeroing. In
the absence of this fop,  client/application has to repetitively issue
write (zero) fop to the server, which is very inefficient method because
of the overheads involved in RPC calls  and acknowledgements.

WRITESAME is a  SCSI T10 command that takes a block of data as input and
writes the same data to other blocks and this write is handled
completely within the storage and hence is known as offload . Linux ,now
has support for SCSI WRITESAME command which is exposed to the user in
the form of BLKZEROOUT ioctl.  BD Xlator can exploit BLKZEROOUT ioctl to
implement this fop. Thus zeroing out operations can be completely
offloaded to the storage device , making it highly efficient.

The fop takes two arguments offset and size. It zeroes out 'size' number
of bytes in an opened file starting from 'offset' position.

This patch adds zerofill support to the following areas:
	- libglusterfs
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

Client applications can exloit this fop by using glfs_zerofill introduced in
libgfapi.FUSE support to this fop has not been added as there is no system call
for this fop.

Changes from previous version 3:
* Removed redundant memory failure log messages

Changes from previous version 2:
* Rebased and fixed build error

Changes from previous version 1:
* Rebased for latest master

TODO :
     * Add zerofill support to trace xlator
     * Expose zerofill capability as part of gluster volume info

Here is a performance comparison of server offloaded zeofill vs zeroing
out using repeated writes.

[root@llmvm02 remote]# time ./offloaded aakash-test log 20

real	3m34.155s
user	0m0.018s
sys	0m0.040s
[root@llmvm02 remote]# time ./manually aakash-test log 20

real	4m23.043s
user	0m2.197s
sys	0m14.457s
[root@llmvm02 remote]# time ./offloaded aakash-test log 25;

real	4m28.363s
user	0m0.021s
sys	0m0.025s
[root@llmvm02 remote]# time ./manually aakash-test log 25

real	5m34.278s
user	0m2.957s
sys	0m18.808s

The argument log is a file which we want to set for logging purpose and
the third argument is size in GB .

As we can see there is a performance improvement of around 20% with this
fop.

Change-Id: I081159f5f7edde0ddb78169fb4c21c776ec91a18
BUG: 1028673
Signed-off-by: Aakash Lal Das 
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5327
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cli,glusterd: Implement 'volume status tasks'

2013-10-09T06:13:16+00:00

oVirt's Gluster Integration needs an inexpensive command that can be
executed every 10 seconds to monitor async tasks and their parameters,
for all volumes.

The solution involves adding a 'tasks' sub-command to 'volume status'
to fetch only the async task IDs, type and other relevant parameters.
Only the originator glusterd participates in this command as all the
information needed is available on all the nodes. This is to make the
command suitable for being executed every 10 seconds.

Change-Id: I1edc607baf29b001a5585079dec681d7c641b3d1
BUG: 1012346
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/6006
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

cli/glusterd: improve rebalance fix-layout status reporting

2013-09-19T16:22:36+00:00

Problem:
Currenly the CLI rebalance status command output does not indicate the
'type' of rebalance, i.e. whether a full rebalance or only a fix-layout
was carried out.

Fix: After the rebalance status of all peers is received by the
originator glusterd, alter it to reflect the type of rebalance
before passing it on to the CLI process.

Change-Id: I1940ffda0d36e25e5b33c84a0ea210394cc9e1d3
BUG: 1004744
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/5826
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd/cli: Geo-Replication "status detail" cmd

2013-09-05T02:50:49+00:00

Provides detailed status info in the following format

                    MASTER   SLAVE 

NODE   HEALTH   UPTIME  FILES SYNCD  FILES PENDING  BYTES PENDING  DELETES PENDING
-----------------------------------------------------------------------------------

This patch introdues "status detail" command to show crawl related
information in CLI. These values are "pulled" from gsyncd when
"status detail" is executed.

Change-Id: I1fdaf7180eacce054a864d34971dc160bd7301e1
BUG: 990420
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/5590
Tested-by: Gluster Build System 
Reviewed-by: Avra Sengupta 
Tested-by: Avra Sengupta 
Reviewed-by: Anand Avati

glusterd/cli changes for distributed geo-rep

2013-07-26T20:19:18+00:00

Commands:
gluster system:: execute gsec_create
gluster volume geo-rep   create [push-pem] [force]
gluster volume geo-rep   start [force]
gluster volume geo-rep   stop [force]
gluster volume geo-rep   delete
gluster volume geo-rep   config
gluster volume geo-rep   status

The geo-replication is distributed. The session will be created, and
gsyncd will be spawned on all relevant nodes, instead of only one
node.

geo-rep: Collecting status detail related data

Added persistent store for saving information about
TotalFilesSynced, TotalSyncTime, TotalBytesSynced

Changes in the status information in socket:
Existing(Ex):
FilesSynced=2;BytesSynced=2507;Uptime=00:26:01;

New(Ex):
FilesSynced=2;BytesSynced=2507;Uptime=00:26:01;SyncTime=0.69978;
TotalSyncTime=2.890044;TotalFilesSynced=6;TotalBytesSynced=143640;

Persistent details stored in
/var/lib/glusterd/geo-replication/${mastervol}/${eSlave}-detail.status

Change-Id: I1db7fc13ffca2e415c05200b0109b1254067f111
BUG: 847839
Original Author: Avra Sengupta 
Original Author: Venky Shankar 
Original Author: Aravinda VK 
Original Author: Amar Tumballi 
Original Author: Csaba Henk 
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/5132
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

rpc: duplicate request cache for nfs

2013-06-21T17:33:49+00:00

Duplicate request cache provides a mechanism for detecting
duplicate rpc requests from clients. DRC caches replies
and on duplicate requests, sends the cached reply instead of
re-processing the request.

Change-Id: I3d62a6c4aa86c92bf61f1038ca62a1a46bf1c303
BUG: 847624
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.org/4049
Reviewed-by: Rajesh Joseph 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd: Log peer op status at the appropriate time

2013-06-19T05:15:12+00:00

Change-Id: Ia8e1af082078f2f791708ba4faa4992bf291dd6e
BUG: 961339
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/5023
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur