glusterfs.git/xlators/features/shard, branch v3.8dev

features/shard: Introduce file size xattr

2015-04-08T15:23:29+00:00

With each inode write FOP, the size and block count of the file will be
updated within the xattr. There are two 64 byte fields that are
intentionally left blank for now for future use when consistency
guarantee is introduced later in sharding.

Change-Id: I40a2e700150c1f199a6bf87909f063c84ab7bb43
BUG: 1207603
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/10097
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System

features/shard: Refactor code

2015-04-07T06:09:28+00:00

* Renamed shard_writev_create_write_shards() to shard_common_resolve_shards()
  to appropriately reflect its functionality and for reuse in other fops too.

* Move code common to MKNOD and CREATE into a macro.

* Cut down on if nesting in shard_lookup_cbk()

Change-Id: I488255499673accd426390c6d42f2b39bab3d637
BUG: 1205661
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/10096
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

features/shard: Create and use xattr_req dict as and when needed

2015-04-04T17:46:07+00:00

Reusing local->xattr_req for the several calls and callbacks per xlator fop
would cause keys set from previous call/cbk (sometimes even by the xlators
below) to remain which in some cases can lead to errors.  For instance, the
presence of "trusted.glusterfs.dht.*" keys (which are remnants of the previous
call/cbk), can cause the GF_IF_INTERNAL_XATTR_GOTO() check in DHT to fail when
the same dict is used to wind [f]setxattr.

Change-Id: I8612d020f83f3dc55e4a34d10ccbdaf11d7b4fdd
BUG: 1205661
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/10095
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/shard: Fail writes if /.shard already exists as a file

2015-04-03T13:33:59+00:00

Change-Id: Id7250ca4637c37a005cf2def43d5b843c1ea6562
BUG: 1205661
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/10094
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/shard: Bug fixes

2015-03-27T18:27:24+00:00

* Return number of bytes written in writev cbk on success
* Eliminate separate inode table for sharding xlator.
* Fix appearance of "shard" as an option within the
  volfile for subvolume of type features/shard.
* Fix values of min and max allowed shard block size
* Return @new as opposed to NULL in shard_create_gfid_dict() on success

Change-Id: I6319d377a196d1c5ceed1f65d337ff8eabcb21f8
BUG: 1205661
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/10003
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/shard: Introducing sharding translator

2015-03-19T17:59:31+00:00

Based on the high-level design by Anand V. Avati which can be found @
https://gist.github.com/avati/af04f1030dcf52e16535#sharding-xlator-stripe-20

Still to-do:
        * complete implementation of inode write fops - [f]truncate,
          zerofill, fallocate, discard
        * introduce transaction mechanism in inode write fops
        * complete readv
        * Handle open with O_TRUNC
        * Handle unlinking of all shards during unlink/rename
        * Compute total ia_size and ia_blocks in lookup, readdirp, etc
        * wind fsync/flush on all shards

        Note: Most of the items above are related. Once we come up
        with a clean way to determine the last shard/shard count for
        a file/file size and the mgmt of sparse regions of the file,
        implementing them becomes trivial.

Change-Id: Id871379b53a4a916e4baa2e06f197dd8c0043b0f
BUG: 1200082
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/9841
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur