glusterfs.git/xlators/mgmt/glusterd/src/glusterd-brick-ops.c, branch v3.5.5

glusterd: Save/restore/sync rebalance dict

2013-12-23T14:58:03+00:00

        Backport of http://review.gluster.org/6492

A dictionary was added to store additional information of a rebalance
process, like the bricks being removed in case of a rebalance started
by remove-brick. This dictionary wasn't being stored/restored or synced
during volume sync, leading to errors like a volume status command
failing. These issues have been fixed in this patch. The rebalance dict
is now stored/restored and also exported/imported during volume sync.

Also, this makes sure that the rebalance dict is only create on
remove-brick start. This adds a bricks decommissioned status to the
information imported/exported during volume sync.

Change-Id: I56fed23dc2de80a96648055fe705e9c3ffd55227
BUG: 1040809
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/6565
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

bd: Add support to create clone, snapshot and merge of LV images.

2013-11-13T19:39:22+00:00

Special xattr names "clone" & "snapshot" can be used to create full and
linked clone of the LV images. GFID of destination posix file (to be
mapped) is passed as a value to the xattr. Destination posix file must
exist before running this operation.

These operations form a basis for offloading storage related operations
from QEMU to GlusterFS.

Syntax for full clone: xattr name: "clone" value: "gfid-of-dest-file"
Syntax for linked clone: xattr name: "snapshot" value: "gfid-of-dest-file"
Syntax for merging: xattr name: "merge" value: "path-to-snapshot-file"

Example:
	setfattr -n clone -v  /media/source
	setfattr -n snapshot -v  /media/source
	setfattr -n merge -v "/media/sn" /media/sn

Change-Id: Id9f984a709d4c2e52a64ae75bb12a8ecb01f8776
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5626
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

bd: posix/multi-brick support to BD xlator

2013-11-13T19:38:42+00:00

Current BD xlator (block backend) has a few limitations such as
* Creation of directories not supported
* Supports only single brick
* Does not use extended attributes (and client gfid) like posix xlator
* Creation of special files (symbolic links, device nodes etc) not
  supported

Basic limitation of not allowing directory creation is blocking
oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM
creates multi-level directories when GlusterFS is used as storage
backend for storing VM images.

To overcome these limitations a new BD xlator with following
improvements is suggested.

* New hybrid BD xlator that handles both regular files and block device
  files
* The volume will have both POSIX and BD bricks. Regular files are
  created on POSIX bricks, block devices are created on the BD brick (VG)
* BD xlator leverages exiting POSIX xlator for most POSIX calls and
  hence sits above the POSIX xlator
* Block device file is differentiated from regular file by an extended
  attribute
* The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a
  posix file to Logical Volume (LV).
* When a client sends a request to set BD_XATTR on a posix file, a new
  LV is created and mapped to posix file. So every block device will
  have a representative file in POSIX brick with 'user.glusterfs.bd'
  (BD_XATTR) set.
* Here after all operations on this file results in LV related
  operations.

For example opening a file that has BD_XATTR set results in opening
the LV block device, reading results in reading the corresponding LV
block device.

When BD xlator gets request to set BD_XATTR via setxattr call, it
creates a LV and information about this LV is placed in the xattr of the
posix file. xattr "user.glusterfs.bd" used to identify that posix file
is mapped to BD.

Usage:
Server side:
[root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2
It creates a distributed gluster volume 'bdvol' with Volume Group vg1
using posix brick /storage/vg1_info in host1 and Volume Group vg2 using
/storage/vg2_info in host2.

[root@host1 ~]# gluster volume start bdvol

Client side:
[root@node ~]# mount -t glusterfs host1:/bdvol /media
[root@node ~]# touch /media/posix
It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick
[root@node ~]# mkdir /media/image
[root@node ~]# touch /media/image/lv1
It also creates regular posix file 'lv1' in either host1:/vg1 or
host2:/vg2 brick
[root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1
[root@node ~]#
Above setxattr results in creating a new LV in corresponding brick's VG
and it sets 'user.glusterfs.bd' with value 'lv: --deltag
>

Changes from previous version V5:
* Removed support for delayed deleting of LVs

Changes from previous version V4:
* Consolidated the patches
* Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR.

Changes from previous version V3:
* Added support in FUSE to support full/linked clone
* Added support to merge snapshots and provide information about origin
* bd_map xlator removed
* iatt structure used in inode_ctx. iatt is cached and updated during
fsync/flush
* aio support
* Type and capabilities of volume are exported through getxattr

Changes from version 2:
* Used inode_context for caching BD size and to check if loc/fd is BD or
  not.
* Added GlusterFS server offloaded copy and snapshot through setfattr
  FOP. As part of this libgfapi is modified.
* BD xlator supports stripe
* During unlinking if a LV file is already opened, its added to delete
  list and bd_del_thread tries to delete from this list when a last
  reference to that file is closed.

Changes from previous version:
* gfid is used as name of LV
* ? is used to specify VG name for creating BD volume in volume
  create, add-brick. gluster volume create volname host:/path?vg
* open-behind issue is fixed
* A replicate brick can be added dynamically and LVs from source brick
  are replicated to destination brick
* A distribute brick can be added dynamically and rebalance operation
  distributes existing LVs/files to the new brick
* Thin provisioning support added.
* bd_map xlator support retained
* setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and
  setfattr -n user.glusterfs.bd -v "thin" creates thin LV
* Capability and backend information added to gluster volume info (and
--xml) so
  that management tools can exploit BD xlator.
* tracing support for bd xlator added

TODO:
* Add support to display snapshots for a given LV
* Display posix filename for list-origin instead of gfid

Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/4809
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

bd_map: Remove bd_map xlator

2013-11-13T19:38:28+00:00

Remove bd_map xlator and CLI related changes.

Change-Id: If7086205df1907127c1a1fa4ba603f1c48421d09
BUG: 1028672
Signed-off-by: M. Mohan Kumar 
Reviewed-on: http://review.gluster.org/5747
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd: modify remove-brick CLI message.

2013-11-08T07:54:26+00:00

In the current context "replica_cnt" is used just to know whether the
specific key exists or not by calling "dict_get_int32", which we can
replace by "dict_get ()". And changing the log message as it is more
appropriate to say "migration of data" rather than "rebalance".

This patch refactors commit 51c6fa7a354826744de98 against BZ 961669

reviewed on : http://review.gluster.org/5566

Change-Id: I48eae206a28d4083975e64407ed8fe4539f9c24b
BUG: 1027270
Signed-off-by: Ravishankar N 
Original patch: Susant Palai 
Reviewed-on: http://review.gluster.org/6001
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: susant palai 
Reviewed-by: Vijay Bellur

glusterd: Don't reset rebalance status on add-brick

2013-09-18T15:58:29+00:00

The rebalance status was being reset to 'Not started' when add-brick was
performed. This would lead to odd cases where a 'rebalance status' on a
volume would show status as 'not started' but would also include the
rebalance statistics. This also affected the showing of asynchronus task
status in 'volume status' command.

By not resetting the status prevent the above issues from happening.
Since we use the running/not-running of the rebalance process as the
check when performing other operations we can safely leave the rebalance
stats collected on an add-brick.

Change-Id: I4c69d9c789d081c6de7e7a81dd0d4eba2e83ec17
BUG: 1006247
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/5895
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System

cli,glusterd: Task parameters in xml output

2013-09-13T10:38:04+00:00

This patch introduces task parameters for the asynchronus task shown in
volume status. The parameters are only given for xml output. The
parameters shown currently are,
- source and destination bricks for replace-brick tasks
  ......
        
          
            Replace brick
            3d1a1005-9d2e-4ae0-bd62-577bc1d333a3
            1
            
              archm:/export/test4
              archm:/export/test-replace1
            
          
        
  ......
- list of bricks being removed for remove-brick tasks
  ......
        
          
            Remove brick
            901c20ca-0da2-41de-8669-5f0caca6b846
            1
            
              archm:/export/test2
              archm:/export/test3
            
          
        
  ......

The changes for non-xml output will be done in a subsequent patch.

Change-Id: I322afe2f83ed8adeddb99f7962c25911204dc204
BUG: 916577
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/5771
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

mgmt/glusterd: Update sub_count on remove brick

2013-09-12T05:51:36+00:00

Change-Id: I7c17de39da03c6b2764790581e097936da406695
BUG: 1002556
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/5893
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Anand Avati

glusterd: Allowing root@hostname::slave georep sessions to be created.

2013-09-05T02:32:12+00:00

non-root@hostname::slave-vol geo-rep sessions are not supported.
only hostname and root@hostname sessions are supported, and are
treated as the same.

Change-Id: I87551e1bd4ff4e0e6520c34eb3d944587cc65476
BUG: 998933
Signed-off-by: Avra Sengupta 
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/5659
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd: add check in remove-brick start variant

2013-08-21T17:45:59+00:00

The 'start' variant of the remove-brick command only applies at the dht
level wherein we can remove  all the bricks of a sub-volume (and remove
multiple such sub-volumes) but not select bricks of it.

This patch disallows removing individual replica bricks of multiple
sub-volumes (i.e. reducing the replcia count of the volume) using
remove-brick 'start'. The preferred method for such an operation is to use
commit force.

This patch also reverts the check to prevent removal of bricks from a
replicate volume (commit 0d415f7)

BUG: 961669
Change-Id: I447ad27f73a0963b5e09fb317bf7267a7a5a6147
Signed-off-by: Ravishankar N 
Reviewed-on: http://review.gluster.org/5566
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Anand Avati