summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* mgmt/glusterd: store for snapshot descriptionshishir gowda2013-11-151-2/+35
| | | | | Change-Id: I0ba50ba2963edf8d890a2dc78d48d42db7f71ae2 Signed-off-by: shishir gowda <sgowda@redhat.com>
* cli/snapshot : Changed the type of few variables.Sachin Pandit2013-11-151-13/+17
| | | | | | | | | | | | | | Minor changes to the type of few varaibles was required as comparison between "unsigned int" and "int" in "snap create parse" was leading to incorrect result. The default snapshot name should be time stamp, instead of that "snapshot" was getting assigned as a snapshot name. This patch also fixes the default snapshot name problem. Change-Id: I5b48444059bc400717b35647edec08a444bcfc48 Signed-off-by: Sachin Pandit <spandit@redhat.com>
* cli/snapshot : Minox fix, string literal.Sachin Pandit2013-11-151-1/+1
| | | | | Change-Id: I2a0b7e244256f1df82beb3e4815d6cacfee50603 Signed-off-by: Sachin Pandit <spandit@redhat.com>
* mgmt/glusterd : Printing error message if volume does not exist.Sachin Pandit2013-11-152-3/+41
| | | | | | | | | If user tries to list the snap details of volumes which does not exist, then corresponding error message is displayed. Change-Id: I205738be3dc632ccb074b639a2088cdd44aa35a7 Signed-off-by: Sachin Pandit <spandit@redhat.com>
* cli/snapshot: Ask user to confirm change of snap max limitshishir gowda2013-11-153-5/+27
| | | | | Change-Id: I6fd89a98c4a111bb6cd3e4d3c2b3d6f783661c27 Signed-off-by: shishir gowda <sgowda@redhat.com>
* glusterd/Jarvis: Added aggr rsp dict in mgmt frameworkAvra Sengupta2013-11-157-12/+154
| | | | | | | Also fixes snapshot config output Change-Id: Ia50d94492009cf73dbb99ba20117b9fa4c41048a Signed-off-by: Avra Sengupta <asengupt@redhat.com>
* Snapshot: fix for snapshot create crashRajesh Joseph2013-11-151-2/+2
| | | | | | | | | GL-205: Gluster snapshot create crashing. runner-arg should have NULL as the last argument. Change-Id: I1bd0090160b53a04a8073c31d91fb77f96f625dc Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
* mgmt/glusterd: set the appropriate status of the snap after createRaghavendra Bhat2013-11-151-0/+3
| | | | | Change-Id: I58a743c92bbd021c3a42c5184ba8acf4db48878a Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* snapcount in volinfo was not getting updated. Fixed itSachin Pandit2013-11-151-1/+1
| | | | | Change-Id: Idafe3cdba149c2a66b89fb3fe0d4d3791d9d089c Signed-off-by: Sachin Pandit <spandit@redhat.com>
* mgmt/glusterd: do not assert if op_errstr is NULL in snapshot config opsRaghavendra Bhat2013-11-151-2/+0
| | | | | | | * op_errstr is allocated and set while returning if there is any error Change-Id: I6e0de80d611aeeee3d25e8c20ab49b8ef42b0bf5 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* mgmt/glusterd: get the xlator object before accessing itRaghavendra Bhat2013-11-151-0/+2
| | | | | Change-Id: I3404106a7e4fa7d32b1d5824e079040d2ed8d76b Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* snapshot: Snapshot restoreRajesh Joseph2013-11-159-55/+933
| | | | | | | | | | | | GL-31: Ability to restore snapshot Implemented snapshot restore for thin logical volume. As of now snapshot restore for CG is not tested. Testing for snapshot restore of a volume is done by changing the snapshot create process to create a thick snapshot. This is done because --merge option to restore thin volume is not working in the latest kernel. Change-Id: Ia3ded7e6c4da5957a74e269a25ba3200e6fb2d8b Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
* mgmt/glusterd: save snapshot config values in storeshishir gowda2013-11-151-1/+71
| | | | | Change-Id: Ia755e5c4af84827cc9b8876054cc48cfdc598876 Signed-off-by: shishir gowda <sgowda@redhat.com>
* mgmt/glusterd: changes to create consistency group out of volumesRaghavendra Bhat2013-11-151-0/+12
| | | | | | | | | * Also send the proper error back to cli incase of any failure * Before taking the snap check whether a snap with the requested name already exists. Change-Id: I0830b31b1f095dd1d3d968c4f8b3cf46dc32d259 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* mgmt/glusterd: store option to update only snap_list.infoshishir gowda2013-11-153-1/+53
| | | | | Change-Id: I16b17ca60b5f9b34b7d238d8a3424a3b7a1dc435 Signed-off-by: shishir gowda <sgowda@redhat.com>
* mgmt/glusterd: snapshot config changesshishir gowda2013-11-1511-169/+656
| | | | | | | | Also refactored code in glusterd for create command Additionally, removed brick-op func from mgmt_iniate_all_phases Change-Id: Iddcc332009c5716adee7f2b04c93b352fb983446 Signed-off-by: shishir gowda <sgowda@redhat.com>
* CLI : Snapshot List,Integration with glusterdSachin Pandit2013-11-153-57/+135
| | | | | | | | | | | | | | | | | | Change in Naming convention: "snap_details", "snap_count" and so on is replaced by "snap-details", "snap-count" so on. Total snapcount introduced. Separate check is made for repeated Volume Name Ex : "gluster snapshot list vol1 vol2 vol1 vol2" is considered as "gluster snapshot list vol1 vol2" *This is still a work in progress* *have to test CG list once CG Store is ready* Change-Id: I45e2904eb8bdbf78de8665f20ba9605c38320307 Signed-off-by: Sachin Pandit <spandit@redhat.com>
* mgmt/glusterd: Fix glusterd crash due to extra unrefshishir gowda2013-11-151-2/+4
| | | | | | Change-Id: I9d600b4d971b7fdcd54da50e4a069eab19648fa6 Original-author: Rajesh Joseph <rajeshatredhat@redhat.com> Signed-off-by: shishir gowda <sgowda@redhat.com>
* Snapshot: Added new XDR typesRajesh Joseph2013-11-153-15/+40
| | | | | | | Added new XDR types for all the snapshot command. Change-Id: I46c02ea8e9c81c7967a773386c4b77b5eb6d5075 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
* protocol/server: avoid rpc reply to the client only if the fop is barrieredRaghavendra Bhat2013-11-151-4/+7
| | | | | | | | * Now, reply submission to the client is avoided even if the fop is not barriered Change-Id: I9d503935c4d851f5dd0e8e4be0fea9c92e636ff3 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* mgmt/glusterd: handle snap volume store and actual volume store separatelyRaghavendra Bhat2013-11-154-21/+64
| | | | | Change-Id: I8b88fe94d0f9ee1089cafdda037abcf2f7a180ca Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* mgmt/glusterd: Snapstore fixes to make info file persistentshishir gowda2013-11-151-6/+11
| | | | | Change-Id: I30cbbeb135c2d0a780e9e414ac0a96739e25647b Signed-off-by: shishir gowda <sgowda@redhat.com>
* mgmt/snapshot: brick op for starting/stopping barriershishir gowda2013-11-156-7/+281
| | | | | Change-Id: Iafbd0ec95de0c41455fb79953fb4bb07721334a5 Signed-off-by: shishir gowda <sgowda@redhat.com>
* mgmt/glusterd: changes to start the brick process of the snap volumeRaghavendra Bhat2013-11-153-10/+54
| | | | | Change-Id: I54db2fa67ebb6b57629f9536c296fbae07a1d159 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* CLI : snapshot list cli interfaceSachin Pandit2013-11-153-54/+518
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | $gluster snapshot list *prints snaps of all volume* $gluster snapshot list -d *prints snaps of all volume with details* $gluster snapshot list vol1 *prints snaps of volume "vol1"* $gluster snapshot list vol1 -d *prints snaps of volume "vol1" with details* $gluster snapshot list vol1 vol2 *prints snaps of volume "vol1" & "vol2" $gluster snapshot list vol1 vol2 -d *prints snaps of volume "vol1" & "vol2" with details* $gluster snapshot list -c cgname *prints snaps of all volume present in the group "cgname"* $gluster snapshot list -c cgname -d *prints snaps of all volume present in the group "cgname" with details* ** As of now you wont be able to see any output as actual snap create is not integrated ** Change-Id: I60eeafc715a51f1c564a270bb4124368038012b1 Signed-off-by: Sachin Pandit <spandit@redhat.com>
* mgmt/glusterd:store for CGshishir gowda2013-11-154-110/+493
| | | | | Change-Id: I6ec888a5553ad29ded032c02c80dd940b2aae007 Signed-off-by: shishir gowda <sgowda@redhat.com>
* mgmt/glusterd: snapshot create commandRaghavendra Bhat2013-11-1512-98/+1513
| | | | | | | | | | | | | | | | | This is still a work in progress. As of now, these things are done: * Take the snapshot of the backend brick * Create the new volume for the snapshot * Create the brick and the client volfiles * Store the snapshot related info in /var/lib/glusterd * Create the snap object representing the snapshot TODO: Start the brick processes for the snapshot Change-Id: I26fbb0f8e5cf004d4c1dbca51819bab1cd1bac15 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* mgmt/glusterd: Snapshot list supportRajesh Joseph2013-11-155-19/+1032
| | | | | | | | Handles snapshot list command issued by cli. Details of all the snapshots will be sent back to the caller in required format. Change-Id: I01e512290548007c06e90b40a59cdde048fab954 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
* mgmt/glusterd: Store for snapshotshishir gowda2013-11-156-13/+644
| | | | | | | | | | | | | | | | | | | | Introduced a new store for storing snapshot list for a given volume. $GLUSTERD_INSTALL_PATH/vols/<volname>/snap_list.info $GLUSTERD_INSTALL_PATH/vols/<volname>/snaps/ $GLUSTERD_INSTALL_PATH/vols/<volname>/snaps/<snap-name>/info <-snapshot volume info $GLUSTERD_INSTALL_PATH/vols/<volname>/snaps/<snap-name>/bricks <-snapshot volume brick dir $GLUSTERD_INSTALL_PATH/vols/<volname>/snaps/<snap-name>/bricks/<infos> <-snapshot volume brick info files store delete options TODO - $GLUSTERD_INSTALL_PATH/CG/ <-place holder for all cg's .../CG/<cg-name>/info <- per cg information placeholder Change-Id: I1f9fd8ff7cc0682d05b33965736a43dca6adb3e9 Signed-off-by: shishir gowda <sgowda@redhat.com>
* glusterd/locks: Adding multiple volume locks supportsAvra Sengupta2013-11-157-129/+285
| | | | | | | Also linking snap create command to mgmt_v3 Change-Id: If2ed29be072e10d0b0bd271d53e48eeaa6501ed7 Signed-off-by: Avra Sengupta <asengupt@redhat.com>
* glusterd: Defining interfaces to make use of mgmt_v3 rpcsAvra Sengupta2013-11-153-1/+1246
| | | | | | | | | Defining separate interfaces for every phase to make use of the rpcs and providing set of integrated interfaces for commands to consume Change-Id: I6d464326c5a8b5875a7c2539c9df072b23fe61a9 Signed-off-by: Avra Sengupta <asengupt@redhat.com>
* glusterd/mgmt_v3: Initating complete synctask on glusterdAvra Sengupta2013-11-1512-218/+1734
| | | | | | | | | | | | glusterd mgmt_v3 is nothing but a complete synctask approach for glusterd to function. The commands making use of this won't be using the op-state machine to inject events and will be using the synctask framework to perform operations across all nodes in the cluster. This patch defines the program and the handlers used. Change-Id: Ibff2c62b0187c40cdea7254c85786297bba60372 Signed-off-by: Avra Sengupta <asengupt@redhat.com>
* cli: snapshot create cli interface.Avra Sengupta2013-11-1511-1/+681
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | $ gluster snapshot help snapshot help - display help for snapshot commands snapshot create <volnames> [-n <snap-name/cg-name>] [-d <description>] - Snapshot Create. $ gluster snapshot create vol1 snapshot create: ???: snap created successfully $ gluster snapshot create vol1 vol2 snapshot create: ???: consistency group created successfully (The ??? will be replaced by the glusterd snap create command with the generated snap-name or cg-name) $ gluster snapshot create vol1 vol2 -n CG1 snapshot create: CG1: consistency group created successfully $ gluster snapshot create vol1 -n snap1 -d Description snapshot create: snap1: snap created successfully $ gluster snapshot create vol1 -n snap1 -d "Description can have -d within quotes" snapshot create: snap1: snap created successfully $ gluster snapshot create vol1 -n snap1 -d Description cant have -d without quotes snapshot create: failed: Options(-n/-d) are not valid descriptions Usage: snapshot create <volnames> [-n <snap-name/cg-name>] [-d <description>] $ gluster snapshot create vol1 -n "Multi word snap name" -d Description snapshot create: failed: Invalid snap name Usage: snapshot create <volnames> [-n <snap-name/cg-name>] [-d <description>] $ gluster snapshot create vol1 -d Description -n "-d" snapshot create: failed: Options(-n/-d) are not valid snap names Usage: snapshot create <volnames> [-n <snap-name/cg-name>] [-d <description>] $ gluster snapshot create vol1 -d -n snap1 snapshot create: failed: No description provided Usage: snapshot create <volnames> [-n <snap-name/cg-name>] [-d <description>] Change-Id: I74b5a8406d72282fbb7ba7d07e0c7fe395148d38 Signed-off-by: Avra Sengupta <asengupt@redhat.com>
* protocol/server: Implement barriering mechanisms for fopsshishir gowda2013-11-154-5/+468
| | | | | | | | | | | | | Barriering allows cbk's responses to be held back until either a timeout or a un-barrier event is received. The event is triggered by a brick-op. barrier-timeout, barrier-queue-lenght, and barrier-fops are currently exposed as a xlator-options only Change-Id: Ifb62d5b2bd2fbb35aca3b537e534156c877ce3a6 Signed-off-by: shishir gowda <sgowda@redhat.com>
* glusterd/utils: Get brick mount's device nameshishir gowda2013-11-152-0/+47
| | | | | Change-Id: I03ff9e8094e7e36b28b521380949c7e9044c2e4e Signed-off-by: shishir gowda <sgowda@redhat.com>
* mgmt/glusterd: Introduce snapshot infrastructureshishir gowda2013-11-1518-190/+2329
| | | | | | | | API's for creating, adding, finding, removing snapshots and consistency groups are provided. Change-Id: Ic28da69a075b062aefdf14754c68259ca58bd427 Signed-off-by: shishir gowda <sgowda@redhat.com>
* glusterd: Volume locks and transaction specific opinfosAvra Sengupta2013-11-151-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | With this patch we are replacing the existing cluster-wide lock taken on glusterds across the cluster, with volume locks which are also taken on glusterds across the cluster, but are volume specific. So with the volume locks we are able to perform more than one gluster operation at the same time, as long as the operations are being performed on different volumes. We maintain a global list of volume-locks (using a dict for a list) where the key is the volume name, and which saves the uuid of the originator glusterd. These locks are held and released per volume transaction. In order to acheive multiple gluster operations occuring at the same time, we also separate opinfos in the op-state-machine, as a part of this patch. To do so, we generate a unique transaction-id (uuid) per gluster transaction. An opinfo is then associated with this transaction id, which is used throughout the transaction. We maintain a run-time global list(using a dict) of transaction-ids, and their respective opinfos to achieve this. Change-Id: Iaad505a854bac8de8f83beec0357eb6cde3f7ea8 BUG: 1011470 Signed-off-by: Avra Sengupta <asengupt@redhat.com>
* fuse: Check the return status from state->resolve_nowupstreamVijaykumar M2013-11-142-7/+49
| | | | | | | | | Change-Id: I85fc6dd393449d365bb908b38c2827b58cb08171 BUG: 1030208 Signed-off-by: Vijaykumar M <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/6262 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gNFS: RFE for NFS connection behaviorSantosh Kumar Pradhan2013-11-1424-159/+1041
| | | | | | | | | | | | | | | Implement reconfigure() for NFS xlator so that volume set/reset wont restart the NFS server process. But few options can not be reconfigured dynamically e.g. nfs.mem-factor, nfs.port etc which needs NFS to be restarted. Change-Id: Ic586fd55b7933c0a3175708d8c41ed0475d74a1c BUG: 1027409 Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com> Reviewed-on: http://review.gluster.org/6236 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gfapi: Closed the logfile fd in glfs_finiPoornima2013-11-141-0/+3
| | | | | | | | | | | | | | | | The logfile fd is not closed even after calling glfs_fini, hence in smb mount if connection to glusterfs volume fails at a point after the log file was opened, the fd would remain open until the process dies. This patch closes the logfile fd in glfs_fini. Change-Id: I608bfac9c6833b42750b0383ad26fd33ee378ee1 BUG: 1030228 Signed-off-by: Poornima <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/6263 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* Transparent data encryption and metadata authenticationEdward Shishkin2013-11-1315-12/+8408
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward@redhat.com> Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cluster/dht - rebalance: handle the rebalance @ inode level (!fd level)Amar Tumballi2013-11-134-128/+148
| | | | | | | | | | | | | * migrate all the fd's on an inode to newer subvol after rebalance * use the migration in progress flag in inode, so all the operations on the inode can make use of it Change-Id: Ib807a46e927a1062688fc15119c916797c52a350 BUG: 1013456 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: http://review.gluster.org/5891 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* libglusterfs/inode: introduce new APIs for ctx handlingAmar Tumballi2013-11-133-21/+265
| | | | | | | | | | | | | | * inode_ctx_reset{0,1,2}() for reseting value1, value2, and both respectively * inode_ctx_get0() - to get the first value only * inode_ctx_set0() - to set the first value only * inode_ctx_get1() - to get the second value only * inode_ctx_set1() - to set the second value only Change-Id: I4dfbdac81d6a3f4e5784e060c76edabb1692ce03 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: http://review.gluster.org/5890 Reviewed-by: Anand Avati <avati@redhat.com> Tested-by: Anand Avati <avati@redhat.com>
* bd: Add test case for bd xlatorM. Mohan Kumar2013-11-131-0/+131
| | | | | | | | | Change-Id: I73a0bfa7085d2e71b2489687fa53f5fe7d1e8ea1 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/6050 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* bd: Add support to create clone, snapshot and merge of LV images.M. Mohan Kumar2013-11-137-56/+654
| | | | | | | | | | | | | | | | | | | | | | | | | | Special xattr names "clone" & "snapshot" can be used to create full and linked clone of the LV images. GFID of destination posix file (to be mapped) is passed as a value to the xattr. Destination posix file must exist before running this operation. These operations form a basis for offloading storage related operations from QEMU to GlusterFS. Syntax for full clone: xattr name: "clone" value: "gfid-of-dest-file" Syntax for linked clone: xattr name: "snapshot" value: "gfid-of-dest-file" Syntax for merging: xattr name: "merge" value: "path-to-snapshot-file" Example: setfattr -n clone -v <gfid-of-dest-file> /media/source setfattr -n snapshot -v <gfid-of-dest-file> /media/source setfattr -n merge -v "/media/sn" /media/sn Change-Id: Id9f984a709d4c2e52a64ae75bb12a8ecb01f8776 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/5626 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* bd: Add aio support to BD xlatorM. Mohan Kumar2013-11-137-24/+651
| | | | | | | | | | | | Volume option bd-aio controls AIO feature for BD xlator. Code taken from posix-aio.c Change-Id: Ib049bd59c9d3f9101d33939838322cfa808de053 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/5748 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* bd: Add BD support to other xlatorsM. Mohan Kumar2013-11-133-26/+67
| | | | | | | | | | | | | | | | Make changes to distributed xlator to work with BD xlator. Unlike files, a block device can't be removed when its opened. So some part of the code were moved down to avoid this situation. Also before truncating a BD file its BD_XATTR should be set otherwise truncate will result in truncating posix file. So file is created with needed BD_XATTR and truncate is invoked. Also enables BD xlator in stripe volume type. Change-Id: If127516e261fac5fc5b137e7fe33e100bc92acc0 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/5235 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* bd: posix/multi-brick support to BD xlatorM. Mohan Kumar2013-11-1321-3/+3289
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current BD xlator (block backend) has a few limitations such as * Creation of directories not supported * Supports only single brick * Does not use extended attributes (and client gfid) like posix xlator * Creation of special files (symbolic links, device nodes etc) not supported Basic limitation of not allowing directory creation is blocking oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM creates multi-level directories when GlusterFS is used as storage backend for storing VM images. To overcome these limitations a new BD xlator with following improvements is suggested. * New hybrid BD xlator that handles both regular files and block device files * The volume will have both POSIX and BD bricks. Regular files are created on POSIX bricks, block devices are created on the BD brick (VG) * BD xlator leverages exiting POSIX xlator for most POSIX calls and hence sits above the POSIX xlator * Block device file is differentiated from regular file by an extended attribute * The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a posix file to Logical Volume (LV). * When a client sends a request to set BD_XATTR on a posix file, a new LV is created and mapped to posix file. So every block device will have a representative file in POSIX brick with 'user.glusterfs.bd' (BD_XATTR) set. * Here after all operations on this file results in LV related operations. For example opening a file that has BD_XATTR set results in opening the LV block device, reading results in reading the corresponding LV block device. When BD xlator gets request to set BD_XATTR via setxattr call, it creates a LV and information about this LV is placed in the xattr of the posix file. xattr "user.glusterfs.bd" used to identify that posix file is mapped to BD. Usage: Server side: [root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2 It creates a distributed gluster volume 'bdvol' with Volume Group vg1 using posix brick /storage/vg1_info in host1 and Volume Group vg2 using /storage/vg2_info in host2. [root@host1 ~]# gluster volume start bdvol Client side: [root@node ~]# mount -t glusterfs host1:/bdvol /media [root@node ~]# touch /media/posix It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# mkdir /media/image [root@node ~]# touch /media/image/lv1 It also creates regular posix file 'lv1' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1 [root@node ~]# Above setxattr results in creating a new LV in corresponding brick's VG and it sets 'user.glusterfs.bd' with value 'lv:<default-extent-size' [root@node ~]# truncate -s5G /media/image/lv1 It results in resizig LV 'lv1'to 5G New BD xlator code is placed in xlators/storage/bd directory. Also add volume-uuid to the VG so that same VG can't be used for other bricks/volumes. After deleting a gluster volume, one has to manually remove the associated tag using vgchange <vg-name> --deltag <trusted.glusterfs.volume-id:<volume-id>> Changes from previous version V5: * Removed support for delayed deleting of LVs Changes from previous version V4: * Consolidated the patches * Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR. Changes from previous version V3: * Added support in FUSE to support full/linked clone * Added support to merge snapshots and provide information about origin * bd_map xlator removed * iatt structure used in inode_ctx. iatt is cached and updated during fsync/flush * aio support * Type and capabilities of volume are exported through getxattr Changes from version 2: * Used inode_context for caching BD size and to check if loc/fd is BD or not. * Added GlusterFS server offloaded copy and snapshot through setfattr FOP. As part of this libgfapi is modified. * BD xlator supports stripe * During unlinking if a LV file is already opened, its added to delete list and bd_del_thread tries to delete from this list when a last reference to that file is closed. Changes from previous version: * gfid is used as name of LV * ? is used to specify VG name for creating BD volume in volume create, add-brick. gluster volume create volname host:/path?vg * open-behind issue is fixed * A replicate brick can be added dynamically and LVs from source brick are replicated to destination brick * A distribute brick can be added dynamically and rebalance operation distributes existing LVs/files to the new brick * Thin provisioning support added. * bd_map xlator support retained * setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and setfattr -n user.glusterfs.bd -v "thin" creates thin LV * Capability and backend information added to gluster volume info (and --xml) so that management tools can exploit BD xlator. * tracing support for bd xlator added TODO: * Add support to display snapshots for a given LV * Display posix filename for list-origin instead of gfid Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/4809 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* bd_map: Remove bd_map xlatorM. Mohan Kumar2013-11-1335-4602/+20
| | | | | | | | | | | Remove bd_map xlator and CLI related changes. Change-Id: If7086205df1907127c1a1fa4ba603f1c48421d09 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/5747 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gfapi: introduce glfs_readdir() and glfs_readdirplus() APIsAnand Avati2013-11-135-2/+96
| | | | | | | | | Change-Id: I6b233bf647585675f233898351bf593f251716cc BUG: 839950 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/6201 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-28 17:28:45 +0000