diff options
Diffstat (limited to 'doc/admin-guide/en-US/markdown/admin_UFO.md')
| -rw-r--r-- | doc/admin-guide/en-US/markdown/admin_UFO.md | 1219 |
1 files changed, 1219 insertions, 0 deletions
diff --git a/doc/admin-guide/en-US/markdown/admin_UFO.md b/doc/admin-guide/en-US/markdown/admin_UFO.md new file mode 100644 index 00000000000..ed65d5ee794 --- /dev/null +++ b/doc/admin-guide/en-US/markdown/admin_UFO.md @@ -0,0 +1,1219 @@ +Managing Unified File and Object Storage +======================================== + +Unified File and Object Storage (UFO) unifies NAS and object storage +technology. It provides a system for data storage that enables users to +access the same data, both as an object and as a file, thus simplifying +management and controlling storage costs. + +Unified File and Object Storage is built upon Openstack's Object Storage +Swift. Open Stack Object Storage allows users to store and retrieve +files and content through a simple Web Service (REST: Representational +State Transfer) interface as objects and GlusterFS, allows users to +store and retrieve files using Native Fuse and NFS mounts. It uses +GlusterFS as a backend file system for Open Stack Swift. It also +leverages on Open Stack Swift's web interface for storing and retrieving +files over the web combined with GlusterFS features like scalability and +high availability, replication, elastic volume management for data +management at disk level. + +Unified File and Object Storage technology enables enterprises to adopt +and deploy cloud storage solutions. It allows users to access and modify +data as objects from a REST interface along with the ability to access +and modify files from NAS interfaces including NFS and CIFS. In addition +to decreasing cost and making it faster and easier to access object +data, it also delivers massive scalability, high availability and +replication of object storage. Infrastructure as a Service (IaaS) +providers can utilize GlusterFS Unified File and Object Storage +technology to enable their own cloud storage service. Enterprises can +use this technology to accelerate the process of preparing file-based +applications for the cloud and simplify new application development for +cloud computing environments. + +OpenStack Object Storage is scalable object storage system and it is not +a traditional file system. You will not be able to mount this system +like traditional SAN or NAS volumes and perform POSIX compliant +operations. + +Components of Object Storage +============================ + +The major components of Object Storage are: + +**Proxy Server** + +All REST requests to the UFO are routed through the Proxy Server. + +**Objects and Containers** + +An object is the basic storage entity and any optional metadata that +represents the data you store. When you upload data, the data is stored +as-is (with no compression or encryption). + +A container is a storage compartment for your data and provides a way +for you to organize your data. Containers can be visualized as +directories in a Linux system. Data must be stored in a container and +hence objects are created within a container. + +It implements objects as files and directories under the container. The +object name is a '/' separated path and UFO maps it to directories until +the last name in the path, which is marked as a file. With this +approach, objects can be accessed as files and directories from native +GlusterFS (FUSE) or NFS mounts by providing the '/' separated path. + +**Accounts and Account Servers** + +The OpenStack Object Storage system is designed to be used by many +different storage consumers. Each user is associated with one or more +accounts and must identify themselves using an authentication system. +While authenticating, users must provide the name of the account for +which the authentication is requested. + +UFO implements accounts as GlusterFS volumes. So, when a user is granted +read/write permission on an account, it means that that user has access +to all the data available on that GlusterFS volume. + +**Authentication and Access Permissions** + +You must authenticate against an authentication service to receive +OpenStack Object Storage connection parameters and an authentication +token. The token must be passed in for all subsequent container or +object operations. One authentication service that you can use as a +middleware example is called `tempauth`. + +By default, each user has their own storage account and has full access +to that account. Users must authenticate with their credentials as +described above, but once authenticated they can manage containers and +objects within that account. If a user wants to access the content from +another account, they must have API access key or a session token +provided by their authentication system. + +Advantages of using GlusterFS Unified File and Object Storage +============================================================= + +The following are the advantages of using GlusterFS UFO: + +- No limit on upload and download files sizes as compared to Open + Stack Swift which limits the object size to 5GB. + +- A unified view of data across NAS and Object Storage technologies. + +- Using GlusterFS's UFO has other advantages like the following: + + - High availability + + - Scalability + + - Replication + + - Elastic Volume management + +Preparing to Deploy Unified File and Object Storage +=================================================== + +This section provides information on pre-requisites and list of +dependencies that will be installed during the installation of Unified +File and Object Storage. + +Pre-requisites +-------------- + +GlusterFS's Unified File and Object Storage needs `user_xattr` support +from the underlying disk file system. Use the following command to +enable `user_xattr` for GlusterFS bricks backend: + +`# mount –o remount,user_xattr ` + +For example, + +`# mount –o remount,user_xattr /dev/hda1 ` + +Dependencies +------------ + +The following packages are installed on GlusterFS when you install +Unified File and Object Storage: + +- curl + +- memcached + +- openssl + +- xfsprogs + +- python2.6 + +- pyxattr + +- python-configobj + +- python-setuptools + +- python-simplejson + +- python-webob + +- python-eventlet + +- python-greenlet + +- python-pastedeploy + +- python-netifaces + +Installing and Configuring Unified File and Object Storage +========================================================== + +This section provides instructions on how to install and configure +Unified File and Object Storage in your storage environment. + +Installing Unified File and Object Storage +------------------------------------------ + +To install Unified File and Object Storage: + +1. Download `rhel_install.sh` install script from [][] . + +2. Run `rhel_install.sh` script using the following command: + + `# sh rhel_install.sh` + +3. Download `swift-1.4.5-1.noarch.rpm` and + `swift-plugin-1.0.-1.el6.noarch.rpm` files from [][]. + +4. Install `swift-1.4.5-1.noarch.rpm` and + `swift-plugin-1.0.-1.el6.noarch.rpm` using the following commands: + + `# rpm -ivh swift-1.4.5-1.noarch.rpm` + + `# rpm -ivh swift-plugin-1.0.-1.el6.noarch.rpm` + + > **Note** + > + > You must repeat the above steps on all the machines on which you + > want to install Unified File and Object Storage. If you install + > the Unified File and Object Storage on multiple servers, you can + > use a load balancer like pound, nginx, and so on to distribute the + > request across the machines. + +Adding Users +------------ + +The authentication system allows the administrator to grant different +levels of access to different users based on the requirement. The +following are the types of user permissions: + +- admin user + +- normal user + +Admin user has read and write permissions on the account. By default, a +normal user has no read or write permissions. A normal user can only +authenticate itself to get a Auth-Token. Read or write permission are +provided through ACLs by the admin users. + +Add a new user by adding the following entry in +`/etc/swift/proxy-server.conf` file: + +`user_<account-name>_<user-name> = <password> [.admin]` + +For example, + +`user_test_tester = testing .admin` + +> **Note** +> +> During installation, the installation script adds few sample users to +> the `proxy-server.conf` file. It is highly recommended that you remove +> all the default sample user entries from the configuration file. + +For more information on setting ACLs, see ?. + +Configuring Proxy Server +------------------------ + +The Proxy Server is responsible for connecting to the rest of the +OpenStack Object Storage architecture. For each request, it looks up the +location of the account, container, or object in the ring and route the +request accordingly. The public API is also exposed through the proxy +server. When objects are streamed to or from an object server, they are +streamed directly through the proxy server to or from the user – the +proxy server does not spool them. + +The configurable options pertaining to proxy server are stored in +`/etc/swift/proxy-server.conf`. The following is the sample +`proxy-server.conf` file: + + [app:proxy-server] + use = egg:swift#proxy + allow_account_management=true + account_autocreate=true + + [filter:tempauth] + use = egg:swift#tempauth user_admin_admin=admin.admin.reseller_admin + user_test_tester=testing.admin + user_test2_tester2=testing2.admin + user_test_tester3=testing3 + + [filter:healthcheck] + use = egg:swift#healthcheck + + [filter:cache] + use = egg:swift#memcache + +By default, GlusterFS's Unified File and Object Storage is configured to +support HTTP protocol and uses temporary authentication to authenticate +the HTTP requests. + +Configuring Authentication System +--------------------------------- + +Proxy server must be configured to authenticate using ` + + `. + +Configuring Proxy Server for HTTPS +---------------------------------- + +By default, proxy server only handles HTTP request. To configure the +proxy server to process HTTPS requests, perform the following steps: + +1. Create self-signed cert for SSL using the following commands: + + cd /etc/swift + openssl req -new -x509 -nodes -out cert.crt -keyout cert.key + +2. Add the following lines to `/etc/swift/proxy-server.conf `under + [DEFAULT] + + bind_port = 443 + cert_file = /etc/swift/cert.crt + key_file = /etc/swift/cert.key + +3. Restart the servers using the following commands: + + swift-init main stop + swift-init main start + +The following are the configurable options: + + Option Default Description + ------------ ------------ ------------------------------- + bind\_ip 0.0.0.0 IP Address for server to bind + bind\_port 80 Port for server to bind + swift\_dir /etc/swift Swift configuration directory + workers 1 Number of workers to fork + user swift swift user + cert\_file Path to the ssl .crt + key\_file Path to the ssl .key + + : proxy-server.conf Default Options in the [DEFAULT] section + + Option Default Description + ------------------------------- ----------------- ----------------------------------------------------------------------------------------------------------- + use paste.deploy entry point for the container server. For most cases, this should be `egg:swift#container`. + log\_name proxy-server Label used when logging + log\_facility LOG\_LOCAL0 Syslog log facility + log\_level INFO Log level + log\_headers True If True, log headers in each request + recheck\_account\_existence 60 Cache timeout in seconds to send memcached for account existence + recheck\_container\_existence 60 Cache timeout in seconds to send memcached for container existence + object\_chunk\_size 65536 Chunk size to read from object servers + client\_chunk\_size 65536 Chunk size to read from clients + memcache\_servers 127.0.0.1:11211 Comma separated list of memcached servers ip:port + node\_timeout 10 Request timeout to external services + client\_timeout 60 Timeout to read one chunk from a client + conn\_timeout 0.5 Connection timeout to external services + error\_suppression\_interval 60 Time in seconds that must elapse since the last error for a node to be considered no longer error limited + error\_suppression\_limit 10 Error count to consider a node error limited + allow\_account\_management false Whether account `PUT`s and `DELETE`s are even callable + + : proxy-server.conf Server Options in the [proxy-server] section + +Configuring Object Server +------------------------- + +The Object Server is a very simple blob storage server that can store, +retrieve, and delete objects stored on local devices. Objects are stored +as binary files on the file system with metadata stored in the file’s +extended attributes (xattrs). This requires that the underlying file +system choice for object servers support xattrs on files. + +The configurable options pertaining Object Server are stored in the file +`/etc/swift/object-server/1.conf`. The following is the sample +`object-server/1.conf` file: + + [DEFAULT] + devices = /srv/1/node + mount_check = false + bind_port = 6010 + user = root + log_facility = LOG_LOCAL2 + + [pipeline:main] + pipeline = gluster object-server + + [app:object-server] + use = egg:swift#object + + [filter:gluster] + use = egg:swift#gluster + + [object-replicator] + vm_test_mode = yes + + [object-updater] + [object-auditor] + +The following are the configurable options: + + Option Default Description + -------------- ------------ ---------------------------------------------------------------------------------------------------- + swift\_dir /etc/swift Swift configuration directory + devices /srv/node Mount parent directory where devices are mounted + mount\_check true Whether or not check if the devices are mounted to prevent accidentally writing to the root device + bind\_ip 0.0.0.0 IP Address for server to bind + bind\_port 6000 Port for server to bind + workers 1 Number of workers to fork + + : object-server.conf Default Options in the [DEFAULT] section + + Option Default Description + ---------------------- --------------- ---------------------------------------------------------------------------------------------------- + use paste.deploy entry point for the object server. For most cases, this should be `egg:swift#object`. + log\_name object-server log name used when logging + log\_facility LOG\_LOCAL0 Syslog log facility + log\_level INFO Logging level + log\_requests True Whether or not to log each request + user swift swift user + node\_timeout 3 Request timeout to external services + conn\_timeout 0.5 Connection timeout to external services + network\_chunk\_size 65536 Size of chunks to read or write over the network + disk\_chunk\_size 65536 Size of chunks to read or write to disk + max\_upload\_time 65536 Maximum time allowed to upload an object + slow 0 If \> 0, Minimum time in seconds for a `PUT` or `DELETE` request to complete + + : object-server.conf Server Options in the [object-server] section + +Configuring Container Server +---------------------------- + +The Container Server’s primary job is to handle listings of objects. The +listing is done by querying the GlusterFS mount point with path. This +query returns a list of all files and directories present under that +container. + +The configurable options pertaining to container server are stored in +`/etc/swift/container-server/1.conf` file. The following is the sample +`container-server/1.conf` file: + + [DEFAULT] + devices = /srv/1/node + mount_check = false + bind_port = 6011 + user = root + log_facility = LOG_LOCAL2 + + [pipeline:main] + pipeline = gluster container-server + + [app:container-server] + use = egg:swift#container + + [filter:gluster] + use = egg:swift#gluster + + [container-replicator] + [container-updater] + [container-auditor] + +The following are the configurable options: + + Option Default Description + -------------- ------------ ---------------------------------------------------------------------------------------------------- + swift\_dir /etc/swift Swift configuration directory + devices /srv/node Mount parent directory where devices are mounted + mount\_check true Whether or not check if the devices are mounted to prevent accidentally writing to the root device + bind\_ip 0.0.0.0 IP Address for server to bind + bind\_port 6001 Port for server to bind + workers 1 Number of workers to fork + user swift Swift user + + : container-server.conf Default Options in the [DEFAULT] section + + Option Default Description + --------------- ------------------ ---------------------------------------------------------------------------------------------------------- + use paste.deploy entry point for the container server. For most cases, this should be `egg:swift#container`. + log\_name container-server Label used when logging + log\_facility LOG\_LOCAL0 Syslog log facility + log\_level INFO Logging level + node\_timeout 3 Request timeout to external services + conn\_timeout 0.5 Connection timeout to external services + + : container-server.conf Server Options in the [container-server] + section + +Configuring Account Server +-------------------------- + +The Account Server is very similar to the Container Server, except that +it is responsible for listing of containers rather than objects. In UFO, +each gluster volume is an account. + +The configurable options pertaining to account server are stored in +`/etc/swift/account-server/1.conf` file. The following is the sample +`account-server/1.conf` file: + + [DEFAULT] + devices = /srv/1/node + mount_check = false + bind_port = 6012 + user = root + log_facility = LOG_LOCAL2 + + [pipeline:main] + pipeline = gluster account-server + + [app:account-server] + use = egg:swift#account + + [filter:gluster] + use = egg:swift#gluster + + [account-replicator] + vm_test_mode = yes + + [account-auditor] + [account-reaper] + +The following are the configurable options: + + Option Default Description + -------------- ------------ ---------------------------------------------------------------------------------------------------- + swift\_dir /etc/swift Swift configuration directory + devices /srv/node mount parent directory where devices are mounted + mount\_check true Whether or not check if the devices are mounted to prevent accidentally writing to the root device + bind\_ip 0.0.0.0 IP Address for server to bind + bind\_port 6002 Port for server to bind + workers 1 Number of workers to fork + user swift Swift user + + : account-server.conf Default Options in the [DEFAULT] section + + Option Default Description + --------------- ---------------- ---------------------------------------------------------------------------------------------------------- + use paste.deploy entry point for the container server. For most cases, this should be `egg:swift#container`. + log\_name account-server Label used when logging + log\_facility LOG\_LOCAL0 Syslog log facility + log\_level INFO Logging level + + : account-server.conf Server Options in the [account-server] section + +Starting and Stopping Server +---------------------------- + +You must start the server manually when system reboots and whenever you +update/modify the configuration files. + +- To start the server, enter the following command: + + `# swift_init main start` + +- To stop the server, enter the following command: + + `# swift_init main stop` + +Working with Unified File and Object Storage +============================================ + +This section describes the REST API for administering and managing +Object Storage. All requests will be directed to the host and URL +described in the `X-Storage-URL HTTP` header obtained during successful +authentication. + +Configuring Authenticated Access +-------------------------------- + +Authentication is the process of proving identity to the system. To use +the REST interface, you must obtain an authorization token using GET +method and supply it with v1.0 as the path. + +Each REST request against the Object Storage system requires the +addition of a specific authorization token HTTP x-header, defined as +X-Auth-Token. The storage URL and authentication token are returned in +the headers of the response. + +- To authenticate, run the following command: + + GET auth/v1.0 HTTP/1.1 + Host: <auth URL> + X-Auth-User: <account name>:<user name> + X-Auth-Key: <user-Password> + + For example, + + GET auth/v1.0 HTTP/1.1 + Host: auth.example.com + X-Auth-User: test:tester + X-Auth-Key: testing + + HTTP/1.1 200 OK + X-Storage-Url: https:/example.storage.com:443/v1/AUTH_test + X-Storage-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554 + X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554 + Content-Length: 0 + Date: Wed, 10 jul 2011 06:11:51 GMT + + To authenticate access using cURL (for the above example), run the + following command: + + curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass:testing' -k + https://auth.example.com:443/auth/v1.0 + + The X-Auth-Url has to be parsed and used in the connection and + request line of all subsequent requests to the server. In the + example output, users connecting to server will send most + container/object requests with a host header of example.storage.com + and the request line's version and account as v1/AUTH\_test. + +> **Note** +> +> The authentication tokens are valid for a 24 hour period. + +Working with Accounts +--------------------- + +This section describes the list of operations you can perform at the +account level of the URL. + +### Displaying Container Information + +You can list the objects of a specific container, or all containers, as +needed using GET command. You can use the following optional parameters +with GET request to refine the results: + + Parameter Description + ----------- -------------------------------------------------------------------------- + limit Limits the number of results to at most *n* value. + marker Returns object names greater in value than the specified marker. + format Specify either json or xml to return the respective serialized response. + +**To display container information** + +- List all the containers of an account using the following command: + + GET /<apiversion>/<account> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + + For example, + + GET /v1/AUTH_test HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 200 Ok + Date: Wed, 13 Jul 2011 16:32:21 GMT + Server: Apache + Content-Type: text/plain; charset=UTF-8 + Content-Length: 39 + + songs + movies + documents + reports + +To display container information using cURL (for the above example), run +the following command: + + curl -v -X GET -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test -k + +### Displaying Account Metadata Information + +You can issue HEAD command to the storage service to view the number of +containers and the total bytes stored in the account. + +- To display containers and storage used, run the following command: + + HEAD /<apiversion>/<account> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + + For example, + + HEAD /v1/AUTH_test HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 204 No Content + Date: Wed, 13 Jul 2011 16:52:21 GMT + Server: Apache + X-Account-Container-Count: 4 + X-Account-Total-Bytes-Used: 394792 + + To display account metadata information using cURL (for the above + example), run the following command: + + curl -v -X HEAD -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test -k + +Working with Containers +----------------------- + +This section describes the list of operations you can perform at the +container level of the URL. + +### Creating Containers + +You can use PUT command to create containers. Containers are the storage +folders for your data. The URL encoded name must be less than 256 bytes +and cannot contain a forward slash '/' character. + +- To create a container, run the following command: + + PUT /<apiversion>/<account>/<container>/ HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + + For example, + + PUT /v1/AUTH_test/pictures/ HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + HTTP/1.1 201 Created + + Date: Wed, 13 Jul 2011 17:32:21 GMT + Server: Apache + Content-Type: text/plain; charset=UTF-8 + + To create container using cURL (for the above example), run the + following command: + + curl -v -X PUT -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/pictures -k + + The status code of 201 (Created) indicates that you have + successfully created the container. If a container with same is + already existed, the status code of 202 is displayed. + +### Displaying Objects of a Container + +You can list the objects of a container using GET command. You can use +the following optional parameters with GET request to refine the +results: + + Parameter Description + ----------- -------------------------------------------------------------------------------------------------------------- + limit Limits the number of results to at most *n* value. + marker Returns object names greater in value than the specified marker. + prefix Displays the results limited to object names beginning with the substring x. beginning with the substring x. + path Returns the object names nested in the pseudo path. + format Specify either json or xml to return the respective serialized response. + delimiter Returns all the object names nested in the container. + +To display objects of a container + +- List objects of a specific container using the following command: + +<!-- --> + + GET /<apiversion>/<account>/<container>[parm=value] HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + +For example, + + GET /v1/AUTH_test/images HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 200 Ok + Date: Wed, 13 Jul 2011 15:42:21 GMT + Server: Apache + Content-Type: text/plain; charset=UTF-8 + Content-Length: 139 + + sample file.jpg + test-file.pdf + You and Me.pdf + Puddle of Mudd.mp3 + Test Reports.doc + +To display objects of a container using cURL (for the above example), +run the following command: + + curl -v -X GET-H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images -k + +### Displaying Container Metadata Information + +You can issue HEAD command to the storage service to view the number of +objects in a container and the total bytes of all the objects stored in +the container. + +- To display list of objects and storage used, run the following + command: + + HEAD /<apiversion>/<account>/<container> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + + For example, + + HEAD /v1/AUTH_test/images HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 204 No Content + Date: Wed, 13 Jul 2011 19:52:21 GMT + Server: Apache + X-Account-Object-Count: 8 + X-Container-Bytes-Used: 472 + + To display list of objects and storage used in a container using + cURL (for the above example), run the following command: + + curl -v -X HEAD -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images -k + +### Deleting Container + +You can use DELETE command to permanently delete containers. The +container must be empty before it can be deleted. + +You can issue HEAD command to determine if it contains any objects. + +- To delete a container, run the following command: + + DELETE /<apiversion>/<account>/<container>/ HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + + For example, + + DELETE /v1/AUTH_test/pictures HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 204 No Content + Date: Wed, 13 Jul 2011 17:52:21 GMT + Server: Apache + Content-Length: 0 + Content-Type: text/plain; charset=UTF-8 + + To delete a container using cURL (for the above example), run the + following command: + + curl -v -X DELETE -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/pictures -k + + The status code of 204 (No Content) indicates that you have + successfully deleted the container. If that container does not + exist, the status code 404 (Not Found) is displayed, and if the + container is not empty, the status code 409 (Conflict) is displayed. + +### Updating Container Metadata + +You can update the metadata of container using POST operation, metadata +keys should be prefixed with 'x-container-meta'. + +- To update the metadata of the object, run the following command: + + POST /<apiversion>/<account>/<container> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <Authentication-token-key> + X-Container-Meta-<key>: <new value> + X-Container-Meta-<key>: <new value> + + For example, + + POST /v1/AUTH_test/images HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + X-Container-Meta-Zoo: Lion + X-Container-Meta-Home: Dog + + HTTP/1.1 204 No Content + Date: Wed, 13 Jul 2011 20:52:21 GMT + Server: Apache + Content-Type: text/plain; charset=UTF-8 + + To update the metadata of the object using cURL (for the above + example), run the following command: + + curl -v -X POST -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images -H ' X-Container-Meta-Zoo: Lion' -H 'X-Container-Meta-Home: Dog' -k + + The status code of 204 (No Content) indicates the container's + metadata is updated successfully. If that object does not exist, the + status code 404 (Not Found) is displayed. + +### Setting ACLs on Container + +You can set the container access control list by using POST command on +container with `x- container-read` and` x-container-write` keys. + +The ACL format is `[item[,item...]]`. Each item can be a group name to +give access to or a referrer designation to grant or deny based on the +HTTP Referer header. + +The referrer designation format is:` .r:[-]value`. + +The .r can also be `.ref, .referer, `or .`referrer`; though it will be +shortened to.r for decreased character count usage. The value can be `*` +to specify any referrer host is allowed access. The leading minus sign +(-) indicates referrer hosts that should be denied access. + +Examples of valid ACLs: + + .r:* + .r:*,bobs_account,sues_account:sue + bobs_account,sues_account:sue + +Examples of invalid ACLs: + + .r: + .r:- + +By default, allowing read access via `r `will not allow listing objects +in the container but allows retrieving objects from the container. To +turn on listings, use the .`rlistings` directive. Also, `.r` +designations are not allowed in headers whose names include the word +write. + +For example, to set all the objects access rights to "public‟ inside the +container using cURL (for the above example), run the following command: + + curl -v -X POST -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images + -H 'X-Container-Read: .r:*' -k + +Working with Objects +-------------------- + +An object represents the data and any metadata for the files stored in +the system. Through the REST interface, metadata for an object can be +included by adding custom HTTP headers to the request and the data +payload as the request body. Objects name should not exceed 1024 bytes +after URL encoding. + +This section describes the list of operations you can perform at the +object level of the URL. + +### Creating or Updating Object + +You can use PUT command to write or update an object's content and +metadata. + +You can verify the data integrity by including an MD5checksum for the +object's data in the ETag header. ETag header is optional and can be +used to ensure that the object's contents are stored successfully in the +storage system. + +You can assign custom metadata to objects by including additional HTTP +headers on the PUT request. The objects created with custom metadata via +HTTP headers are identified with the`X-Object- Meta`- prefix. + +- To create or update an object, run the following command: + + PUT /<apiversion>/<account>/<container>/<object> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + ETag: da1e100dc9e7becc810986e37875ae38 + Content-Length: 342909 + X-Object-Meta-PIN: 2343 + + For example, + + PUT /v1/AUTH_test/pictures/dog HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + ETag: da1e100dc9e7becc810986e37875ae38 + + HTTP/1.1 201 Created + Date: Wed, 13 Jul 2011 18:32:21 GMT + Server: Apache + ETag: da1e100dc9e7becc810986e37875ae38 + Content-Length: 0 + Content-Type: text/plain; charset=UTF-8 + + To create or update an object using cURL (for the above example), + run the following command: + + curl -v -X PUT -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/pictures/dog -H 'Content- + Length: 0' -k + + The status code of 201 (Created) indicates that you have + successfully created or updated the object. If there is a missing + content-Length or Content-Type header in the request, the status + code of 412 (Length Required) is displayed. (Optionally) If the MD5 + checksum of the data written to the storage system does not match + the ETag value, the status code of 422 (Unprocessable Entity) is + displayed. + +#### Chunked Transfer Encoding + +You can upload data without knowing the size of the data to be uploaded. +You can do this by specifying an HTTP header of Transfer-Encoding: +chunked and without using a Content-Length header. + +You can use this feature while doing a DB dump, piping the output +through gzip, and then piping the data directly into Object Storage +without having to buffer the data to disk to compute the file size. + +- To create or update an object, run the following command: + + PUT /<apiversion>/<account>/<container>/<object> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <authentication-token-key> + Transfer-Encoding: chunked + X-Object-Meta-PIN: 2343 + + For example, + + PUT /v1/AUTH_test/pictures/cat HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + Transfer-Encoding: chunked + X-Object-Meta-PIN: 2343 + 19 + A bunch of data broken up + D + into chunks. + 0 + +### Copying Object + +You can copy object from one container to another or add a new object +and then add reference to designate the source of the data from another +container. + +**To copy object from one container to another** + +- To add a new object and designate the source of the data from + another container, run the following command: + + COPY /<apiversion>/<account>/<container>/<sourceobject> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: < authentication-token-key> + Destination: /<container>/<destinationobject> + + For example, + + COPY /v1/AUTH_test/images/dogs HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + Destination: /photos/cats + + HTTP/1.1 201 Created + Date: Wed, 13 Jul 2011 18:32:21 GMT + Server: Apache + Content-Length: 0 + Content-Type: text/plain; charset=UTF-8 + + To copy an object using cURL (for the above example), run the + following command: + + curl -v -X COPY -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' -H 'Destination: /photos/cats' -k https://example.storage.com:443/v1/AUTH_test/images/dogs + + The status code of 201 (Created) indicates that you have + successfully copied the object. If there is a missing content-Length + or Content-Type header in the request, the status code of 412 + (Length Required) is displayed. + + You can also use PUT command to copy object by using additional + header `X-Copy-From: container/obj`. + +- To use PUT command to copy an object, run the following command: + + PUT /v1/AUTH_test/photos/cats HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + X-Copy-From: /images/dogs + + HTTP/1.1 201 Created + Date: Wed, 13 Jul 2011 18:32:21 GMT + Server: Apache + Content-Type: text/plain; charset=UTF-8 + + To copy an object using cURL (for the above example), run the + following command: + + curl -v -X PUT -H 'X-Auth-Token: AUTH_tkde3ad38b087b49bbbac0494f7600a554' + -H 'X-Copy-From: /images/dogs' –k + https://example.storage.com:443/v1/AUTH_test/images/cats + + The status code of 201 (Created) indicates that you have + successfully copied the object. + +### Displaying Object Information + +You can issue GET command on an object to view the object data of the +object. + +- To display the content of an object run the following command: + + GET /<apiversion>/<account>/<container>/<object> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <Authentication-token-key> + + For example, + + GET /v1/AUTH_test/images/cat HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 200 Ok + Date: Wed, 13 Jul 2011 23:52:21 GMT + Server: Apache + Last-Modified: Thu, 14 Jul 2011 13:40:18 GMT + ETag: 8a964ee2a5e88be344f36c22562a6486 + Content-Length: 534210 + [.........] + + To display the content of an object using cURL (for the above + example), run the following command: + + curl -v -X GET -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images/cat -k + + The status code of 200 (Ok) indicates the object‟s data is displayed + successfully. If that object does not exist, the status code 404 + (Not Found) is displayed. + +### Displaying Object Metadata + +You can issue HEAD command on an object to view the object metadata and +other standard HTTP headers. You must send only authorization token as +header. + +- To display the metadata of the object, run the following command: + +<!-- --> + + HEAD /<apiversion>/<account>/<container>/<object> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <Authentication-token-key> + +For example, + + HEAD /v1/AUTH_test/images/cat HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 204 No Content + Date: Wed, 13 Jul 2011 21:52:21 GMT + Server: Apache + Last-Modified: Thu, 14 Jul 2011 13:40:18 GMT + ETag: 8a964ee2a5e88be344f36c22562a6486 + Content-Length: 512000 + Content-Type: text/plain; charset=UTF-8 + X-Object-Meta-House: Cat + X-Object-Meta-Zoo: Cat + X-Object-Meta-Home: Cat + X-Object-Meta-Park: Cat + +To display the metadata of the object using cURL (for the above +example), run the following command: + + curl -v -X HEAD -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images/cat -k + +The status code of 204 (No Content) indicates the object‟s metadata is +displayed successfully. If that object does not exist, the status code +404 (Not Found) is displayed. + +### Updating Object Metadata + +You can issue POST command on an object name only to set or overwrite +arbitrary key metadata. You cannot change the object‟s other headers +such as Content-Type, ETag and others using POST operation. The POST +command will delete all the existing metadata and replace it with the +new arbitrary key metadata. + +You must prefix **X-Object-Meta-** to the key names. + +- To update the metadata of an object, run the following command: + + POST /<apiversion>/<account>/<container>/<object> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <Authentication-token-key> + X-Object-Meta-<key>: <new value> + X-Object-Meta-<key>: <new value> + + For example, + + POST /v1/AUTH_test/images/cat HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + X-Object-Meta-Zoo: Lion + X-Object-Meta-Home: Dog + + HTTP/1.1 202 Accepted + Date: Wed, 13 Jul 2011 22:52:21 GMT + Server: Apache + Content-Length: 0 + Content-Type: text/plain; charset=UTF-8 + + To update the metadata of an object using cURL (for the above + example), run the following command: + + curl -v -X POST -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/images/cat -H ' X-Object- + Meta-Zoo: Lion' -H 'X-Object-Meta-Home: Dog' -k + + The status code of 202 (Accepted) indicates that you have + successfully updated the object‟s metadata. If that object does not + exist, the status code 404 (Not Found) is displayed. + +### Deleting Object + +You can use DELETE command to permanently delete the object. + +The DELETE command on an object will be processed immediately and any +subsequent operations like GET, HEAD, POST, or DELETE on the object will +display 404 (Not Found) error. + +- To delete an object, run the following command: + + DELETE /<apiversion>/<account>/<container>/<object> HTTP/1.1 + Host: <storage URL> + X-Auth-Token: <Authentication-token-key> + + For example, + + DELETE /v1/AUTH_test/pictures/cat HTTP/1.1 + Host: example.storage.com + X-Auth-Token: AUTH_tkd3ad38b087b49bbbac0494f7600a554 + + HTTP/1.1 204 No Content + Date: Wed, 13 Jul 2011 20:52:21 GMT + Server: Apache + Content-Type: text/plain; charset=UTF-8 + + To delete an object using cURL (for the above example), run the + following command: + + curl -v -X DELETE -H 'X-Auth-Token: + AUTH_tkde3ad38b087b49bbbac0494f7600a554' + https://example.storage.com:443/v1/AUTH_test/pictures/cat -k + + The status code of 204 (No Content) indicates that you have + successfully deleted the object. If that object does not exist, the + status code 404 (Not Found) is displayed. + + []: http://download.gluster.com/pub/gluster/glusterfs/3.2/UFO/ |