From 8907f67ba215172b01a7018adcbb063fcc4570e9 Mon Sep 17 00:00:00 2001 From: Humble Devassy Chirammal Date: Mon, 30 Mar 2015 12:21:05 +0530 Subject: doc: restructure developer docs to new layout The developer oriented information is scattered in source and its very difficult to identify which are those. With this patch subdirs are created under developer-guide which will be the parent for developer notes. The changes suggested in http://review.gluster.org/#/c/8827/ are also included in this patch. Change-Id: I4c8510d52c49f4066225f72cac8f97f087d6c70c BUG: 1206539 Signed-off-by: Humble Devassy Chirammal Reviewed-on: http://review.gluster.org/10038 Tested-by: Gluster Build System Reviewed-by: Lalatendu Mohanty Reviewed-by: Kaleb KEITHLEY --- doc/developer-guide/network_compression.md | 71 ++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 doc/developer-guide/network_compression.md (limited to 'doc/developer-guide/network_compression.md') diff --git a/doc/developer-guide/network_compression.md b/doc/developer-guide/network_compression.md new file mode 100644 index 00000000000..7327591ef63 --- /dev/null +++ b/doc/developer-guide/network_compression.md @@ -0,0 +1,71 @@ +#On-Wire Compression + Decompression + +The 'compression translator' compresses and decompresses data in-flight +between client and bricks. + +###Working +When a writev call occurs, the client compresses the data before sending it to +brick. On the brick, compressed data is decompressed. Similarly, when a readv +call occurs, the brick compresses the data before sending it to client. On the +client, the compressed data is decompressed. Thus, the amount of data sent over +the wire is minimized. Compression/Decompression is done using Zlib library. + +During normal operation, this is the format of data sent over wire: + +~~~ + + trailer(8 bytes) +~~~ + +The trailer contains the CRC32 checksum and length of original uncompressed +data. This is used for validation. + +###Usage + +Turning on compression xlator: + +~~~ +gluster volume set network.compression on +~~~ + +###Configurable parameters (optional) + +**Compression level** +~~~ +gluster volume set network.compression.compression-level 8 +~~~ + +~~~ +0 : no compression +1 : best speed +9 : best compression +-1 : default compression +~~~ + +**Minimum file size** + +~~~ +gluster volume set network.compression.min-size 50 +~~~ + +Data is compressed only when its size exceeds the above value in bytes. + +**Other paramaters** + +Other less frequently used parameters include `network.compression.mem-level` +and `network.compression.window-size`. More details can about these options +can be found by running `gluster volume set help` command. + +###Known Issues and Limitations + +* Compression translator cannot work with striped volumes. +* Mount point hangs when writing a file with write-behind xlator turned on. To +overcome this, turn off `performance.write-behind` entirely OR +set`performance.strict-write-ordering` to on. +* For glusterfs versions <= 3.5, compression traslator can ONLY work with pure +distribute volumes. This limitation is caused by AFR not being able to +propagate xdata. This issue has been fixed in glusterfs versions > 3.5 + +###TODO +Although zlib offers high compression ratio, it is very slow. We can make the +translator pluggable to add support for other compression methods such as +[lz4 compression](https://code.google.com/p/lz4/) -- cgit