## Major Changes and Features ### Dual Licensing All of GlusterFS 3.4.0 is available under this dual License: - GNU Lesser General Public License, version 3 or any later version (LGPLv3 or later), or - GNU General Public License, version 2 (GPLv2) ### libgfapi libgfapi provides API access to data that is stored in gluster volumes. Description of the API can be found [here] (https://forge.gluster.org/glusterfs-core/glusterfs/blobs/release-3.4/api/src/glfs.h). qemu and samba are already integrated with libgfapi. You can now access gluster volumes directly from qemu and samba without having to come through a regular filesystem mount point. ### Quorum enforcement from Trusted Storage Pool Quorum can now be enforced from the Trusted Storage Pool. Connectivity information present in glusterd is used to determine quorum. Server Side quorum can be enabled by: *\#gluster volume set cluster.server-quorum-type server* *\#gluster volume set cluster.server-quorum-ratio * More details about this feature can be found [here] (http://www.gluster.org/community/documentation/index.php/Features/Server-quorum). ### Improvements for Virtual Machine Image Storage A number of improvements have been performed to let Gluster volumes provide storage for Virtual Machine Images. Some of them include: - qemu - libgfapi integration. - Causal ordering in write-behind translator. - Tunables for a gluster volume in group-virt.example. The above result in significant improvements in performance for VM hosting. ### Synchronous Replication Improvements GlusterFS 3.4 features significant improvements in performance for the replication (AFR) translator. This is in addition to bug fixes for volumes that used replica 3. ### Open Cluster Framework compliant Resource Agents Resource Agents (RA) plug glusterd into Open Cluster Framework (OCF) compliant cluster resource managers, like Pacemaker. The glusterd RA manages the glusterd daemon like any upstart or systemd job would, except that Pacemaker can do it in a cluster-aware fashion. The volume RA starts a volume and monitors individual brick daemons in a cluster aware fashion, recovering bricks when their processes fail. ### POSIX ACL support over NFSv3 setfacl and getfacl commands now can be used on a nfs mount that exports a gluster volume to set or read posix ACLs. ### 3.3.x compatibility The new op-version infrastructure provides compatibility with 3.3.x release of GlusterFS. 3.3.x clients can talk to 3.4.x servers and the vice-versa is also possible. If a volume option that corresponds to 3.4 is enabled, then 3.3 clients cannot mount the volume. ### Packaging changes New RPMs for libgfapi and OCF RA are present with 3.4.0. ### Brick port changes Brick ports will now listen from 49152 onwards (instead of 24009 onwards as with previous releases). The brick port assignment scheme is now compliant with IANA guidelines. ### Experimental Features The following features are experimental with this release: - RDMA-connection manager (RDMA-CM). - Block Device translator. - support for NUFA translator. ## Minor Improvements: - The Ext4 file system change which affected readdir workloads for Gluster volumes has been addressed. - More options for selecting read-child with afr available now. - Custom layouts possible with distribute translator. - No 32-aux-gid limit - SSL support for socket connections. - Known issues with replica count greater than 2 addressed. - quick-read and md-cache translators have been refactored. - open-behind translator introduced. - Ability to avoid glusterfs bind to reserved ports. - statedumps are now created in /var/run/gluster instead of /tmp by default. - UFO Rebased to Openstack Swift 1.8.0 (Grizzly). Note that UFO will be renamed G4S (Gluster For Swift) and decoupled from GlusterFS in the next release. The UFO source in the GlusterFS ufo subdirectory is deprecated following this release. The G4S source is now in the separate git repository [here] (ssh://git.gluster.com/gluster-swift.git). - glusterfs.spec.in mirrors the Fedora glusterfs.spec (delta UFO packages until G4S packages land in Fedora.) - Gluster NFS server now listens by default on port 2049. This allows auto negotiation with NFS clients. - timestamps are now all in UTC ### Known Issues: - The following configuration changes are necessary for qemu and samba integration with libgfapi to work seamlessly: 1) gluster volume set server.allow-insecure on 2) Edit /etc/glusterfs/glusterd.vol to contain this line: option rpc-auth-allow-insecure on Post 2), restarting glusterd would be necessary. - RDMA connection manager needs IPoIB for connection establishment. More details can be found [here] (https://github.com/gluster/glusterfs/blob/master/doc/features/rdma-cm-in-3.4.0.txt). - When NUFA is enabled, client mounts fail outside the trusted storage pool. - Quota enforcement does not work correct with NFS access. - replace-brick operation does not work fine in all cases in this release. More details can be found [here] (https://bugzilla.redhat.com/show_bug.cgi?id=996047) - For Block Device translator based volumes open-behind translator at the client side needs to be disabled. - More information on this can be found in this mailing list [post](http://www.gluster.org/pipermail/gluster-users/2013-July/036750.html) A freshly installed peer cannot be probed into a freshly upgraded cluster. This is a limitation brought about by the introduction of op-versions. There are two ways to workaround this issue, * enable any new feature (for eg. open-behind) on a volume in the cluster, and then perform the probe. Or, if you don't want to enable any new features * delete the 'operating-version' line from /var/lib/glusterd/glusterd.info on the newly installed peer and restart glusterd on the peer. If the file doesn't exist (because of lazy uuid generation), try to probe the peer from the cluster (which will fail) to get the file created.