glusterfs-snapshot.git, branch master_upstream

glusterd: Volume locks and transaction specific opinfos

2013-10-10T07:13:47+00:00

With this patch we are replacing the existing cluster-wide
lock taken on glusterds across the cluster, with volume locks
which are also taken on glusterds across the cluster, but are
volume specific. So with the volume locks we are able to perform
more than one gluster operation at the same time, as long as the
operations are being performed on different volumes.

We maintain a global list of volume-locks (using a dict for a list)
where the key is the volume name, and which saves the uuid of the
originator glusterd. These locks are held and released per volume
transaction.

In order to acheive multiple gluster operations occuring at the
same time, we also separate opinfos in the op-state-machine, as a
part of this patch. To do so, we generate a unique transaction-id
(uuid) per gluster transaction. An opinfo is then associated with
this transaction id, which is used throughout the transaction. We
maintain a run-time global list(using a dict) of transaction-ids,
and their respective opinfos to achieve this.

Change-Id: Iaad505a854bac8de8f83beec0357eb6cde3f7ea8
Upstream Review Url: http://review.gluster.org/5994/
BUG: 1011470
Signed-off-by: Avra Sengupta

updated rfc.sh for master_upstream branch

2013-10-10T07:00:09+00:00

Change-Id: If87f8bda4baac26c51e7152618c223aef722fb36
Signed-off-by: Avra Sengupta

protocol/server: eliminate an unnecessary frame create and destroy

2013-10-02T23:57:22+00:00

connection_cleanup creates a frame solely so that it can be copied.

But the process of copying creates a new frame, and there's nothing
apparently magic about the one being copied; we can eliminate an
unnecessary create and destroy.

Which in the grand scheme of things probably isn't the worst thing
we do, but it's low hanging fruit.

Change-Id: I4a23b84a53e086137b7d4167ad8c20b673d1ffe5
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/6019
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

gfapi.py: support dynamic loading of versioned libraries

2013-10-02T20:04:03+00:00

Currently gfapi.py only loads libraries by filename ending in ".so".  On
an installed system without development packages, the .so filenames
are not available. ctypes.util.find_library() can be used to detect the
files dynamically.

In addition to this, also fixing some minor indention errors and package
the library into the Python site-packages path. Python applications and
libraries can now access libgfapi through 'from glusterfs import gfapi'.

Change-Id: I71e38dabd3ade5dcf24813bf2fc25cda91b571c6
BUG: 1005146
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/5835
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System

tests: Gather only files which need to built

2013-10-02T06:29:56+00:00

Previous code would rather provide a filter but never look
for newly "Added" files leading to failures in subsequent
build.

This patch fixes such issue by actually verifying pedantically
for only files which need to be tested.

Change-Id: Ia716acf82ee60f8ffe5e36257f1cc866c6062718
BUG: 904005
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/6016
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

libglusterfs: add glusterfs-acl.h to Makefile.am

2013-10-01T02:04:03+00:00

Without this, glusterfs-acl.h is left out of 'make dist'
and building RPM fails.

Change-Id: I5dc9dc8eecdea4c4c0d06f0b3da23bd2df4e944e
Signed-off-by: Anand Avati 
BUG: 1009210
Reviewed-on: http://review.gluster.org/6015
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System

Logging : Improved the log message on unlock failure in afr_unlock_inodelk_cbk.

2013-09-30T19:54:17+00:00

"unlock failed on 1 unlock" seems meaningless in the log message. Improved it.

Change-Id: If67d3f9d4aa5310d0b6728a6c89fa58a5cc93d12
BUG: 1012947
Signed-off-by: Anuradha 
Reviewed-on: http://review.gluster.org/6012
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System

glusterd: Calculate volume op-versions only on set/reset

2013-09-30T04:39:38+00:00

The volume op-versions are calculated during a volume set/reset, reading a
volume from disk and importing a volume during probe or volume sync. The
calculation of the volume op-version depends on the clusters op-version as some
features are enabled automatically depending on the clusters op-version. We
also don't store the volume op-versions persistently and don't export the
volume op-versions during sync. Due to this, there can occur cases which will
lead to inconsistencies in volumes in different peers. One such case is below,

Consider, a cluster made up 3 peers P1, P2 and P3, operating at op-version N.
The cluster has two volumes V1 and V2, which have volume op-versions N (since
volume op-version cannot be greater than cluster op-version). We have,
 Cluster-op-version = N
 V1 op-version = N
 V2 op-version = N
A set operation on V1 causes the clusters op-version to be bumped up to N+1.
Assume that there exist some features that are automatically enabled on
op-version N+1. The op-version of V2 remains at N as no operation has been
performed on it. So,
 Cluster op-version = N+1
 V1 op-version = N+1
 V2 op-version = N
Now, we probe a new peer P4. On the new peer we will have the following
op-versions,
 Cluster op-version = N+1
 V1 op-version = N+1
 V2 op-version = N+1
This happens because we don't send volume op-versions during the sync after
probe. P4 will freshly calculate the op-version of V2 (assuming features have
been auto enabled due to the cluster op-version being N+1) as N+1.

Another case is when glusterd on a peer restarts. Assume P3 was restarted,
glusterd will recalculate the volume op-versions during the restore state.
Again, op-version of V2 will be calculated as N+1 assuming auto enabled
features. This will lead to inconsistency in the volume representation in
memory and on disk, as glusterd will assume the volume contains auto enabled
features, but the volfiles don't contain them as they were not regenrated.

These kind of issues can be solved by calculating the volume op-version only
when features are enabled and disabled (ie. during volume set/reset),
persisting the volume-op-versions and exporting/importing them.

Change-Id: I52de0668c92628622e85f4588fb28829a7231132
BUG: 1005043
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/5568
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

glusterd: Fix storing volumes on setting global opts

2013-09-30T00:04:16+00:00

Glusterd would not store all the volumes when a global options were set.
When setting a global option, like 'nfs.*' options, glusterd used to
modify the volinfo for all the volumes, but would store only the volinfo
for the named volume. This lead to mismatch in the persisted and the
in-memory representation of those volumes, which lead to problems like
peers being rejected because of volume mismatches.

Change-Id: I8bca10585e34b7135cb32af0055dbd462b3fb9b5
BUG: 1012400
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/6007
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Amar Tumballi 
Reviewed-by: Anand Avati

glusterd: Set errstr appropriately on peer op failure

2013-09-30T00:01:57+00:00

Change-Id: I27f5f7cd54115d7b236b42f6beaaa05a8b379dd7
BUG: 1010153
Signed-off-by: Krutika Dhananjay 
Reviewed-on: http://review.gluster.org/5978
Reviewed-by: Harshavardhana 
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System