glusterfs.git, branch v3.7.6

Add v3.7.6 release notes

2015-11-08T21:50:58+00:00

Change-Id: I8ccd5463429d87ba07521809c19586f0e6a64f48
Signed-off-by: Raghavendra Talur

tier/libgfdb: Replacing ASCII query file with binary

2015-11-08T13:22:02+00:00

Earlier, when the database was queried we used to save
all the queried records in an ASCII format in the query file.
This caused issues like filename having ASCII delimiter and used
to take a lot of space. The tier.c file also had a lot of parsing code.

Here we changed the format of the query file to binary.
All the logic of serialization and formating of query record is done
by libgfdb. Libgfdb provides API,
gfdb_write_query_record() and gfdb_read_query_record(),
which the user i.e tier migrator and CTR xlator can use to
write to and read from query file.
With this binary format we save on disk space i.e reduce to 50% atleast
as we are saving GFID's in binary format 16 bytes and not the string format
which takes 36 bytes + We are not saving path of the file + we are also saving on
ASCII delimiters.

The on disk format of query record is as follows,

+---------------------------------------------------------------------------+
| Length of serialized query record |       Serialized Query Record         |
+---------------------------------------------------------------------------+
             4 bytes                     Length of serialized query record
                                                      |
                                                      |
     -------------------------------------------------|
     |
     |
     V
   Serialized Query Record Format:
   +---------------------------------------------------------------------------+
   | GFID |  Link count   |    |.....                      | FOOTER |
   +---------------------------------------------------------------------------+
     16 B        4 B         Link Length                                  4 B
                                |                                          |
                                |                                          |
   -----------------------------|                                          |
   |                                                                       |
   |                                                                       |
   V                                                                       |
   Each  will be serialized as                                  |
   +-----------------------------------------------+                       |
   | PGID | BASE_NAME_LENGTH |      BASE_NAME      |                       |
   +-----------------------------------------------+                       |
     16 B       4 B             BASE_NAME_LENGTH                           |
                                                                           |
                                                                           |
   ------------------------------------------------------------------------|
   |
   |
   V
   FOOTER is a magic number 0xBAADF00D indicating the end of the record.
   This also serves as a serialized schema validator.

Backport of http://review.gluster.org/#/c/12354/

> Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a
> BUG: 1272207
> Signed-off-by: Joseph Fernandes 
> Reviewed-on: http://review.gluster.org/12354
> Reviewed-by: N Balachandran 
> Tested-by: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

Change-Id: I170c579027f2594a58706f826e3ddf89e34022f4
BUG: 1263619
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12535
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/ctr: Ignore bitrot related fops

2015-11-08T04:04:18+00:00

Ignore bitrot related fops since they are internal fops.

Backport of http://review.gluster.org/#/c/12512/

Change-Id: I5b676fe450d266b95bbd25aeca0d54436e098917
BUG: 1278640
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12533
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier correct promotion cycle calculation

2015-11-07T21:44:38+00:00

This is a backport of 12480

The tier translator should only choose candidate files for promotion
from the most recent cycle, not a multiple of the most recent cycles.
Otherwise user observed behavior can be inconsistent. Remove related
test in tier.t that is subject to race condition.

> Change-Id: I9ad1523cac00f904097ce468efa6ddd515857024
> BUG: 1275524
> Signed-off-by: root 
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12480
> Reviewed-by: Joseph Fernandes
> Tested-by: Gluster Build System 
Signed-off-by: Dan Lambright 

Signed-off-by: Dan Lambright 

Conflicts:
	tests/basic/tier/tier.t
	xlators/cluster/dht/src/tier.c

Change-Id: Ic4587bf1b5d26ba377a12a4ce8e329362988a33b
BUG: 1275483
Reviewed-on: http://review.gluster.org/12536
Tested-by: NetBSD Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/ctr: Ignore CTR Lookup heal insert errors

2015-11-07T18:32:33+00:00

CTR doesnt read from the DB, so to make sure that file records are
created it does a heal during a lookup. It remembers the decision in
the inode context cache and retrys periodically. When the volume is
restarted it looses all the inode cache from the previous time and CTR
lookup heals tries the heal again, but this time it finds that the records
are already there from sql and logs this error, and remembers this until the
volume is restarted or inode is flushed out of inode cache of the brick.

Solution: the log levels should be reduced to trace for this case and
customers need not see this.

Backport of http://review.gluster.org/#/c/12491/

Change-Id: I5f11e83c306ae912a90f12220f62bb7563b8eb96
BUG: 1279059
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12534
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tier/ctr: Correcting the internal fop calculation

2015-11-07T02:36:44+00:00

Correcting the internal fop calculation method, as it had wrong logic.

backport of http://review.gluster.org/#/c/12418/

Change-Id: I1deeae8b67dd967159853b494e89a3f46572c962
BUG: 1275483
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/12423
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tests/tier : Corrected filename in run-tests.sh

2015-11-06T20:25:20+00:00

bug-1214222-directories_miising_after_attach_tier.t
was renamed to bug-1214222-directories_missing_after_attach_tier.t
 but run-tests.sh was not updated.

> Change-Id: I64d6475ffb08e3252e56b4083cb0e828ba3584d9
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/12528
> Tested-by: NetBSD Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

Change-Id: I9b79ab07442a9c59b8669d19bf5b26b21f1d4a4c
BUG: 1278850
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12531
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier: fix lookup-unhashed on tiered volumes

2015-11-06T16:24:47+00:00

During attach tier the commit hash must be copied to the hot tier.

This is a backport of 12498.

> Change-Id: I91b92fd8e98696993433856e1436409b657c439d
> BUG: 1277716
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/12498
> Tested-by: NetBSD Build System 
> Tested-by: Gluster Build System 
> Reviewed-by: Jeff Darcy 
Signed-off-by: Dan Lambright 

Change-Id: I012c8ce9ef1dbb9550f109a1141afbecaa4fa54d
BUG: 1278603
Reviewed-on: http://review.gluster.org/12525
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

tests: Move ec-readdir.t to bad tests

2015-11-06T13:22:13+00:00

 >Change-Id: Ie7f6d25cbc617ff347aeb7d77fc0a60924c83f09
 >BUG: 1276989
 >Signed-off-by: Pranith Kumar K 
 >Reviewed-on: http://review.gluster.org/12481
 >Tested-by: Raghavendra Talur 
 >Reviewed-by: Raghavendra Talur 

BUG: 1278744
Change-Id: I39161d7cb8bfc5928c71b35e5689c74344a02134
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12529
Reviewed-by: mohammed rafi  kc 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

common-ha: Corrected refresh-config output parsing

2015-11-06T12:35:01+00:00

>>>> Sample program with the earlier changes -
output=$(dbus-send --print-reply --system \
--dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr \
org.ganesha.nfsd.exportmgr.RemoveExport uint16:5 2>&1\
  | grep -v "^method return")
ret=$?
echo "${output}"
echo $ret
sleep 1
output=$(dbus-send --system --dest=org.ganesha.nfsd \
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport \
string:/usr/etc/ganesha/exports/export.vol3.conf \
string:"EXPORT(Path=/vol3)" 2>&1  | grep -v "^method return")
ret=$?
echo "${output}"
echo $ret

Output:

1

1

Even if the command was successfully executed, 'grep -v' has
filtered out the output.

>>>> Sample program with the current changes -

output=$(dbus-send --print-reply --system --dest=org.ganesha.nfsd \
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport\
 uint16:5 2>&1)
ret=$?
echo "${output}"
echo $ret
sleep 1
output=$(dbus-send --print-reply --system --dest=org.ganesha.nfsd \
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport \
string:/usr/etc/ganesha/exports/export.vol3.conf \
string:"EXPORT(Path=/vol3)" 2>&1)
ret=$?
echo "${output}"
echo $ret

Output:
method return sender=:1.155 -> dest=:1.174 reply_serial=2
0
method return sender=:1.155 -> dest=:1.175 reply_serial=2
   string "1 exports added"
0

Change-Id: If9a38e825b2c1a87101de303f9494a0769a9e897
BUG: 1254494
Signed-off-by: Soumya Koduri 
Reviewed-on: http://review.gluster.org/12439
Tested-by: NetBSD Build System 
Reviewed-by: jiffin tony Thottan 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY