glusterfs.git, branch v3.6.4 glusterd: Fix management encryption issues with GlusterD 2015-07-13T10:13:55+00:00 Kaushal M kaushal@redhat.com 2015-07-07T07:22:30+00:00 50b0c7baad43dcd378a5740f154b38ec0ffe0f00 Backport of commit 01b82c6 from master Management encryption was enabled incorrectly in GlusterD leading to issues of cluster deadlocks. This has been fixed with this commit. The fix is in two parts, 1. Correctly enable encrytion for the TCP listener in GlusterD and re-enable own-threads for encrypted connections. Without this, GlusterD could try to esatblish the blocking SSL connects in the epoll thread, for eg. when handling friend updates, which could lead to cluster deadlocks. 2. Explicitly enable encryption for outgoing peer connections. Without enabling encryption explicitly for outgoing connections was causing SSL socket events to be handled in the epoll thread. Some events, like disconnects during peer detach, could lead to connection attempts to happen in the epoll thread, leading to deadlocks again. Change-Id: I438c2b43f7b1965c0e04d95c000144118d36272c BUG: 1241785 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/11612 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
  Backport of commit 01b82c6 from master

Management encryption was enabled incorrectly in GlusterD leading to
issues of cluster deadlocks. This has been fixed with this commit. The
fix is in two parts,

1. Correctly enable encrytion for the TCP listener in GlusterD and
re-enable own-threads for encrypted connections.
  Without this, GlusterD could try to esatblish the blocking SSL
  connects in the epoll thread, for eg. when handling friend updates,
  which could lead to cluster deadlocks.

2. Explicitly enable encryption for outgoing peer connections.
  Without enabling encryption explicitly for outgoing connections was
  causing SSL socket events to be handled in the epoll thread. Some
  events, like disconnects during peer detach, could lead to connection
  attempts to happen in the epoll thread, leading to deadlocks again.

Change-Id: I438c2b43f7b1965c0e04d95c000144118d36272c
BUG: 1241785
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/11612
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
glusterd: use a real host name (instead of numeric) when we have one 2015-07-13T06:24:05+00:00 Jeff Darcy jdarcy@redhat.com 2015-07-08T13:52:15+00:00 c7d8d2ea1e4ad9e247b5876cf17d3a6e0d9af9ad Change-Id: Ie9cc201204d3d613e3e585cab066a07283db902c BUG: 1241275 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/11569 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> 
Change-Id: Ie9cc201204d3d613e3e585cab066a07283db902c
BUG: 1241275
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/11569
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
glusterd: Store peerinfo after updating hostnames 2015-07-10T07:29:44+00:00 Kaushal M kaushal@redhat.com 2015-06-23T10:10:59+00:00 d35d7c1bf986cacb65da912123b9a34ebe946bef Backport of https://review.gluster.org/11365 Change-Id: I1d36ac63de810061d60edb28b6f591ae45d5cd3a BUG: 1234846 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/11395 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
  Backport of https://review.gluster.org/11365

Change-Id: I1d36ac63de810061d60edb28b6f591ae45d5cd3a
BUG: 1234846
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/11395
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
protocol/server: Correctly reconfigure auth.ssl-allow 2015-07-10T07:17:01+00:00 Kaushal M kaushal@redhat.com 2015-07-01T06:14:53+00:00 96987f8e582667defff313e84f85011bff927d3b auth.ssl-allow wasn't being handled during reconfigure. This prevented the ssl-allow list from being live reloaded. Change-Id: If3435793a5684881b012de77cb254b1847b37810 BUG: 1238074 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/11493 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
auth.ssl-allow wasn't being handled during reconfigure. This prevented
the ssl-allow list from being live reloaded.

Change-Id: If3435793a5684881b012de77cb254b1847b37810
BUG: 1238074
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/11493
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
cluster/afr: Do not attempt entry self-heal if the last lookup on entry failed on src 2015-06-19T06:38:01+00:00 Krutika Dhananjay kdhananj@redhat.com 2015-06-08T06:06:12+00:00 d86a238f29c1519bad37bd38d12227bd69d1947f Backport of: http://review.gluster.org/11119 Test bug-948686.t was causing shd to dump core due to gfid being NULL. This was due to the volume being stopped while index heal's in progress, causing afr_selfheal_unlocked_lookup_on() to fail sometimes on the src brick with ENOTCONN. And when afr_selfheal_newentry_mark() copies the gfid off the src iatt, it essentially copies null gfid. This was causing the assertion as part of xattrop in protocol/client to fail. Change-Id: I81723567af824ce4a9fa37e309eeeab8404ac71e BUG: 1233036 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/11309 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
        Backport of: http://review.gluster.org/11119

Test bug-948686.t was causing shd to dump core due to gfid being NULL.
This was due to the volume being stopped while index heal's in progress,
causing afr_selfheal_unlocked_lookup_on() to fail sometimes on the src brick
with ENOTCONN. And when afr_selfheal_newentry_mark() copies the gfid off the
src iatt, it essentially copies null gfid. This was causing the assertion
as part of xattrop in protocol/client to fail.

Change-Id: I81723567af824ce4a9fa37e309eeeab8404ac71e
BUG: 1233036
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/11309
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
afr: allow readdir to proceed for directories in split-brain 2015-06-15T16:28:59+00:00 Ravishankar N ravishankar@redhat.com 2015-06-10T13:04:52+00:00 83778a592f75ef18ce5e22e14adc9bf118705a17 Problem: afr_read_txn() bails out if read_subvol==-1. This meant that for directories that were in entry split-brain, FOPS like readdir, access, stat etc were not allowed. Fix: Except for getxattr, all other FOPS are wound on the first up child of afr. Change-Id: Iacec8fbb1e75c4d2094baa304f62331c81a6f670 BUG: 1230242 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/10776 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Anuradha Talur <atalur@redhat.com> (cherry picked from commit 49b428433a03fcf709fdc8c08603b4cf02198e0a) Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/11162 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
Problem:
afr_read_txn() bails out if read_subvol==-1. This meant that for
directories that were in entry split-brain, FOPS like readdir, access,
stat etc were not allowed.

Fix:
Except for getxattr, all other FOPS are wound on the first up child
of afr.

Change-Id: Iacec8fbb1e75c4d2094baa304f62331c81a6f670
BUG: 1230242
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/10776
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
(cherry picked from commit 49b428433a03fcf709fdc8c08603b4cf02198e0a)
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/11162
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
afr: honour selfheal enable/disable volume set options 2015-06-15T13:18:44+00:00 Ravishankar N ravishankar@redhat.com 2015-05-30T04:53:33+00:00 fecebde3fbae17ace970a4d9c440f6455161dc62 Backport of http://review.gluster.org/11012 Note: http://review.gluster.org/9459 is not backported to 3.6 but the change it makes to afr_get_heal_info() (i.e. handling ret values) is needed for heal info to work correctly and tests/basic/afr/client-side-heal.t to pass. -------------------------- afr-v1 had the following volume set options that are used to enable/ disable self-heals from happening in AFR xlator when loaded in the client graph: cluster.metadata-self-heal cluster.data-self-heal cluster.entry-self-heal In afr-v2, these 3 heals can happen from the client if there is an inode refresh. This patch allows such heals to proceed only if the corresponding volume set options are set to true. -------------------------- Change-Id: Iebf863758d902fd2f95be320c6791d4e15f634e7 BUG: 1230259 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: http://review.gluster.org/11170 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anuradha Talur <atalur@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
Backport of  http://review.gluster.org/11012
Note: http://review.gluster.org/9459 is not backported to 3.6 but the change it
makes to afr_get_heal_info() (i.e. handling ret values) is needed for heal info
to work correctly and tests/basic/afr/client-side-heal.t to pass.

--------------------------
afr-v1 had the following volume set options that are used to enable/ disable
self-heals from happening in AFR xlator when loaded in the client graph:
cluster.metadata-self-heal
cluster.data-self-heal
cluster.entry-self-heal

In afr-v2, these 3 heals can happen from the client if there is an inode
refresh. This patch allows such heals to proceed only if the corresponding
volume set options are set to true.
--------------------------

Change-Id: Iebf863758d902fd2f95be320c6791d4e15f634e7
BUG: 1230259
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/11170
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
core: fix Ubuntu code audit (cppcheck) results 2015-06-10T12:03:01+00:00 Kaleb S. KEITHLEY kkeithle@redhat.com 2014-07-22T21:03:56+00:00 4a1a1c9945656b3197b2677dd6c2142f02ade4dc See also http://review.gluster.org/#/c/8064/, BZ 1109180, and http://review.gluster.org/#/c/7693/, BZ 1091677 AFAICT these are false positives: [geo-replication/src/gsyncd.c:100]: (error) Memory leak: str [geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv [xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde [xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr Test program: [extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments. [tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments. the remainder are fixed with this change-set: [cli/src/cli-rpc-ops.c:8883]: (error) Possible null pointer dereference: local [cli/src/cli-rpc-ops.c:8886]: (error) Possible null pointer dereference: local [contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'. [contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'. [xlators/cluster/dht/src/dht-rebalance.c:1734]: (error) Possible null pointer dereference: ctx [xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local [xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1718]: (error) Possible null pointer dereference: command [xlators/mgmt/glusterd/src/glusterd-replace-brick.c:942]: (error) Resource leak: file [xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1026]: (error) Resource leak: file [xlators/mgmt/glusterd/src/glusterd-sm.c:249]: (error) Possible null pointer dereference: new_ev_ctx [xlators/mgmt/glusterd/src/glusterd-snapshot.c:6917]: (error) Possible null pointer dereference: volinfo [xlators/mgmt/glusterd/src/glusterd-utils.c:4517]: (error) Possible null pointer dereference: this [xlators/mgmt/glusterd/src/glusterd-utils.c:6662]: (error) Possible null pointer dereference: this [xlators/mgmt/glusterd/src/glusterd-utils.c:7708]: (error) Possible null pointer dereference: this [xlators/mount/fuse/src/fuse-bridge.c:4687]: (error) Uninitialized variable: finh [xlators/mount/fuse/src/fuse-bridge.c:3080]: (error) Possible null pointer dereference: state [xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it). [xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf Rerunning cppcheck after fixing the above: As before, test program: [extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments. [tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments. As before, false positive: [geo-replication/src/gsyncd.c:100]: (error) Memory leak: str [geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv [xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde [xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr [xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr False positive after fix: [xlators/performance/quick-read/src/quick-read.c:584]: (error) Possible null pointer dereference: iobuf Change-Id: Ia5a256281156dd1df2fa900caea7694d9d0a7077 BUG: 1122290 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/8351 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
See also http://review.gluster.org/#/c/8064/, BZ 1109180, and
http://review.gluster.org/#/c/7693/, BZ 1091677

AFAICT these are false positives:

[geo-replication/src/gsyncd.c:100]: (error) Memory leak: str
[geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde
[xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

Test program:

[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments.

the remainder are fixed with this change-set:

[cli/src/cli-rpc-ops.c:8883]: (error) Possible null pointer dereference: local
[cli/src/cli-rpc-ops.c:8886]: (error) Possible null pointer dereference: local
[contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'.
[contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'.
[xlators/cluster/dht/src/dht-rebalance.c:1734]: (error) Possible null pointer dereference: ctx
[xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local
[xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1718]: (error) Possible null pointer dereference: command
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:942]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-replace-brick.c:1026]: (error) Resource leak: file
[xlators/mgmt/glusterd/src/glusterd-sm.c:249]: (error) Possible null pointer dereference: new_ev_ctx
[xlators/mgmt/glusterd/src/glusterd-snapshot.c:6917]: (error) Possible null pointer dereference: volinfo
[xlators/mgmt/glusterd/src/glusterd-utils.c:4517]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:6662]: (error) Possible null pointer dereference: this
[xlators/mgmt/glusterd/src/glusterd-utils.c:7708]: (error) Possible null pointer dereference: this
[xlators/mount/fuse/src/fuse-bridge.c:4687]: (error) Uninitialized variable: finh
[xlators/mount/fuse/src/fuse-bridge.c:3080]: (error) Possible null pointer dereference: state
[xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it).
[xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf

Rerunning cppcheck after fixing the above:

As before, test program:

[extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[tests/basic/fops-sanity.c:55]: (error) Buffer overrun possible for long command line arguments.

As before, false positive:

[geo-replication/src/gsyncd.c:100]: (error) Memory leak: str
[geo-replication/src/gsyncd.c:403]: (error) Memory leak: argv
[xlators/nfs/server/src/nlm4.c:1201]: (error) Possible null pointer dereference: fde
[xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

False positive after fix:

[xlators/performance/quick-read/src/quick-read.c:584]: (error) Possible null pointer dereference: iobuf

Change-Id: Ia5a256281156dd1df2fa900caea7694d9d0a7077
BUG: 1122290
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/8351
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
tests/uss.t: wait for connection to complete before accessing 2015-06-10T12:01:16+00:00 Raghavendra Bhat raghavendra@redhat.com 2015-06-03T09:32:16+00:00 cd822f8c6f0809b1b3ac3816d98f2bfd7dd5f2c6 Change-Id: Ia7e406586a1959b6a2024f90e1574c399574a536 BUG: 1227670 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11059 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Change-Id: Ia7e406586a1959b6a2024f90e1574c399574a536
BUG: 1227670
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/11059
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
cluster/afr: Treat op_ret >= 0 as success in afr_final_errno() 2015-06-03T12:50:35+00:00 Krutika Dhananjay kdhananj@redhat.com 2015-05-27T13:33:12+00:00 bc37696f76ee7575ab7321e3ee1ebaaefe759caa Backport of: http://review.gluster.org/10946 Change-Id: I6ca36fee5dc46f2a2afe04f3359c4102f45c3ae0 BUG: 1225745 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/10961 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> 
        Backport of: http://review.gluster.org/10946

Change-Id: I6ca36fee5dc46f2a2afe04f3359c4102f45c3ae0
BUG: 1225745
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/10961
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>