core: run many bricks within one glusterfsd process

This patch adds support for multiple brick translator stacks running
in a single brick server process.  This reduces our per-brick memory usage by
approximately 3x, and our appetite for TCP ports even more.  It also creates
potential to avoid process/thread thrashing, and to improve QoS by scheduling
more carefully across the bricks, but realizing that potential will require
further work.

Multiplexing is controlled by the "cluster.brick-multiplex" global option.  By
default it's off, and bricks are started in separate processes as before.  If
multiplexing is enabled, then *compatible* bricks (mostly those with the same
transport options) will be started in the same process.

Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb
BUG: 1385758
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: https://review.gluster.org/14763
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

1 files changed, 43 insertions, 0 deletions

diff --git a/tests/basic/mpx-compat.t b/tests/basic/mpx-compat.t
new file mode 100644
index 00000000000..3de0f6fe7cb
--- /dev/null
+++ b/tests/basic/mpx-compat.t
@@ -0,0 +1,43 @@
+#!/bin/bash
+#This test tests that self-heals don't perform fsync when durability is turned
+#off
+
+. $(dirname $0)/../include.rc
+. $(dirname $0)/../traps.rc
+. $(dirname $0)/../volume.rc
+
+function count_processes {
+	# It would generally be a good idea to use "pgrep -x" to ensure an
+	# exact match, but the version of pgrep we have on NetBSD (a.k.a.
+	# the worst operating system ever) doesn't support that option.
+	# Fortunately, "glusterfsd" isn't the prefix of any other name,
+	# so this works anyway.  For now.
+	pgrep glusterfsd | wc -w
+}
+
+TEST glusterd
+TEST $CLI volume set all cluster.brick-multiplex yes
+push_trapfunc "$CLI volume set all cluster.brick-multiplex off"
+push_trapfunc "cleanup"
+
+# Create two vanilla volumes.
+TEST $CLI volume create $V0 $H0:$B0/brick-${V0}-{0,1}
+TEST $CLI volume create $V1 $H0:$B0/brick-${V1}-{0,1}
+
+# Start both.
+TEST $CLI volume start $V0
+TEST $CLI volume start $V1
+
+# There should be only one process for compatible volumes.  We can't use
+# EXPECT_WITHIN here because it could transiently see one process as two are
+# coming up, and yield a false positive.
+sleep $PROCESS_UP_TIMEOUT
+EXPECT "1" count_processes
+
+# Make the second volume incompatible with the first.
+TEST $CLI volume stop $V1
+TEST $CLI volume set $V1 server.manage-gids no
+TEST $CLI volume start $V1
+
+# There should be two processes this time (can't share protocol/server).
+EXPECT_WITHIN $PROCESS_UP_TIMEOUT "2" count_processes