glusterfs.git/rpc, branch v3.7dev cli/glusterd: Added support for dispersed volumes 2014-07-11T17:34:24+00:00 Xavier Hernandez xhernandez@datalab.es 2014-05-15T08:35:14+00:00 1392da3e237d8ea080573909015916e3544a6d2c Two new options have been added to the 'create' command of the cli interface: disperse [<count>] redundancy <count> Both are optional. A dispersed volume is created by specifying, at least, one of them. If 'disperse' is missing or it's present but '<count>' does not, the number of bricks enumerated in the command line is taken as the disperse count. If 'redundancy' is missing, the lowest optimal value is assumed. A configuration is considered optimal (for most workloads) when the disperse count - redundancy count is a power of 2. If the resulting redundancy is 1, the volume is created normally, but if it's greater than 1, a warning is shown to the user and he/she must answer yes/no to continue volume creation. If there isn't any optimal value for the given number of bricks, a warning is also shown and, if the user accepts, a redundancy of 1 is used. If 'redundancy' is specified and the resulting volume is not optimal, another warning is shown to the user. A distributed-disperse volume can be created using a number of bricks multiple of the disperse count. Change-Id: Iab93efbe78e905cdb91f54f3741599f7ea6645e4 BUG: 1118629 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/7782 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Two new options have been added to the 'create' command of the cli
interface:

    disperse [<count>] redundancy <count>

Both are optional. A dispersed volume is created by specifying, at
least, one of them. If 'disperse' is missing or it's present but
'<count>' does not, the number of bricks enumerated in the command
line is taken as the disperse count.

If 'redundancy' is missing, the lowest optimal value is assumed. A
configuration is considered optimal (for most workloads) when the
disperse count - redundancy count is a power of 2. If the resulting
redundancy is 1, the volume is created normally, but if it's greater
than 1, a warning is shown to the user and he/she must answer yes/no
to continue volume creation. If there isn't any optimal value for
the given number of bricks, a warning is also shown and, if the user
accepts, a redundancy of 1 is used.

If 'redundancy' is specified and the resulting volume is not optimal,
another warning is shown to the user.

A distributed-disperse volume can be created using a number of bricks
multiple of the disperse count.

Change-Id: Iab93efbe78e905cdb91f54f3741599f7ea6645e4
BUG: 1118629
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/7782
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
socket/glusterd/client: enable SSL for management 2014-07-10T14:37:12+00:00 Jeff Darcy jdarcy@redhat.com 2014-07-03T14:01:20+00:00 b42688786f25420de671ea06030edf4371058433 The feature is controlled by presence of the following file: /var/lib/glusterd/secure-access See the comment near the definition of SECURE_ACCESS_FILE in glusterfs.h for the rationale. With this enabled, the following rules apply to connections: UNIX-domain sockets never have SSL. Management-port sockets (both connecting and accepting, in daemons and CLI) have SSL based on presence of the file. Other IP sockets have SSL based on the existing client.ssl and server.ssl volume options. Transport multi-threading is explicitly turned off in glusterd (it would otherwise be turned on when SSL is) due to multi-threading issues. Tests have been elided to avoid risk of leaving a file which will cause all subsequent tests to run with management SSL still enabled. IMPLEMENTATION NOTE The implementation is a bit messy, and consists of two stages. First we decide whether to set the relevant fields in our context structure, based on presence of the sentinel file OR a command-line override. Later we decide whether a particular connection should actually use SSL, based on the context flags plus what kind of connection we're making[1] and what kind of daemon we're in[2]. [1] inbound, outbound to glusterd port, other outbound [2] glusterd, glusterfsd, other TESTING NOTE Instead of just running one special test for this feature, the ideal would be to run all tests with management SSL enabled. However, it would be inappropriate or premature to set up an optional feature in the patch itself. Therefore, the method of choice is to submit a separate patch on top, which modifies "cleanup" in include.rc to recreate the secure-access file and associated SSL certificate/key files before each test. Change-Id: I0e04d6d08163893e24ec8c031748c5c447d7f780 BUG: 1114604 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/8094 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
The feature is controlled by presence of the following file:

	/var/lib/glusterd/secure-access

See the comment near the definition of SECURE_ACCESS_FILE in glusterfs.h
for the rationale.  With this enabled, the following rules apply to
connections:

	UNIX-domain sockets never have SSL.

	Management-port sockets (both connecting and accepting, in
	daemons and CLI) have SSL based on presence of the file.

	Other IP sockets have SSL based on the existing client.ssl and
	server.ssl volume options.

Transport multi-threading is explicitly turned off in glusterd (it would
otherwise be turned on when SSL is) due to multi-threading issues.
Tests have been elided to avoid risk of leaving a file which will cause
all subsequent tests to run with management SSL still enabled.

IMPLEMENTATION NOTE
The implementation is a bit messy, and consists of two stages.  First we
decide whether to set the relevant fields in our context structure, based
on presence of the sentinel file OR a command-line override.  Later we
decide whether a particular connection should actually use SSL, based on the
context flags plus what kind of connection we're making[1] and what kind of
daemon we're in[2].

[1] inbound, outbound to glusterd port, other outbound
[2] glusterd, glusterfsd, other

TESTING NOTE
Instead of just running one special test for this feature, the ideal
would be to run all tests with management SSL enabled.  However, it
would be inappropriate or premature to set up an optional feature in the
patch itself.  Therefore, the method of choice is to submit a separate
patch on top, which modifies "cleanup" in include.rc to recreate the
secure-access file and associated SSL certificate/key files before each
test.

Change-Id: I0e04d6d08163893e24ec8c031748c5c447d7f780
BUG: 1114604
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/8094
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
rpc: Do not reset @ping_started to 0 in ping callback 2014-07-09T13:57:20+00:00 Krutika Dhananjay kdhananj@redhat.com 2014-07-08T02:48:27+00:00 b9f1d1a120b4469d51d2a96eecc7ce83516593ba This is to avoid indefinite recursion of the following kind, that could lead to a stack overflow: rpc_clnt_start_ping() -> rpc_clnt_ping() -> rpc_clnt_submit() -> rpc_clnt_start_ping() -> rpc_clnt_ping() -> rpc_clnt_submit() ... and so on, since it is possible that before rpc_clnt_start_ping() is called a second time by the thread executing this codepath, the response to previous ping request could ALWAYS come by and cause epoll thread to reset conn->ping_started to 0. This patch also fixes the issue of excessive ping traffic, which was due to the client sending one ping rpc for every fop in the worst case. Also removed dead code in glusterd. Change-Id: I7c5e6ae3b1c9d23407c0a12a319bdcb43ba7a359 BUG: 1116243 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/8257 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
This is to avoid indefinite recursion of the following kind, that could
lead to a stack overflow:

rpc_clnt_start_ping() -> rpc_clnt_ping() -> rpc_clnt_submit() ->
rpc_clnt_start_ping() -> rpc_clnt_ping() -> rpc_clnt_submit() ...
and so on,

since it is possible that before rpc_clnt_start_ping() is called a
second time by the thread executing this codepath, the response to
previous ping request could ALWAYS come by and cause epoll thread to
reset conn->ping_started to 0.

This patch also fixes the issue of excessive ping traffic, which was
due to the client sending one ping rpc for every fop in the worst case.

Also removed dead code in glusterd.

Change-Id: I7c5e6ae3b1c9d23407c0a12a319bdcb43ba7a359
BUG: 1116243
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/8257
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
socket: add certificate-depth and cipher-list options for SSL 2014-07-04T11:18:00+00:00 Jeff Darcy jdarcy@redhat.com 2014-07-03T13:27:13+00:00 83c09b75a8fbc3a46fc0e76f805e061e949678f1 Change-Id: I82757f8461807301a4a4f28c4f5bf7f0ee315113 BUG: 1114604 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/8040 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Change-Id: I82757f8461807301a4a4f28c4f5bf7f0ee315113
BUG: 1114604
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/8040
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
porting: Port for FreeBSD rebased from Mike Ma's efforts 2014-07-03T00:20:34+00:00 Harshavardhana harsha@harshavardhana.net 2014-06-21T09:00:23+00:00 1b74cf992986287a510fe3b28a8ee7554e8b0992 - Provides a working Gluster Management Daemon, CLI - Provides a working GlusterFS server, GlusterNFS server - Provides a working GlusterFS client - execinfo port from FreeBSD is moved into ./contrib/libexecinfo for ease of portability on NetBSD. (FreeBSD 10 and OSX provide execinfo natively) - More portability cleanups for Darwin, FreeBSD and NetBSD - Provides a new rc script for FreeBSD Change-Id: I8dff336f97479ca5a7f9b8c6b730051c0f8ac46f BUG: 1111774 Original-Author: Mike Ma <mikemandarine@gmail.com> Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/8141 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> 
- Provides a working Gluster Management Daemon, CLI
- Provides a working GlusterFS server, GlusterNFS server
- Provides a working GlusterFS client
- execinfo port from FreeBSD is moved into ./contrib/libexecinfo
  for ease of portability on NetBSD. (FreeBSD 10 and OSX provide
  execinfo natively)
- More portability cleanups for Darwin, FreeBSD and NetBSD
- Provides a new rc script for FreeBSD

Change-Id: I8dff336f97479ca5a7f9b8c6b730051c0f8ac46f
BUG: 1111774
Original-Author: Mike Ma <mikemandarine@gmail.com>
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/8141
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
rpc/auth: allow SSL identity to be used for authorization 2014-07-02T09:47:05+00:00 Jeff Darcy jdarcy@redhat.com 2014-04-17T23:21:05+00:00 caa8a4ea50734378e7e19f70b39a837c58e9d229 Access to a volume is now controlled by the following options, based on whether SSL is enabled or not. * server.ssl-allow: get identity from certificate, no password needed * auth.allow: get identity and matching password from command line It is not possible to allow both simultaneously, since the connection itself is either using SSL or it isn't. Change-Id: I5a5be66520f56778563d62f4b3ab35c66cc41ac0 BUG: 1114604 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/3695 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Access to a volume is now controlled by the following options, based on
whether SSL is enabled or not.

 * server.ssl-allow: get identity from certificate, no password needed

 * auth.allow: get identity and matching password from command line

It is not possible to allow both simultaneously, since the connection
itself is either using SSL or it isn't.

Change-Id: I5a5be66520f56778563d62f4b3ab35c66cc41ac0
BUG: 1114604
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/3695
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
mgmt/glusterd: display snapd status as part of volume status 2014-07-01T05:30:58+00:00 Raghavendra Bhat raghavendra@redhat.com 2014-06-19T10:21:39+00:00 c6f040524d75011c44dcc9afdfef80c60c78f7f7 * Made changes to save the port used by snapd in the info file for the volume i.e. <glusterd-working-directory>/vols/<volname>/info This is how the gluster volume status of a volume would look like for which the uss feature is enabled. [root@tatooine ~]# gluster volume status vol Status of volume: vol Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick tatooine:/export1/vol 49155 Y 5041 Snapshot Daemon on localhost 49156 Y 5080 NFS Server on localhost 2049 Y 5087 Task Status of Volume vol ------------------------------------------------------------------------------ There are no active volume tasks Change-Id: I8f3e5d7d764a728497c2a5279a07486317bd7c6d BUG: 1111041 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/8114 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaushal M <kaushal@redhat.com> 
* Made changes to save the port used by snapd in the info file for the volume
  i.e. <glusterd-working-directory>/vols/<volname>/info

This is how the gluster volume status of a volume would look like for which the
uss feature is enabled.

[root@tatooine ~]# gluster volume status vol
Status of volume: vol
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick tatooine:/export1/vol                             49155   Y       5041
Snapshot Daemon on localhost                            49156   Y       5080
NFS Server on localhost                                 2049    Y       5087

Task Status of Volume vol
------------------------------------------------------------------------------
There are no active volume tasks

Change-Id: I8f3e5d7d764a728497c2a5279a07486317bd7c6d
BUG: 1111041
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/8114
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
gNFS: Fix multi-homed m/c issue in NFS subdir auth 2014-06-25T07:41:52+00:00 Santosh Kumar Pradhan spradhan@redhat.com 2014-06-06T06:52:04+00:00 d3f0de90d0c5166e63f5764d2f21703fd29ce976 NFS subdir authentication doesn't correctly handle multi-homed (host with multiple NIC having multiple IP addr) OR multi-protocol (IPv4 and IPv6) network addresses. When user/admin sets HOSTNAME in gluster CLI for NFS subdir auth, mnt3_verify_auth() routine does not iterate over all the resolved n/w addrs returned by getaddrinfo() n/w API. Instead, it just tests with the one returned first. 1. Iterate over all the n/w addrs (linked list) returned by getaddrinfo(). 2. Move the n/w mask calculation part to mnt3_export_fill_hostspec() instead of doing it in mnt3_verify_auth() i.e. calculating for each mount request. It does not change for MOUNT req. 3. Integrate "subnet support code rpc-auth.addr.<volname>.allow" and "NFS subdir auth code" to remove code duplication. Change-Id: I26b0def52c22cda35ca11766afca3df5fd4360bf BUG: 1102293 Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com> Reviewed-on: http://review.gluster.org/8048 Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> 
NFS subdir authentication doesn't correctly handle multi-homed
(host with multiple NIC having multiple IP addr) OR multi-protocol
(IPv4 and IPv6) network addresses.

When user/admin sets HOSTNAME in gluster CLI for NFS subdir auth,
mnt3_verify_auth() routine does not iterate over all the resolved
n/w addrs returned by getaddrinfo() n/w API. Instead, it just tests
with the one returned first.

1. Iterate over all the n/w addrs (linked list) returned by getaddrinfo().
2. Move the n/w mask calculation part to mnt3_export_fill_hostspec()
   instead of doing it in mnt3_verify_auth() i.e. calculating for each
   mount request. It does not change for MOUNT req.
3. Integrate "subnet support code rpc-auth.addr.<volname>.allow"
   and "NFS subdir auth code" to remove code duplication.

Change-Id: I26b0def52c22cda35ca11766afca3df5fd4360bf
BUG: 1102293
Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>
Reviewed-on: http://review.gluster.org/8048
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
glusterd: Fail peer probe/detach commands when peer detach is ongoing 2014-06-16T17:03:12+00:00 Krishnan Parthasarathi kparthas@redhat.com 2014-06-16T12:09:08+00:00 6e33d855815726bd8ebf1c7cdc3cf233ff7ac231 Change-Id: Ifd8099bc235eb395e8fd9ead3197bef71c78042b BUG: 1109812 Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/8079 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
Change-Id: Ifd8099bc235eb395e8fd9ead3197bef71c78042b
BUG: 1109812
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/8079
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Get snapshot info dynamically via new rpc and infra for snapview-server to refresh snaplist 2014-06-15T17:33:23+00:00 Anand Subramanian anands@redhat.com 2014-05-15T02:49:14+00:00 ec845d93e74644bbfe27e0b085a38dbd7c02480f BUG: 1105439 Change-Id: I4bb312a53d88f6f4955e69a3ef2b4955ec17f26d Signed-off-by: Anand Subramanian <anands@redhat.com> Reviewed-on: http://review.gluster.org/8001 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> 
BUG: 1105439
Change-Id: I4bb312a53d88f6f4955e69a3ef2b4955ec17f26d
Signed-off-by: Anand Subramanian <anands@redhat.com>
Reviewed-on: http://review.gluster.org/8001
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>