glusterfs.git/rpc, branch v3.8dev

rpc-lib: Fixing the coverity issues

2015-04-10T11:29:42+00:00

Coverity CIDs:
1210973
1124887
1124888
1124682
1124849
1124503

Change-Id: I012f6cf9d14753f572ab94aae6d442d1ef8df79a
BUG: 789278
Signed-off-by: Nandaja Varma 
Reviewed-on: http://review.gluster.org/9600
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

rpc: fix deadlock when unref is inside conn->lock

2015-04-10T03:03:48+00:00

In ping-timer implementation, the timer event takes a ref on the rpc
object. This ref needs to be removed after every timeout event.
ping-timer mechanism could be holding the last ref. For e.g, when a peer
is detached and its rpc object was unref'd. In this case, ping-timer
mechanism would try to acquire conn->mutex to perform the 'last' unref
while being inside the critical section already. This will result in a
deadlock.

Change-Id: I74f80dd08c9348bd320a1c6d12fc8cd544fa4aea
BUG: 1206134
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9613
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

socket: use OpenSSL multi-threading interfaces

2015-04-09T09:55:37+00:00

OpenSSL isn't thread-safe unless you register these locking and thread
ID functions.  Most often the crashes would occur around
X509_verify_cert, even though it's insane that the certificate parsing
functions wouldn't be thread-safe.  The bug for this was filed over
two years ago, but it didn't seem like a high priority because the bug
didn't bite anyone until it caused a spurious regression-test failure.
Ironically, that was on a test for a *different* spurious
regression-test failure, which I guess is just deserts[1] for leaving
this on the to-do list so long.

[1] Yes, it really is "deserts" in that phrase - not as in very dry
places, but from late Latin "deservire" meaning to serve well or
zealously.  Aren't commit messages educational?

Change-Id: I2a6c0e9b361abf54efa10ffbbbe071404f82b0d9
BUG: 906763
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/10075
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

build: add more files to .gitignore

2015-04-06T13:00:39+00:00

Change-Id: Icef0d7f443f7caf3aa386d3a6978f98cf3a5a4af
BUG: 1198849
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10132
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System

rpc: Fixing dereferencing after null check

2015-04-02T13:37:53+00:00

CID: 1124607
The pointer variable is checked for NULL and
logged accordingly.

Change-Id: Ied0d7f7ff33da22198eca65f14816b943cae5541
BUG: 789278
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/9674
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos 
Reviewed-by: Vijay Bellur

socket: use TLS 1.2 instead of 1.0

2015-03-27T18:29:33+00:00

Change-Id: I96e9b37e4855f5e12b2dbecf1f0b0887b21ad5ad
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/9949
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: CLI commands to create and manage tiered volumes.

2015-03-19T13:32:28+00:00

A tiered volume is a normal volume with some number of new bricks
representing "hot" storage. The "hot" bricks can be attached or
detached dynamically to a normal volume. When this happens, a new graph
is constructed. The root of the new graph is an instance of the tier
translator. One subvolume of the tier translator leads to the old volume,
and another leads to the new hot bricks.

attach-tier  [ ]  ... [force]

volume detach-tier  [replica ] 
... 

gluster volume rebalance  tier start

gluster volume rebalance  tier stop

gluster volume rebalance  tier status

The "tier start" CLI command starts a server side daemon. The daemon
initiates file level migration based on caching policies. The daemon's
status can be monitored and stopped.

Note development on the "tier status" command is incomplete. It will be
added in a subsequent patch.

When the "hot" storage is detached, the tier translator is removed
from the graph and the tiered volume reverts to its original state as
described in the volume's info file.

For more background and design see the feature page [1].

[1]
http://www.gluster.org/community/documentation/index.php/Features/data-classification

Change-Id: Ic8042ce37327b850b9e199236e5be3dae95d2472
BUG: 1194753
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9753
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur

cli/glusterd: cli command implementation for bitrot features

2015-03-19T01:34:07+00:00

CLI command for bitrot features.

volume bitrot  enable|disable

Above command will enable/disable bitrot feature for particular volume.

BUG: 1170075
Change-Id: Ie84002ef7f479a285688fdae99c7afa3e91b8b99
Signed-off-by: Gaurav Kumar Garg     
Signed-off-by: Anand nekkunti        
Signed-off-by: Dominic P Geevarghese 
Reviewed-on: http://review.gluster.org/9866
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/quota : Introducing inode quota

2015-03-19T01:24:12+00:00

==========================================================================
                             Inode quota
==========================================================================
= Currently, the only way to retrieve the number of files/objects in a   =
= directory or volume is to do a crawl of the entire directory/volume.   =
= This is expensive and is not scalable.                                 =
=                                                                        =
= The proposed mechanism will provide an easier alternative to determine =
= the count of files/objects in a directory or volume.                   =
=                                                                        =
= The new mechanism proposes to store count of objects/files as part of  =
= an extended attribute of a directory. Each directory's extended        =
= attribute value will indicate the number of files/objects present      =
= in a tree with the directory being considered as the root of the tree. =
=                                                                        =
= The count value can be accessed by performing a getxattr().            =
= Cluster translators like afr, dht and stripe will perform aggregation  =
= of count values from various bricks when getxattr() happens on the key =
= associated with file/object count.                                     =

A new interface is introduced:
------------------------------
        limit-objects  : limit the number of inodes at directory level
        list-objects   : list the directories where the limit is set
        remove-objects : remove the limit from the directory

==========================================================================

CLI COMMAND:
gluster volume quota  limit-objects   []

*  is a hard-limit for number of objects limitation for path ""
  If hard-limit is exceeded, creation of file/directory is no longer
permitted.

*  is a soft-limit for number of objects creation for path ""
  If soft-limit is exceeded, a warning is issued for each creation.

CLI COMMAND:
gluster volume quota  remove-objects [path]

==========================================================================

CLI COMMAND:
gluster volume quota  list-objects [path] ...

Sample output:
------------------
  Path                   Hard-limit Soft-limit   Used  Available
Soft-limit exceeded?
Hard-limit exceeded?
  ------------------------------------------------------------------------
--------------------------------------
  /dir                      10       80%          10       0
Yes
        Yes

==========================================================================

[root@snapshot-28 dir]# ls
a  b  file11  file12  file13  file14  file15  file16  file17
[root@snapshot-28 dir]# touch a1
touch: cannot touch `a1': Disk quota exceeded
* Nine files are created in directory "dir" and directory is included in
* the
count too. Hence the limit "10" is reached and further file creation
fails

==========================================================================

Note: We have also done some re-factoring in cli for volume name
validation. New function cli_validate_volname is created

==========================================================================

Change-Id: I1823497de4f790a2a20ebb1770293472ea33ee2b
BUG: 1190108
Signed-off-by: Sachin Pandit 
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/9769
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog: RPC'fy {libgf}changelog

2015-03-19T01:22:36+00:00

This patch introduces RPC based communication between the changelog
translator and libgfchangelog. It replaces the old pathetic stream
based interaction that existed earlier (due to time constraints :-/).

Changelog, upon initialization starts a RPC server (rpcsvc) allowing
clients to invoke a probe API as a bootup mechanism to request for
event notifications. During probe, clients can choose an event
filter specifying the type(s) of events they are interested in. As
of now there is no way to change the event notification set once
the probe RPC call is made, but that is easier to implement.

The actual event notifications is done on a separate RPC session.
The client (libgfchangelog) itself starts and RPC server which the
changelog translator "connects back" during probe. Notifications
are dispatched by a bunch of threads from the server (translator)
and the client optionally orders them if ordered notifications
are requried. FOPs fill in their respective event details in a
buffer (rot-buffs to be particular) and a bunch of threads
(consumers) swap the buffers out of roatation and dispatch them
via RPC. To avoid writer starvation, then number of dispatcher
threads is one less than the number of buffer list in rot-buffs.x

libgfchangelog becomes purely callback based -- upon event
notification from the server (and re-ordering them if required)
invoke a callback routine specified by consumer(s).

A major part of the patch is also aimed at providing backward
compatibility for geo-replication, which was one of the main
consumer of the stream based API. Also, this patch does not\
"turn on" event notifications for all fops, just a bunch which
is currently in requirement. Another pain point is that the
server does not filter events before dispatching it to the
clients. That load is taken up by the client itself (although
it's done at the library layer rather than making it hard on
the callback implementor). This needs improvement and care
needs to be taken to not load the server up with expensive
filtering mechanisms.

Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9708
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur