glusterfs.git/xlators/mgmt/glusterd/src/glusterd-sm.h, branch v3.8.5

glusterd: Add a new event to handle multi-net probes

2016-03-29T04:43:35+00:00

This allows GlusterD to send updates to all other nodes when attaching
new addresses using multi-net peer probe.

Change-Id: I62846be750ab3721912e7b49656594347ea61723
BUG: 1320458
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/13817
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

glusterd: Use generation number to find peerinfo in RPC notifications

2015-05-07T07:10:43+00:00

The generation number for each peerinfo object is unique. It can be used
to find the exact peerinfo object, which is required for peer RPC
notifications.

Using hostname and uuid matching to find peerinfos can return incorrect
peerinfos to be returned in certain cases like multi network peer probe.
This could cause updates to happen to incorrect peerinfos.

Change-Id: Ia0aada8214fd6d43381e5afd282e08d53a277251
BUG: 1215018
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10495
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Atin Mukherjee

glusterd: Use uint32_t for peerinfo generation number

2015-04-30T08:23:55+00:00

Using a uint64_t for the peerinfo generation number was overkill for how
the generation number is used within GlusterD. It also prevented
GlusterD from running on 32-bit architechtures, as uatomic_add_return
doesn't support 64-bit values on 32-bit architechtures.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  b78dba4 Use 32-bit generation number
  2c37e4b Change other generation number variables to uint32_t

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I0f310f56a4fb97d6bcbc23255a379ed5bb1ed9e1
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10425
Reviewed-by: Anand Nekkunti 
Tested-by: Gluster Build System 
Reviewed-by: Emmanuel Dreyfus 
Tested-by: Emmanuel Dreyfus 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Replace transaction peers lists

2015-04-13T06:30:02+00:00

Transaction peer lists were used in GlusterD to peers belonging to a
transaction. This was needed to prevent newly added peers performing
partial transactions, which could be incorrect.

This was accomplished by creating a seperate transaction peers list at
the beginning of every transaction. A transaction peers list referenced
the peerinfo data structures of the peers which were present at the
beginning of the transaction. RCU protection of peerinfos referenced by
the transaction peers list is a hard problem and difficult to do
correctly.

To have proper RCU protection of peerinfos, the transaction peers lists
have been replaced by an alternative method to identify peers that
belong to a transaction. The alternative method is to the global peers
list along with generation numbers to identify peers that should belong
to a transaction.

This change introduces a global peer list generation number, and a
generation number for each peerinfo object. Whenever a peerinfo object
is created, the global generation number is bumped, and the peerinfos
generation number is set to the bumped global generation.

With the above changes, the algorithm to identify peers belonging to a
transaction with RCU protection is as follows,
- At the beginning of a transaction, the current global generation
  number is saved
- To identify if a peers belonging to the transaction,
  - Start a RCU read critical section
  - For each peer in the global peers list,
    - If the peers generation number is not greater than the saved
      generation number, continue with the action on the peer
  - End the RCU read critical section

The above algorithm guarantees that,
- The peer list is not modified when a transaction is iterating through
  it
- The transaction actions are only done on peers that were present when
  the transaction started

But, as a transaction could iterate over the peers list multiple times,
the algorithm cannot guarantee that same set of peers will be selected
every time. A peer could get deleted between two iterations of the list
within a transaction. This problem existed with transaction peers list
as well, but unlike before now it will not lead to invalid memory access
and potential crashes. This problem will be addressed seprately.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  52ded5b Add timespec_cmp
  44aedd8 Add create timestamp to peerinfo
  7bcbea5 Fix some silly mistakes
  13e3241 Add start time to opinfo
  17a6727 Use timestamp comparisions to identify xaction peers instead
          of a xaction peer list
  3be05b6 Correct check for peerinfo age
  70d5b58 Use read-critical sections for peer list iteration
  ba4dbca Use peerinfo timestamp checks in op-sm instead of xaction peer
          list
  d63f811 Add more peer status checks when iterating peers list in
          glusterd-syncop
  1998a2a Timestamp based peer list traversal of mgmtv3 xactions
  f3c1a42 Remove transaction peer lists
  b8b08ee Remove unused labels
  32e5f5b Remove 'npeers' usage
  a075fb7 Remove 'npeers' from mgmt-v3 framework
  12c9df2 Use generation number instead of timestamps.
  9723021 Remove timespec_cmp
  80ae2c6 Remove timespec.h include
  a9479b0 Address review comments on 10147/4

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I9be1033525c0a89276f5b5d83dc2eb061918b97f
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10147
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

build: make contrib/uuid dependency optional

2015-04-10T11:39:52+00:00

On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.

libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.

Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.

Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Remove compilation warning

2015-03-18T09:56:12+00:00

In glusterd_peerinfo_destroy, cast the passed 'strcut rcu_head *'
pointer to 'gd_rcu_head *' before use in caa_container_of() to prevent
the incompatible-pointer compilation warning.

Also, refactor peerinfo->head to peerinfo->rcu_head to reduce confusion
when reading code.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  aa4a0bc Rename peerinfo->head to peerinfo->rcu_head
  c79144b Cast struct rcu_head * to gd_rcu_head * to prevent warning
  1d222c3 More head -> rcu_head renames

[1]: https://github.com/kshlm/glusterfs/tree/urcu

BUG: 1191030
Change-Id: I7ede02090413839563ce44fdf6289697b28777e7
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/9922
Reviewed-by: Atin Mukherjee 
Tested-by: Gluster Build System

glusterd: Protect the peer list and peerinfos with RCU.

2015-03-16T09:19:14+00:00

The peer list and the peerinfo objects are now protected using RCU.
Design patterns described in the Paul McKenney's RCU dissertation [1]
(sections 5 and 6) have been used to convert existing non-RCU protected
code to RCU protected code.

Currently, we are only targetting guaranteeing the existence of the
peerinfo objects, ie., we are only looking to protect deletes, not all
updaters. We chose this, as protecting all updates is a much more
complex task.

The steps used to accomplish this are,

1. Remove all long lived direct references to peerinfo objects (apart
from the peerinfo list). This includes references in glusterd_peerctx_t
(RPC), glusterd_friend_sm_event_t (friend state machine) and others.
This way no one has a reference to deleted peerinfo object.

2. Replace the direct references with indirect references, ie., use
peer uuid and peer hostname as indirect references to the peerinfo
object. Any reader or updater now uses the indirect references to get to
the actual peerinfo object, using glusterd_peerinfo_find. Cases where a
peerinfo cannot be found are handled gracefully.

3. The readers get and use the peerinfo object only within a RCU read
critical section. This prevents the object from being deleted/freed when
in actual use.

4. The deletion of a peerinfo object is done in a ordered manner
(glusterd_peerinfo_destroy). The object is first removed from the
peerinfo list using an atomic list remove, but the list head is not
reset to allow existing list readers to complete correctly. We wait for
readers to complete, before resetting the list head. This removes the
object from the list completely. After this no new readers can get a
reference to the object, and it can be freed.

This change was developed on the git branch at [2]. This commit is a
combination of the following commits on the development branch.
  d7999b9 Protect the glusterd_conf_t->peers_list with RCU.
  0da85c4 Synchronize before INITing peerinfo list head after removing
          from list.
  32ec28a Add missing rcu_read_unlock
  8fed0b8 Correctly exit read critical section once peer is found.
  63db857 Free peerctx only on rpc destruction
  56eff26 Cleanup style issues
  e5f38b0 Indirection for events and friend_sm
  3c84ac4 In __glusterd_probe_cbk goto unlock only if peer already
          exists
  141d855 Address review comments on 9695/1
  aaeefed Protection during peer updates
  6eda33d Revert "Synchronize before INITing peerinfo list head after
          removing from list."
  f69db96 Remove unneeded line
  b43d2ec Address review comments on 9695/4
  7781921 Address review comments on 9695/5
  eb6467b Add some missing semi-colons
  328a47f Remove synchronize_rcu from
          glusterd_friend_sm_transition_state
  186e429 Run part of glusterd_friend_remove in critical section
  55c0a2e Fix gluster (peer status/ pool list) with no peers
  93f8dcf Use call_rcu to free peerinfo
  c36178c Introduce composite struct, gd_rcu_head

[1]: http://www.rdrop.com/~paulmck/RCU/RCUdissertation.2004.07.14e1.pdf
[2]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: Ic1480e59c86d41d25a6a3d159aa3e11fbb3cbc7b
BUG: 1191030
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/9695
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Replace libglusterfs lists with liburcu lists

2015-03-04T07:50:22+00:00

This patch replaces usage of the libglusterfs lists data structures and
API in glusterd with the lists data structures and API from liburcu. The
liburcu data structes and APIs are a drop-in replacement for
libglusterfs lists.

All usages have been changed to keep the code consistent, and free from
confusion.

NOTE: glusterd_conf_t->xprt_list still uses the libglusterfs data
structures and API, as it holds rpc_transport_t objects, which is not a
part of glusterd and is not being changed in this patch.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  6dac576 Replace libglusterfs lists with liburcu lists
  a51b5ab Fix compilation issues
  d98a06f Fix merge issues
  a5d918e Remove merge remnant
  1cca113 More style cleanup
  1917be3 Address review comments on 9624/1
  8d10f13 Use cds_lists for glusterd_svc_t
  524ad5d Add rculist header in glusterd-conn-helper.c
  646f294 glusterd: add list_add_order API honouring rcu

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: Ic613c5b6e496a677b9d3de15fc042a0492109fb0
BUG: 1191030
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9624
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Gaurav Kumar Garg 
Reviewed-by: Anand Nekkunti

glusterd: Maintain per transaction xaction_peers list in syncop & mgmt_v3

2014-12-23T04:14:14+00:00

In current implementation xaction_peers list is maintained in a global variable
(glustrd_priv_t) for syncop/mgmt_v3. This means consistency and atomicity of
peerinfo list across transactions is not guranteed when multiple syncop/mgmt_v3
transaction are going through.

We had got into a problem in mgmt_v3-locks.t which was failing spuriously, the
reason for that was two volume set operations (in two different volume) was
going through simultaneouly and both of these transaction were manipulating the
same xaction_peers structure which lead to a corrupted list. Because of which in
some cases unlock request to peer was never triggered and we end up with having
stale locks.

Solution is to maintain a per transaction local xaction_peers list for every
syncop.

Please note I've identified this problem in op-sm area as well and a separate
patch will be attempted to fix it.

Finally thanks to Krishnan Parthasarathi and Kaushal M for your constant help to
get to the root cause.

Change-Id: Ib1eaac9e5c8fc319f4e7f8d2ad965bc1357a7c63
BUG: 1173414
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/9269
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M