glusterfs.git/xlators/mgmt/glusterd/src/glusterd-syncop.h, branch v3.8.8

glusterd : allocate peerid to store in frame->cookie

2015-05-28T07:09:43+00:00

commit a1de3b05 was using peerid from the stack and storing it in the
frame->cookie and in the subsequent callback it was referred. The existance of
this variable is not guranteed in the cbk since its not dynamically allocated.
Fix is to dynmacially manage peerid in the frame cookie.

This patch also fixes one problem in gd_sync_task_begin () where unlock is not
triggered if the cluster is running with lesser than 3.6 op-version resulting into
commands failing with another transaction is in progress.

Change-Id: I0d22cf663df53ef3769585703944577461061312
BUG: 1223213
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/10842
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

glusterd: populate errstr if gd_syncop_submit_request fails

2015-05-20T09:30:28+00:00

Change-Id: Ie4a3edef5d553fc07de53b46f9485c46a4305245
BUG: 1219732
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/10659
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Replace transaction peers lists

2015-04-13T06:30:02+00:00

Transaction peer lists were used in GlusterD to peers belonging to a
transaction. This was needed to prevent newly added peers performing
partial transactions, which could be incorrect.

This was accomplished by creating a seperate transaction peers list at
the beginning of every transaction. A transaction peers list referenced
the peerinfo data structures of the peers which were present at the
beginning of the transaction. RCU protection of peerinfos referenced by
the transaction peers list is a hard problem and difficult to do
correctly.

To have proper RCU protection of peerinfos, the transaction peers lists
have been replaced by an alternative method to identify peers that
belong to a transaction. The alternative method is to the global peers
list along with generation numbers to identify peers that should belong
to a transaction.

This change introduces a global peer list generation number, and a
generation number for each peerinfo object. Whenever a peerinfo object
is created, the global generation number is bumped, and the peerinfos
generation number is set to the bumped global generation.

With the above changes, the algorithm to identify peers belonging to a
transaction with RCU protection is as follows,
- At the beginning of a transaction, the current global generation
  number is saved
- To identify if a peers belonging to the transaction,
  - Start a RCU read critical section
  - For each peer in the global peers list,
    - If the peers generation number is not greater than the saved
      generation number, continue with the action on the peer
  - End the RCU read critical section

The above algorithm guarantees that,
- The peer list is not modified when a transaction is iterating through
  it
- The transaction actions are only done on peers that were present when
  the transaction started

But, as a transaction could iterate over the peers list multiple times,
the algorithm cannot guarantee that same set of peers will be selected
every time. A peer could get deleted between two iterations of the list
within a transaction. This problem existed with transaction peers list
as well, but unlike before now it will not lead to invalid memory access
and potential crashes. This problem will be addressed seprately.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  52ded5b Add timespec_cmp
  44aedd8 Add create timestamp to peerinfo
  7bcbea5 Fix some silly mistakes
  13e3241 Add start time to opinfo
  17a6727 Use timestamp comparisions to identify xaction peers instead
          of a xaction peer list
  3be05b6 Correct check for peerinfo age
  70d5b58 Use read-critical sections for peer list iteration
  ba4dbca Use peerinfo timestamp checks in op-sm instead of xaction peer
          list
  d63f811 Add more peer status checks when iterating peers list in
          glusterd-syncop
  1998a2a Timestamp based peer list traversal of mgmtv3 xactions
  f3c1a42 Remove transaction peer lists
  b8b08ee Remove unused labels
  32e5f5b Remove 'npeers' usage
  a075fb7 Remove 'npeers' from mgmt-v3 framework
  12c9df2 Use generation number instead of timestamps.
  9723021 Remove timespec_cmp
  80ae2c6 Remove timespec.h include
  a9479b0 Address review comments on 10147/4

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I9be1033525c0a89276f5b5d83dc2eb061918b97f
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10147
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Maintain local xaction_peer list for op-sm

2015-03-26T07:10:45+00:00

http://review.gluster.org/9269 addresses maintaining local xaction_peers in
syncop and mgmt_v3 framework. This patch is to maintain local xaction_peers list
for op-sm framework as well.

Change-Id: Idd8484463fed196b3b18c2df7f550a3302c6e138
BUG: 1204727
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/9972
Reviewed-by: Anand Nekkunti 
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

glusterd: Replace libglusterfs lists with liburcu lists

2015-03-04T07:50:22+00:00

This patch replaces usage of the libglusterfs lists data structures and
API in glusterd with the lists data structures and API from liburcu. The
liburcu data structes and APIs are a drop-in replacement for
libglusterfs lists.

All usages have been changed to keep the code consistent, and free from
confusion.

NOTE: glusterd_conf_t->xprt_list still uses the libglusterfs data
structures and API, as it holds rpc_transport_t objects, which is not a
part of glusterd and is not being changed in this patch.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  6dac576 Replace libglusterfs lists with liburcu lists
  a51b5ab Fix compilation issues
  d98a06f Fix merge issues
  a5d918e Remove merge remnant
  1cca113 More style cleanup
  1917be3 Address review comments on 9624/1
  8d10f13 Use cds_lists for glusterd_svc_t
  524ad5d Add rculist header in glusterd-conn-helper.c
  646f294 glusterd: add list_add_order API honouring rcu

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: Ic613c5b6e496a677b9d3de15fc042a0492109fb0
BUG: 1191030
Signed-off-by: Kaushal M 
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/9624
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Gaurav Kumar Garg 
Reviewed-by: Anand Nekkunti

glusterd: Maintain per transaction xaction_peers list in syncop & mgmt_v3

2014-12-23T04:14:14+00:00

In current implementation xaction_peers list is maintained in a global variable
(glustrd_priv_t) for syncop/mgmt_v3. This means consistency and atomicity of
peerinfo list across transactions is not guranteed when multiple syncop/mgmt_v3
transaction are going through.

We had got into a problem in mgmt_v3-locks.t which was failing spuriously, the
reason for that was two volume set operations (in two different volume) was
going through simultaneouly and both of these transaction were manipulating the
same xaction_peers structure which lead to a corrupted list. Because of which in
some cases unlock request to peer was never triggered and we end up with having
stale locks.

Solution is to maintain a per transaction local xaction_peers list for every
syncop.

Please note I've identified this problem in op-sm area as well and a separate
patch will be attempted to fix it.

Finally thanks to Krishnan Parthasarathi and Kaushal M for your constant help to
get to the root cause.

Change-Id: Ib1eaac9e5c8fc319f4e7f8d2ad965bc1357a7c63
BUG: 1173414
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/9269
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

glusterd: print the peer name instead of a null UUID in a rpc failure message

2014-10-10T06:33:31+00:00

This patch improves the failure message by printing the correct peer name
instead of a blank uuid in case of rpc connection is lost/broken.

Change-Id: Ia232792051f23896883b239982cb48130e3ce60e
BUG: 1146902
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/8597
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Kaushal M

gluster: GlusterFS Volume Snapshot Feature

2014-04-11T23:29:17+00:00

This is the initial patch for the Snapshot feature. Current patch
includes following features:
* Snapshot create
* Snapshot delete
* Snapshot restore
* Snapshot list
* Snapshot info
* Snapshot status
* Snapshot config

Change-Id: I2f46920c0d61c515f6a60e0f8b46fff886d9f6a9
BUG: 1061685
Signed-off-by: shishir gowda 
Signed-off-by: Sachin Pandit 
Signed-off-by: Vijaikumar M 
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Rajesh Joseph 
Signed-off-by: Joseph Fernandes 
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/7128
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd-syncop: Fix unlocking and collating errors

2013-06-04T20:27:35+00:00

* Only those peers which were locked need to be unlocked.
* Fix location of collating errors in callbacks. The callback functions
  could miss collating errors if there was an rpc error.

Change-Id: Ie27c2f1ec197da4f5077a4d6e032127954ce87cd
BUG: 948686
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/5087
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Gluster Build System

Revert "glusterd: Fix spurious wakeups in glusterd syncops"

2013-05-05T04:35:52+00:00

This reverts commit efa154bb0a4cac34d5a9610ec25d38eebe495f22.

-- Following is Avati's analysis (edited) from gerrit --
The claim of the patch (being reverted) is that it in some cases cbkfn
is missed.  This is wrong analysis. cbk_fn is _always_ called. The patch
treats ret > 0 as a "missed cbk". ret > 0 only means socket submission
was not complete, and is queued to submit asynchronously when POLLOUT is
raised.  This is sufficient to guarantee that cbkfn is going to be
called (either the socket errors or submission succeeds and reply
eventually arrives).

This commit also removes spurious barrier_wake(s).

call backs are guaranteed to be called even if the transport is
disconnected. This means, a 'wake' would be called if rpc_clnt_submit is
called.  Also, we count both successful and failed operations in a
particular batch of operations for the synctask_barrier_wait.  So,
calling synctask_barrier_wake on failure of rpc_clnt_submit (say, due to
network failure) would result in a spurious wake.

Change-Id: I7d508c2a54b74a65b82f097742206bc777afc53a
BUG: 948686
Signed-off-by: Krishnan Parthasarathi 
Reviewed-on: http://review.gluster.org/4922
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati