glusterfs.git/xlators/experimental/jbr-server/src/gen-fops.py, branch v4.1.6

core: remove experimental xlators and associated tests

2018-05-09T17:41:37+00:00

experimental xlators removed from 4.1

Signed-off-by: ShyamsundarR 
Change-Id: I34419ce22ca09b7626b8f9382c377a614fd9fed8
Updates: bz#1575386

core/various: python3 compat, prepare for python2 -> python3

2018-05-02T11:28:46+00:00

see https://review.gluster.org/#/c/19788/

use print fn from __future__

Change-Id: If5075d8d9ca9641058fbc71df8a52aa35804cda4
updates: #411
Signed-off-by: Kaleb S. KEITHLEY

core/build/various: python3 compat, prepare for python2 -> python3

2018-04-12T11:04:27+00:00

Note 1) we're not supposed to be using #!/usr/bin/env python, see
https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines

Note 2) we're also not supposed to be using "!/usr/bin/python,
see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out

The previous patch (https://review.gluster.org/19767) tried to do too
much in one patch, so it was abandoned.

This patch does two things:
1) minor cleanup of configure(.ac) to explicitly use python2
2) change all the shebang lines to #!/usr/bin/python2 and add them
where they were missing based on warnings emitted during rpmbuild.

In a follow-up patch python2 will eventually be changed to python3.

Before that python2-isms (e.g. print, string.join(), etc.) need to be
converted to python3. Some of those can be rewritten in version agnostic
python. E.g. print statements become print() with "from __future_ import
print_function". The python 2to3 utility will be used for some of those.
Also Aravinda has given guidance in the comments to the first patch for
changes.

updates: #411
Change-Id: I471730962b2526022115a1fc33629fb078b74338
Signed-off-by: Kaleb S. KEITHLEY

jbr: Sending rollback from failed fop to fdl

2016-11-08T19:25:25+00:00

In case of a failed fop, the failure is detected
by the leader in the jbr-server in two places. First
during a quorum check of +ve responses when it
receives responses from all the followers. At this
point if the fop hasn't been successfully journaled
at a quorum of followers (as in there is no merit in
trying the fop in the leader as the quorum will never
be met), then we fail the fop.

Also if this quorum is met, then the fop is tried on
the leader, and after the leader completes the fop
a quorum check similar to the previous one is done
again, this time including the leaders outcome. If
quorum is not met, then we fail the fop.

In both these cases, when the fop fails we send a -ve
ack to the client. With this patch, now we will also
send a rollback through a GF_FOP_IPC to all the followers(and
also to the leader in the second case of failure). This
rollback will contain the index and term number of the
fop which failed. This will be recorded in the respective
journals of the bricks and will be used to rollback the
fop on that brick later.

A subsequent write, and it's respective rollback would
look something like the following in the journal.

The trusted.jbr.term and trusted.jbr.index present in the
dict of both the logs, relate them, and the presence of
"rollback-fop" in the dict of IPC indicates that it is a
rollback fop, and the value 13(stands for GF_FOP_WRITE)
indicates what kind of rollback operation it is.

=== GF_FOP_WRITE
fd = 
vector = <158 bytes>
offset = 0 (0x0)
flags = 32769 (0x8001)
xdata = dict {
 trusted.jbr.term = 0 <2 bytes>
 trusted.jbr.index = 4 <2 bytes>
}
=== GF_FOP_IPC
xdata = dict {
 trusted.jbr.term = 0 <2 bytes>
 trusted.jbr.index = 4 <2 bytes>
 rollback-fop = 13 <3 bytes>
}

Change-Id: I70b6a143d20697153d58e2f719e34ecd1ed160a5
BUG: 1349385
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/14783
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System

jbr/locking: Define path for lock/unlock fops in JBR

2016-05-26T15:27:43+00:00

lock/unlock fops need to be handled differently than
other 'regular' fops, so as to avoid chances of deadlock
in blocking calls. This patch addresses the same in the
following manner, with a caveat.

1. On receiving the fop if the node is a follower, it
   performs the operation (irrespective of it being
   lock/unlock fop), and returns the result.

2. If the node is a leader it follows the following paths
   for lock and unlock fops:

   For lock fops :
   -> It performs the fop on itself. If it is a failure, it
      sends -ve ack to the client. If it is successful, it
      dispatches the fop to the followers.
   -> On receiving responses from the followers, it checks
      for quorum (including the leader's outcome). If
      quorum is met, it sends +ve ack to the client.
   -> If quorum is not met, then it *should* issue a rollback
      to the followers, followed by the rollback on the leader.
      It should then send -ve ack to he client.

   For unlock fops:
   -> It dispatches the fop on the followers first.
   -> On receiving responses from the followers, it performs
      the fop on itself. On completion, it checks for quorum
      (including the leader's outcome). If quorum is met, it
      sends +ve ack to the client.
   -> If quorum is not met, then it *should* issue a rollback
      on itslef, followed by the rollback on the followers.
      It should then send -ve ack to he client.

Caveat:
   -> jbr-server does not have a rollback framework yet,
      and hence this patch does not perform the rollbacks as
      discussed in the failure scenarios above. The rollback
      framework will be a different dependent patch.

Change-Id: I26961b27cb85f324c1ffeee80e82ec082ffa4465
BUG: 1333370
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/14226
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

jbr: Making fop functions more modular to reuse more code

2016-05-25T18:52:44+00:00

Putting bigger chunks of re-usable code like leader checks
and init into functions thereby reducing the size of the
'fop' call.

Introduced 'perform_local_op' in the 'fop' call, where
regular functions as of now just call dispatch, but fops
like 'lk' can do their fop specific operations.

Introduced selective_generate to allow certain functions
for a particular fop to be generated. The rest of the
functions can be customised and added in jbr.c

Change-Id: I3754ed68983e763329e14a2faef911428e36e4f0
BUG: 1336328
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/14355
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

nsr/jbr: Renaming nsr to jbr

2016-04-25T19:05:35+00:00

As per community consensus, we have decided to rename
nsr to jbr(Journal-Based-Replication). This is the patch
to rename the "nsr" code to "jbr"

Change-Id: Id2a9837f2ec4da89afc32438b91a1c302bb4104f
BUG: 1328043
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/13899
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy