glusterfs.git/rpc/rpc-lib/src/rpc-clnt-ping.c, branch v5.7 Land part 2 of clang-format changes 2018-09-12T12:22:45+00:00 Gluster Ant bugzilla-bot@gluster.org 2018-09-12T12:22:45+00:00 e16868dede6455cab644805af6fe1ac312775e13 Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4 Signed-off-by: Nigel Babu <nigelb@redhat.com> 
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
rpc/rpc-lib/src/rpc-clnt-ping.c:move to GF_MALLOC() instead of GF_CALLOC() when 2018-08-23T09:29:02+00:00 Yaniv Kaul ykaul@redhat.com 2018-08-21T20:34:52+00:00 cd95c072f4bc13c5c920aae359e8e9b84725ea2c It doesn't make sense to calloc (allocate and clear) memory when the code right away fills that memory with data. It may be optimized by the compiler, or have a microscopic performance improvement. Please review carefully, especially for string allocation, with the terminating NULL string. Only compile-tested! Change-Id: Ifb30412ddf1bfa509f52e0454454929b266e5658 updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> 
It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.

Please review carefully, especially for string allocation, with the
terminating NULL string.

Only compile-tested!

Change-Id: Ifb30412ddf1bfa509f52e0454454929b266e5658
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
cleanup: xlator_t structure's 'client_latency' variable is not used 2018-03-19T03:30:31+00:00 Sven Fischer sven@fischer-abc.de 2018-03-17T22:30:38+00:00 de52876407040a8cae9656ede9f66084d4cb45ac - Removed unused struct member and its one time usage. - cleaned up wrong white space member 'client_latency' was not used otherwise since it was added by commit 07cc8679cdf3b29680f4f105d0222da168d8bfc1 Author: Kevin Vigor <kvigor@fb.com> Date: Tue Mar 21 08:23:25 2017 -0700 Halo Replication feature for AFR translator Change-Id: Ibb0ea828d4090bbe8897f6af326b317884162a00 BUG: 1495153 Signed-off-by: Sven Fischer <sven@fischer-abc.de> 
  - Removed unused struct member and its one time usage.
  - cleaned up wrong white space

member 'client_latency' was not used otherwise since it was added by

commit 07cc8679cdf3b29680f4f105d0222da168d8bfc1
Author: Kevin Vigor <kvigor@fb.com>
Date:   Tue Mar 21 08:23:25 2017 -0700

    Halo Replication feature for AFR translator

Change-Id: Ibb0ea828d4090bbe8897f6af326b317884162a00
BUG: 1495153
Signed-off-by: Sven Fischer <sven@fischer-abc.de>
rpc: Eliminate conn->lock contention by using more granular locks 2017-11-28T13:02:06+00:00 Mohit Agrawal moagrawa@redhat.com 2017-08-14T04:45:45+00:00 04fb94c99160ae1c6dea02624fefd47eb48da810 rpc_clnt_submit() acquires conn->lock before call to rpc_transport_submit_request() and subsequent queuing of frame into saved_frames list. However, as part of handling RPC_TRANSPORT_MSG_RECEIVED and RPC_TRANSPORT_MSG_SENT notifications in rpc_clnt_notify(), conn->lock is again used to atomically update conn->last_received and conn->last_sent event timestamps. So when conn->lock is acquired as part of submitting a request, a parallel POLLIN notification gets blocked at rpc layer until the request submission completes and the lock is released. To get around this, this patch call clock_gettime (instead to call gettimeofday) to update event timestamps in conn->last_received and conn->last_sent and to call clock_gettime don't need to call mutex_lock because it (clock_gettime) is thread safe call. Note: Run fio on vm after apply the patch, iops is improved after apply the patch. Change-Id: I347b5031d61c426b276bc5e07136a7172645d763 BUG: 1467614 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> 
rpc_clnt_submit() acquires conn->lock  before call to
rpc_transport_submit_request() and subsequent queuing of frame into
saved_frames list. However, as part of handling RPC_TRANSPORT_MSG_RECEIVED
and RPC_TRANSPORT_MSG_SENT notifications in rpc_clnt_notify(), conn->lock
is again used to atomically update conn->last_received and conn->last_sent
event timestamps.

So when conn->lock is acquired as part of submitting a request,
a parallel POLLIN notification gets blocked at rpc layer until the request
submission completes and the lock is released.

To get around this, this patch call clock_gettime (instead to call gettimeofday)
to update event timestamps in conn->last_received and conn->last_sent and to
call clock_gettime don't need to call mutex_lock because it (clock_gettime)
is thread safe call.

Note: Run fio on vm after apply the patch, iops is improved after apply
      the patch.

Change-Id: I347b5031d61c426b276bc5e07136a7172645d763
BUG: 1467614
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
stack: change gettimeofday() to clock_gettime() 2017-11-06T14:23:49+00:00 Amar Tumballi amarts@redhat.com 2017-09-25T11:14:06+00:00 125fc934e7f4c669f67e5eec5b0e3dae3a2b6d72 For achieving the above, needed below changes too. * more sanity into how 'frame->op' is assigned. * infra to have 'stats' as separate section in 'xlator_t' structure Updates #137 Change-Id: I36679bf9577f3ed00a695b4e7d92870dcb3db8e1 Signed-off-by: Amar Tumballi <amarts@redhat.com> 
For achieving the above, needed below changes too.

* more sanity into how 'frame->op' is assigned.
* infra to have 'stats' as separate section in 'xlator_t' structure

Updates #137

Change-Id: I36679bf9577f3ed00a695b4e7d92870dcb3db8e1
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Halo Replication feature for AFR translator 2017-05-02T10:23:53+00:00 Kevin Vigor kvigor@fb.com 2017-03-21T15:23:25+00:00 07cc8679cdf3b29680f4f105d0222da168d8bfc1 Summary: Halo Geo-replication is a feature which allows Gluster or NFS clients to write locally to their region (as defined by a latency "halo" or threshold if you like), and have their writes asynchronously propagate from their origin to the rest of the cluster. Clients can also write synchronously to the cluster simply by specifying a halo-latency which is very large (e.g. 10seconds) which will include all bricks. In other words, it allows clients to decide at mount time if they desire synchronous or asynchronous IO into a cluster and the cluster can support both of these modes to any number of clients simultaneously. There are a few new volume options due to this feature: halo-shd-latency: The threshold below which self-heal daemons will consider children (bricks) connected. halo-nfsd-latency: The threshold below which NFS daemons will consider children (bricks) connected. halo-latency: The threshold below which all other clients will consider children (bricks) connected. halo-min-replicas: The minimum number of replicas which are to be enforced regardless of latency specified in the above 3 options. If the number of children falls below this threshold the next best (chosen by latency) shall be swapped in. New FUSE mount options: halo-latency & halo-min-replicas: As descripted above. This feature combined with multi-threaded SHD support (D1271745) results in some pretty cool geo-replication possibilities. Operational Notes: - Global consistency is gaurenteed for synchronous clients, this is provided by the existing entry-locking mechanism. - Asynchronous clients on the other hand and merely consistent to their region. Writes & deletes will be protected via entry-locks as usual preventing concurrent writes into files which are undergoing replication. Read operations on the other hand should never block. - Writes are allowed from _any_ region and propagated from the origin to all other regions. The take away from this is care should be taken to ensure multiple writers do not write the same files resulting in a gfid split-brain which will require resolution via split-brain policies (majority, mtime & size). Recommended method for preventing this is using the nfs-auth feature to define which region for each share has RW permissions, tiers not in the origin region should have RO perms. TODO: - Synchronous clients (including the SHD) should choose clients from their own region as preferred sources for reads. Most of the plumbing is in place for this via the child_latency array. - Better GFID split brain handling & better dent type split brain handling (i.e. create a trash can and move the offending files into it). - Tagging in addition to latency as a means of defining which children you wish to synchronously write to Test Plan: - The usual suspects, clang, gcc w/ address sanitizer & valgrind - Prove tests Reviewers: jackl, dph, cjh, meyering Reviewed By: meyering Subscribers: ethanr Differential Revision: https://phabricator.fb.com/D1272053 Tasks: 4117827 Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1 BUG: 1428061 Signed-off-by: Kevin Vigor <kvigor@fb.com> Reviewed-on: http://review.gluster.org/16099 Reviewed-on: https://review.gluster.org/16177 Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> 
Summary:
Halo Geo-replication is a feature which allows Gluster or NFS clients to write
locally to their region (as defined by a latency "halo" or threshold if you
like), and have their writes asynchronously propagate from their origin to the
rest of the cluster.  Clients can also write synchronously to the cluster
simply by specifying a halo-latency which is very large (e.g. 10seconds) which
will include all bricks.

In other words, it allows clients to decide at mount time if they desire
synchronous or asynchronous IO into a cluster and the cluster can support both
of these modes to any number of clients simultaneously.

There are a few new volume options due to this feature:
  halo-shd-latency:  The threshold below which self-heal daemons will
  consider children (bricks) connected.

  halo-nfsd-latency: The threshold below which NFS daemons will consider
  children (bricks) connected.

  halo-latency: The threshold below which all other clients will
  consider children (bricks) connected.

  halo-min-replicas: The minimum number of replicas which are to
  be enforced regardless of latency specified in the above 3 options.
  If the number of children falls below this threshold the next
  best (chosen by latency) shall be swapped in.

New FUSE mount options:
  halo-latency & halo-min-replicas: As descripted above.

This feature combined with multi-threaded SHD support (D1271745) results in
some pretty cool geo-replication possibilities.

Operational Notes:
- Global consistency is gaurenteed for synchronous clients, this is provided by
  the existing entry-locking mechanism.
- Asynchronous clients on the other hand and merely consistent to their region.
  Writes & deletes will be protected via entry-locks as usual preventing
  concurrent writes into files which are undergoing replication.  Read operations
  on the other hand should never block.
- Writes are allowed from _any_ region and propagated from the origin to all
  other regions.  The take away from this is care should be taken to ensure
  multiple writers do not write the same files resulting in a gfid split-brain
  which will require resolution via split-brain policies (majority, mtime &
  size).  Recommended method for preventing this is using the nfs-auth feature to
  define which region for each share has RW permissions, tiers not in the origin
  region should have RO perms.

TODO:
- Synchronous clients (including the SHD) should choose clients from their own
  region as preferred sources for reads.  Most of the plumbing is in place for
  this via the child_latency array.
- Better GFID split brain handling & better dent type split brain handling
  (i.e. create a trash can and move the offending files into it).
- Tagging in addition to latency as a means of defining which children you wish
  to synchronously write to

Test Plan:
- The usual suspects, clang, gcc w/ address sanitizer & valgrind
- Prove tests

Reviewers: jackl, dph, cjh, meyering

Reviewed By: meyering

Subscribers: ethanr

Differential Revision: https://phabricator.fb.com/D1272053

Tasks: 4117827

Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1
BUG: 1428061
Signed-off-by: Kevin Vigor <kvigor@fb.com>
Reviewed-on: http://review.gluster.org/16099
Reviewed-on: https://review.gluster.org/16177
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
socket: socket disconnect should wait for poller thread exit 2016-12-22T04:49:19+00:00 Rajesh Joseph rjoseph@redhat.com 2016-12-13T09:58:42+00:00 af6769675acbbfd780fa2ece8587502d6d579372 When SSL is enabled or if "transport.socket.own-thread" option is set then socket_poller is run as different thread. Currently during disconnect or PARENT_DOWN scenario we don't wait for this thread to terminate. PARENT_DOWN will disconnect the socket layer and cleanup resources used by socket_poller. Therefore before disconnect we should wait for poller thread to exit. Change-Id: I71f984b47d260ffd979102f180a99a0bed29f0d6 BUG: 1404181 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/16141 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> 
When SSL is enabled or if "transport.socket.own-thread" option is set
then socket_poller is run as different thread. Currently during
disconnect or PARENT_DOWN scenario we don't wait for this thread
to terminate. PARENT_DOWN will disconnect the socket layer and
cleanup resources used by socket_poller.

Therefore before disconnect we should wait for poller thread to exit.

Change-Id: I71f984b47d260ffd979102f180a99a0bed29f0d6
BUG: 1404181
Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-on: http://review.gluster.org/16141
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
remove unused variables 2016-05-06T09:31:31+00:00 Prasanna Kumar Kalever prasanna.kalever@redhat.com 2015-08-17T05:46:27+00:00 db610ee85447de413e56b7cee26175fcb6dbb04a Cleaning dead initializations. Change-Id: I53ae506593cd10441d61df3b2c9249544a7871f7 BUG: 1254067 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/11932 Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> 
Cleaning dead initializations.

Change-Id: I53ae506593cd10441d61df3b2c9249544a7871f7
BUG: 1254067
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-on: http://review.gluster.org/11932
Tested-by: Prasanna Kumar Kalever <pkalever@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
rpc-clnt: Check for transport object during rpc connection cleanup 2015-08-08T10:50:24+00:00 Soumya Koduri skoduri@redhat.com 2015-08-06T00:52:38+00:00 6e8c137fb7deca75817d617d40ebd58bbcae186a While doing glfs_fini(), all the xlators are first notified of PARENT_DOWN. protocol-client xlator on receving that notification does rpc_clnt_disable which disassociates rpc->conn with its transport object and does socket shutdown. So any further references to conn->trans should not happen during rpc connection cleanup which is done mainly as part of epoll event handling of EPOLLERR/EPOLLHUP. BUG: 1250797 Change-Id: I7d0346451d40372ad59a6a2326176fbb1e694712 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/11845 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> 
While doing glfs_fini(), all the xlators are first notified of
PARENT_DOWN. protocol-client xlator on receving that notification
does rpc_clnt_disable which disassociates rpc->conn with its
transport object and does socket shutdown. So any further references
to conn->trans should not happen during rpc connection cleanup which
is done mainly as part of epoll event handling of EPOLLERR/EPOLLHUP.

BUG: 1250797
Change-Id: I7d0346451d40372ad59a6a2326176fbb1e694712
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-on: http://review.gluster.org/11845
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
rpc: fix possible deadlock left behind in d448fd1 2015-06-17T07:53:00+00:00 Krishnan Parthasarathi kparthas@redhat.com 2015-05-19T09:31:08+00:00 e902df70f8157db4db503b7ec3c2635b08b3dcb2 See http://review.gluster.org/9613 for more details. Change-Id: I05ac0267b8c6f4e9b354acbbdf5469835455fb10 Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/10821 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> 
See http://review.gluster.org/9613 for more details.

Change-Id: I05ac0267b8c6f4e9b354acbbdf5469835455fb10
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/10821
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>