| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.
Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs
This change although big, is just moving around the headers and
making it correct when including these headers from other sources.
This helps us correctly include libglusterfs includes without
namespace conflicts.
Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: At the time of processing GF_EVENT_PARENT_DOWN
at brick xlator, it forwards the event to next xlator
only while xlator ensures no stub is in progress.
At io-thread xlator it decreases stub_cnt before the process
a stub and notify EVENT to next xlator
Solution: Introduce a new counter to save stub_cnt and decrease
the counter after process the stub completely at io-thread
xlator.
To avoid brick crash at the time of call xlator_mem_cleanup
move only brick xlator if detach brick name has found in
the graph
Note: Thanks to pranith for sharing a simple reproducer to
reproduce the same
fixes bz#1637934
Change-Id: I1a694a001f7a5417e8771e3adf92c518969b6baa
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Current resource cleanup sequence is not
perfect while brick mux is enabled
Solution: 1) Destroying xprt after cleanup all fd associated
with a client
2) Before call fini for brick xlators ensure no stub
should be running on a brick
Change-Id: I86195785e428f57d3ef0da3e4061021fafacd435
fixes: bz#1631357
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
|
|
|
|
| |
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
> Add options to disable new features
> Commit ID: c071992e8d
> https://review.gluster.org/#/c/18291/
> By Michael Goulet <mgoulet@fb.com>
This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: Ice477fdf4b8934f9fac0b4a2f6c93db97429a586
Signed-off-by: Varsha Rao <varao@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
> performance/io-threads: Add watchdog to cover up a possible thread leak
> Commit ID: 8b6804f75c
> https://review.gluster.org/#/c/18239/
> By Shreyas Siravara <sshreyas@fb.com>
This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: Id057c34a2abb9fc6dfb4afcd5c7bbbfe5693bbb8
Signed-off-by: Varsha Rao <varao@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following release-3.8-fb branch patch is upstreamed:
> io-stats: Expose io-thread queue depths
> Commit ID: 69509ee7d2
> https://review.gluster.org/#/c/18143/
> By Shreyas Siravara <sshreyas@fb.com>
Changes in this patch:
- Replace iot_pri_t with gf_fop_pri_t
- Replace IOT_PRI_{HI, LO, NORMAL, MAX, LEAST} with
GF_FOP_PRI_{HI, LO, NORMAL, MAX, LEAST}
- Use dict_unref() instead of dict_destroy()
This patch is required to forward port io-threads namespace patch.
Updates: #401
Change-Id: I1b47a63185a441a30fbc423ca1015df7b36c2518
Signed-off-by: Varsha Rao <varao@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some functions were allocating 64K booleans, which are (crazily) mapped to
4-byte ints, for a total of 256KB per call. Changed to use bitfields instead,
so usage is now only 8KB per call. This was the impediment to changing the
io-threads stack size, so that has been adjusted too.
Change-Id: I8781c4f2c8f2b830f4535e366995fac8dd0a8653
BUG: 1418095
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: https://review.gluster.org/15745
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
io-threads starts the thread in 'init()' but doesn't clean them up
on 'fini()'. It relies on PARENT_DOWN to exit threads but there can
be cases where event before PARENT_UP the graph init code can think
of issuing fini(). This code path is hit when glfs_init() is called
on a volume that is in 'stopped' state. It leads to a crash in ganesha
process, because the io-thread tries to access freed memory.
Fix:
Ideal fix would be to wait for all fops in io-thread list to be completed on
PARENT_DOWN, and have fini() do cleanup of threads. Because there is no proper
documentation about how PARENT_DOWN/fini are supposed to be used,
we are getting different kinds of sequences in different higher level protocols.
So for now cleaning up in both PARENT_DOWN and fini(). Fuse doesn't call fini()
gfapi is not calling PARENT_DOWN in some cases, so for now I don't see
another way out.
BUG: 1396793
Change-Id: I9c9154e7d57198dbaff0f30d3ffc25f6d8088aec
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/15888
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
When glfs_fini() is called on a volume where client.io-threads is enabled,
fini() will free up iothread xl's private structure but there would be some
threads that are sleeping which would wakeup after the timedwait completes
leading to accessing already free'd memory.
Fix:
As part of parent-down, exit all sleeping threads.
BUG: 1381830
Change-Id: I0bb8d90241112c355fb22ee3fbfd7307f475b339
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/15620
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the full "queue of queues" approach where each client gets its
own queue (per priority) and we round-robin among them.
Change-Id: I73955d1b9bb93f2ff781b48dfe2509009c519ec6
BUG: 1360402
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/14904
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will be unnecessary, and mostly in the way, as real fairness
guarantees are implemented.
Change-Id: Ic61ec1c9e9add58385f1a4eafcfe2cc554ceefc8
BUG: 1360402
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/14989
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: Gluster Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).
When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.
BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The 'least-rate-limit' io-threads translator option enables
throttling of least priority operations. This is initially intended
as a debug/diagnostic tool for users who might experience
overloaded servers via background activity (i.e., self-heal).
least-rate-limit defines the maximum number of least priority
operations the io-threads translator will dequeue in one second.
If the specified rate limit is met, the worker threads sleep for
the minimal amount of time before the next least priority operation
becomes available (or until a new request arrives).
The requests/second metric is generic and relative to a variety of
factors involved with a background operation (server, storage,
etc.). The most recent measured rate ("cached least rate") is added
to the io-threads state dump content (kill -USR1) to serve as a
reference point to throttle background activity under particular
conditions.
Change-Id: I80f2282992137d57b1becaa5c6ae3858c066862a
BUG: 853680
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/4119
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ib9b9eab2d2e90fce21ef32f0f884101938978859
BUG: 843377
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3736
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
RCA:
In cases when self-heal is in progress, self-heal fops are starved
because of least-priority. This affects other fops with conflicting
inode, entry locks with self-heal.
Fix:
This patch provides configuring enable/disable of least-priority.
Additional changes:
Moved RCHECKSUM fop to low instead of least because it will still
affect the performance of other fops if RCHECKSUM is in LEAST
priority.
Tests:
Tested that the enabling/disabling of fops is working fine.
Tested that RCHECKSUM fop priority is assigned LOW when
least-priority is disabled.
Change-Id: I8d72872b33e2ac7e1ad3afb27582682b0eb98a80
BUG: 843704
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3743
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that the license was not changed in any of the following:
.../argp-standalone/...
.../booster/...
.../cli/...
.../contrib/...
.../extras/...
.../glusterfsd/...
.../glusterfs-hadoop/...
.../mod_clusterfs/...
.../scheduler/...
.../swift/...
The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server
Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.
If you find any errors or ommissions please don't hesitate to let me know.
The complete list of files with the license change is:
libglusterfs/src/byte-order.h
libglusterfs/src/call-stub.c
libglusterfs/src/call-stub.h
libglusterfs/src/checksum.c
libglusterfs/src/checksum.h
libglusterfs/src/circ-buff.c
libglusterfs/src/circ-buff.h
libglusterfs/src/common-utils.c
libglusterfs/src/common-utils.h
libglusterfs/src/compat-errno.c
libglusterfs/src/compat-errno.h
libglusterfs/src/compat.c
libglusterfs/src/compat.h
libglusterfs/src/daemon.c
libglusterfs/src/daemon.h
libglusterfs/src/defaults.c
libglusterfs/src/defaults.h
libglusterfs/src/dict.c
libglusterfs/src/dict.h
libglusterfs/src/event-history.c
libglusterfs/src/event-history.h
libglusterfs/src/event.c
libglusterfs/src/event.h
libglusterfs/src/fd-lk.c
libglusterfs/src/fd-lk.h
libglusterfs/src/fd.c
libglusterfs/src/fd.h
libglusterfs/src/gf-dirent.c
libglusterfs/src/gf-dirent.h
libglusterfs/src/globals.c
libglusterfs/src/globals.h
libglusterfs/src/glusterfs.h
libglusterfs/src/graph-print.c
libglusterfs/src/graph-utils.h
libglusterfs/src/graph.c
libglusterfs/src/hashfn.c
libglusterfs/src/hashfn.h
libglusterfs/src/iatt.h
libglusterfs/src/inode.c
libglusterfs/src/inode.h
libglusterfs/src/iobuf.c
libglusterfs/src/iobuf.h
libglusterfs/src/latency.c
libglusterfs/src/latency.h
libglusterfs/src/list.h
libglusterfs/src/lkowner.h
libglusterfs/src/locking.h
libglusterfs/src/logging.c
libglusterfs/src/logging.h
libglusterfs/src/mem-pool.c
libglusterfs/src/mem-pool.h
libglusterfs/src/mem-types.h
libglusterfs/src/options.c
libglusterfs/src/options.h
libglusterfs/src/rbthash.c
libglusterfs/src/rbthash.h
libglusterfs/src/run.c
libglusterfs/src/run.h
libglusterfs/src/scheduler.c
libglusterfs/src/scheduler.h
libglusterfs/src/stack.c
libglusterfs/src/stack.h
libglusterfs/src/statedump.c
libglusterfs/src/statedump.h
libglusterfs/src/syncop.c
libglusterfs/src/syncop.h
libglusterfs/src/syscall.c
libglusterfs/src/syscall.h
libglusterfs/src/timer.c
libglusterfs/src/timer.h
libglusterfs/src/trie.c
libglusterfs/src/trie.h
libglusterfs/src/xlator.c
libglusterfs/src/xlator.h
libglusterfsclient/src/libglusterfsclient-dentry.c
libglusterfsclient/src/libglusterfsclient-internals.h
libglusterfsclient/src/libglusterfsclient.c
libglusterfsclient/src/libglusterfsclient.h
rpc/rpc-lib/src/auth-glusterfs.c
rpc/rpc-lib/src/auth-null.c
rpc/rpc-lib/src/auth-unix.c
rpc/rpc-lib/src/protocol-common.h
rpc/rpc-lib/src/rpc-clnt.c
rpc/rpc-lib/src/rpc-clnt.h
rpc/rpc-lib/src/rpc-transport.c
rpc/rpc-lib/src/rpc-transport.h
rpc/rpc-lib/src/rpcsvc-auth.c
rpc/rpc-lib/src/rpcsvc-common.h
rpc/rpc-lib/src/rpcsvc.c
rpc/rpc-lib/src/rpcsvc.h
rpc/rpc-lib/src/xdr-common.h
rpc/rpc-lib/src/xdr-rpc.c
rpc/rpc-lib/src/xdr-rpc.h
rpc/rpc-lib/src/xdr-rpcclnt.c
rpc/rpc-lib/src/xdr-rpcclnt.h
rpc/rpc-transport/rdma/src/name.c
rpc/rpc-transport/rdma/src/name.h
rpc/rpc-transport/rdma/src/rdma.c
rpc/rpc-transport/rdma/src/rdma.h
rpc/rpc-transport/socket/src/name.c
rpc/rpc-transport/socket/src/name.h
rpc/rpc-transport/socket/src/socket.c
rpc/rpc-transport/socket/src/socket.h
xlators/cluster/afr/src/afr-common.c
xlators/cluster/afr/src/afr-dir-read.c
xlators/cluster/afr/src/afr-dir-read.h
xlators/cluster/afr/src/afr-dir-write.c
xlators/cluster/afr/src/afr-dir-write.h
xlators/cluster/afr/src/afr-inode-read.c
xlators/cluster/afr/src/afr-inode-read.h
xlators/cluster/afr/src/afr-inode-write.c
xlators/cluster/afr/src/afr-inode-write.h
xlators/cluster/afr/src/afr-lk-common.c
xlators/cluster/afr/src/afr-mem-types.h
xlators/cluster/afr/src/afr-open.c
xlators/cluster/afr/src/afr-self-heal-algorithm.c
xlators/cluster/afr/src/afr-self-heal-algorithm.h
xlators/cluster/afr/src/afr-self-heal-common.c
xlators/cluster/afr/src/afr-self-heal-common.h
xlators/cluster/afr/src/afr-self-heal-data.c
xlators/cluster/afr/src/afr-self-heal-entry.c
xlators/cluster/afr/src/afr-self-heal-metadata.c
xlators/cluster/afr/src/afr-self-heal.h
xlators/cluster/afr/src/afr-self-heald.c
xlators/cluster/afr/src/afr-self-heald.h
xlators/cluster/afr/src/afr-transaction.c
xlators/cluster/afr/src/afr-transaction.h
xlators/cluster/afr/src/afr.c
xlators/cluster/afr/src/afr.h
xlators/cluster/afr/src/pump.c
xlators/cluster/afr/src/pump.h
xlators/cluster/dht/src/dht-common.c
xlators/cluster/dht/src/dht-common.h
xlators/cluster/dht/src/dht-diskusage.c
xlators/cluster/dht/src/dht-hashfn.c
xlators/cluster/dht/src/dht-helper.c
xlators/cluster/dht/src/dht-inode-read.c
xlators/cluster/dht/src/dht-inode-write.c
xlators/cluster/dht/src/dht-layout.c
xlators/cluster/dht/src/dht-linkfile.c
xlators/cluster/dht/src/dht-mem-types.h
xlators/cluster/dht/src/dht-rebalance.c
xlators/cluster/dht/src/dht-rename.c
xlators/cluster/dht/src/dht-selfheal.c
xlators/cluster/dht/src/dht.c
xlators/cluster/dht/src/nufa.c
xlators/cluster/dht/src/switch.c
xlators/cluster/stripe/src/stripe-helpers.c
xlators/cluster/stripe/src/stripe-mem-types.h
xlators/cluster/stripe/src/stripe.c
xlators/cluster/stripe/src/stripe.h
xlators/features/index/src/index-mem-types.h ¹
xlators/features/index/src/index.c ¹
xlators/features/index/src/index.h ¹
xlators/performance/io-cache/src/io-cache.c
xlators/performance/io-cache/src/io-cache.h
xlators/performance/io-cache/src/ioc-inode.c
xlators/performance/io-cache/src/ioc-mem-types.h
xlators/performance/io-cache/src/page.c
xlators/performance/io-threads/src/io-threads.c
xlators/performance/io-threads/src/io-threads.h
xlators/performance/io-threads/src/iot-mem-types.h
xlators/performance/md-cache/src/md-cache-mem-types.h
xlators/performance/md-cache/src/md-cache.c
xlators/performance/quick-read/src/quick-read-mem-types.h
xlators/performance/quick-read/src/quick-read.c
xlators/performance/quick-read/src/quick-read.h
xlators/performance/read-ahead/src/page.c
xlators/performance/read-ahead/src/read-ahead-mem-types.h
xlators/performance/read-ahead/src/read-ahead.c
xlators/performance/read-ahead/src/read-ahead.h
xlators/performance/symlink-cache/src/symlink-cache.c
xlators/performance/write-behind/src/write-behind-mem-types.h
xlators/performance/write-behind/src/write-behind.c
xlators/protocol/auth/addr/src/addr.c ¹
xlators/protocol/auth/login/src/login.c ¹
xlators/protocol/client/src/client-callback.c
xlators/protocol/client/src/client-handshake.c
xlators/protocol/client/src/client-helpers.c
xlators/protocol/client/src/client-lk.c
xlators/protocol/client/src/client-mem-types.h
xlators/protocol/client/src/client.c
xlators/protocol/client/src/client.h
xlators/protocol/client/src/client3_1-fops.c
¹ Copyright only, license reverted to original
Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: Idc0db1cb2c26dcb0b36a58d4d43f2da10cde62c5
BUG: 812515
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/3186
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
|
| |
Change-Id: I7b4e7c467b833bc5896808e6e1d1b1a0322c4fdb
BUG: 3483
Reviewed-on: http://review.gluster.com/318
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I2d10f2be44f518f496427f257988f1858e888084
BUG: 3348
Reviewed-on: http://review.gluster.com/200
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I3914467611e573cccee0d22df93920cf1b2eb79f
BUG: 3348
Reviewed-on: http://review.gluster.com/182
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
|
|
|
|
|
|
|
|
| |
Signed-off-by: Anand Avati <avati@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 2241 (GlusterFs Stat Actions Degrade During I/O)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=2241
|
|
|
|
|
|
|
|
| |
Signed-off-by: Kaushik BV <kaushikbv@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 1159 ()
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=1159
|
|
|
|
|
|
|
|
| |
Signed-off-by: Vijay Bellur <vijay@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 971 (dynamic volume management)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=971
|
|
|
|
|
|
|
|
| |
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Signed-off-by: Vijay Bellur <vijay@dev.gluster.com>
BUG: 1388 ()
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=1388
|
|
|
|
|
|
|
|
|
|
|
| |
Memory accounting Changes. Thanks to Vinayak Hegde and Csaba Henk for their
contributions.
Signed-off-by: Vijay Bellur <vijay@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 329 (Replacing memory allocation functions with mem-type functions)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=329
|
|
|
|
|
|
|
|
|
|
|
|
| |
* conditional for scaling up threads was wrong
* ETIMEDOUT check was performed wrongly
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 583 (filesystem access hangs while deleting large files)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=583
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch lets io-threads work with a single queue and multiple
threads work on picking the next request from the queue and process
it.
Whenever the number of pending requests in the queue double, a new
worker thread is spawned.
Workers expire after a (configurable) timeout of inactivity
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 583 (filesystem access hangs while deleting large files)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=583
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We're performing a calculation for skewing idle time
that resulted in a timespec.tv_nsec value becoming larger than
1000 million or less than 0, forcing sem_timedwait to return
with an EINVAL instead of waiting for a request notification
from sem_post in iot_notify_worker(). This resulted in a missed
notification that resulted in a hang followed by a timeout
on the protocol/client side.
This commit avoids the over- and under-flow in tv_nsec by
skewing the tv_sec value instead.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It seems that use of mutexes is resulting in pretty high thread
sleep and wake-up cost. What is worse, if a worker thread has
acquired a lock, there is a possibility of the main glusterfs thread
being put to sleep. We change the use of mutexes into spinlock.
At the same time, we cannot anymore use condvars for notification since
the condvar interface depends on mutexes itself. Semaphores come to
out rescue. Luckily, even the pthread semaphores have a timedwait
interface to allow our idle worker threads to make an exit decision.
Further, it is possible that spinlocks are not available on all systems
so all this is curtained behind #defines so we can fall back to
mutexes and condvars implementation.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
| |
We've had complaints from users who've used
autoscaling option with default settings for min and
max threads, about high memory consumption because of
the large default value for max-threads.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit brings in support for allocation of iot_request_t's
in io-threads through the use of the mem-pool. We're hoping
that the overheads of hundreds and thousands of small allocations
can be avoided through this.
The important point to note is that the memory pool is not
for the translator as a whole but there is one small memory
pool for each worker thread. Not only does that help us
avoid malloc overheads for small allocations like iot_request_t
but also avoid contention on the heap data structures when multiple
threads want an iot_request_t from the pool.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch cleans up io-threads behaviour regarding the
range values that can be specified for min-threads
and max-threads. THe major change is that the min threads
have been reduced to 2 to signify that io-threads needs minimum
two threads for its operation, while keeping the default number of
threads at 16. The idea is to decouple the default thread count
from the minimum thread count.
Note to Avati:
This applies over Raghu's indentation and logging take-3 patch.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
Going by the memory usage for each threads, it is prudent to
have lower number of threads by default and let users who understand
the memory consequences increase the thread count for themselves.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
The default stack size on Linux is around 8 MiB for each
thread. This is clearly too high for our purpose. This commit reduces
the stack size down to 1 MiB.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit finally makes the autoscaling feature visible to the user.
Know that we're now using two separate thread-pools, one for data
requests, called ordered thread-pool in io-threads, and the other
for meta-data requests, called un-ordered thread-pool.
We do not expose this information to the user to keep io-threads
simple. Consequently, when the user specifies a min-threads and
max-threads value, the number of threads assigned to each pool
is equal, i.e. both pools start with their min threads set to half of
the option "min-threads" and both scale up their threads at most up to
half of option "max-threads".
Volfile options will be added to the wiki and user-guide.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
The default is also to provide no scaling. For both, ordered and
unordered request pools, when scaling is off, we maintain atleast the
minimum number of threads specified in the volfile.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Now we have the remaining fops going through the ordered
thread-pool.
To route a request through ordered thread, we use
iot_schedule_ordered(..) and the worker thread for
ordered requests is iot_worker_ordered(..)
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds everything needed to:
a. Get un-ordered request going through the un-ordered
thread-pool. This happens through, the
iot_schedule_unordered(..). The unordered thread-pool
consists of thread running the iot_worker_unordered(..)
function.
b. Make threads in the un-ordered thread pool start-up
and exit depending on the thread state.
Note that at this point the requests that need
ordering are still going through iot_schedule(..).
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
| |
Worker threads were represented as a list in iot_conf_t
which made us traverse the list of workers in order to
decide which thread gets the request. Now we represent the
workers as a dynamically allocated array so that we can just index
into the array to schedule the file.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
This patch changes the per-thread request queue from a custom circular
linked list, into the standard list.h list which is easier to
understand and has a cleaner interface.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
| |
updated copyright header to include 2009.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
iot_queue() and iot_dequeue() functions were using a io-threads
translator-wide lock which would be contended for by every worker
thread waiting for IO requests.
This patch reduces the granularity by turning the
lock into a per-worker lock.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
| |
Since we're not dependent on this io-thread internal state(i.e.
cache_size and current_size) to rate limit requests, we can remove
these two data members and code that checks for these.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|