diff options
| author | Susant Palai <spalai@redhat.com> | 2016-08-04 12:23:40 +0530 |
|---|---|---|
| committer | Niels de Vos <ndevos@redhat.com> | 2016-08-10 03:21:54 -0700 |
| commit | 7b17f93685d080676089302ef005b8c182335adf (patch) | |
| tree | 68ebaf6ffa32bbfd80415db8e657e5b166a0066c /xlators | |
| parent | d5976e2ca90f16074216a32e267e2652acd32bd9 (diff) | |
dht/rebalance: allocate migrator thread pool dynamically
Problems: The maximum number of migratior threads created was static set
to "40". And the number of these threads get created in rebalance depends
on the number of cores user has. If the number of cores exceeds 40, a
crash or memory corruption can be seen.
Fix: Make the migratior thread pool dynamic.
> Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
> BUG: 1362069
> Reviewed-on: http://review.gluster.org/15000
> Smoke: Gluster Build System <jenkins@build.gluster.org>
> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
(cherry picked from commit b8e8bfc7e4d3eaf76bb637221bc6392ec10ca54b)
Change-Id: Ifbdac8a1a396363dd75e2f6bcb454070cfdbf839
BUG: 1362069
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/15061
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Diffstat (limited to 'xlators')
| -rw-r--r-- | xlators/cluster/dht/src/dht-rebalance.c | 17 |
1 files changed, 14 insertions, 3 deletions
diff --git a/xlators/cluster/dht/src/dht-rebalance.c b/xlators/cluster/dht/src/dht-rebalance.c index 5f3b5daf31b..77f6ec2f823 100644 --- a/xlators/cluster/dht/src/dht-rebalance.c +++ b/xlators/cluster/dht/src/dht-rebalance.c @@ -21,7 +21,6 @@ #define GF_DISK_SECTOR_SIZE 512 #define DHT_REBALANCE_PID 4242 /* Change it if required */ #define DHT_REBALANCE_BLKSIZE (128 * 1024) -#define MAX_MIGRATOR_THREAD_COUNT 40 #define MAX_MIGRATE_QUEUE_COUNT 500 #define MIN_MIGRATE_QUEUE_COUNT 200 @@ -3550,7 +3549,7 @@ gf_defrag_start_crawl (void *data) int thread_index = 0; int err = 0; int thread_spawn_count = 0; - pthread_t tid[MAX_MIGRATOR_THREAD_COUNT]; + pthread_t *tid = NULL; gf_boolean_t is_tier_detach = _gf_false; this = data; @@ -3687,7 +3686,8 @@ gf_defrag_start_crawl (void *data) gf_dht_mt_container_t); if (!defrag->queue) { - gf_log (this->name, GF_LOG_INFO, "No memory for queue"); + gf_log (this->name, GF_LOG_ERROR, "No memory for " + "queue"); ret = -1; goto out; } @@ -3699,6 +3699,15 @@ gf_defrag_start_crawl (void *data) gf_msg_debug (this->name, 0, "thread_spawn_count: %d", thread_spawn_count); + tid = GF_CALLOC (thread_spawn_count, sizeof (pthread_t), + gf_common_mt_pthread_t); + if (!tid) { + gf_log (this->name, GF_LOG_ERROR, "Insufficient memory " + "for tid"); + ret = -1; + goto out; + } + defrag->current_thread_count = thread_spawn_count; /*Spawn Threads Here*/ @@ -3788,6 +3797,8 @@ out: pthread_join (tid[i], NULL); } + GF_FREE (tid); + if (defrag->cmd == GF_DEFRAG_CMD_START_TIER) { /* Wait for the tier fixlayout to * complete if its was started.*/ |