diff options
author | Varun Shastry <vshastry@redhat.com> | 2013-04-19 12:34:51 +0530 |
---|---|---|
committer | Krishnan Parthasarathi <kparthas@redhat.com> | 2013-08-12 00:48:14 +0530 |
commit | 184e88bfc8f9c6b180c56adfd029e2aaece1297f (patch) | |
tree | 154de47dd6fe5b3e91201ffde27e7e732381aaa9 /xlators/mgmt | |
parent | a1fe3d040a8c9b032cbcb5e831383628cddfa39a (diff) |
features/quota: Improvements to quota
Old implementation
* Client side implementation of quota
- Not secure
- Increased traffic in updating the ctx
New Implementation
* 2 stages of quota implementation is done: Soft and hard quota
Upon reaching soft quota limit on the directory it logs/alerts in the quota
daemon log (ie DEFAULT_LOG_DIR/quotad.log) and no more writes allowed after
hard quota limit. After reaching the soft-limit the daemon alerts the
user/admin repeatively for every 'alert-time', which is configurable.
* Quota is moved to server-side.
There will be 2 quota xlators
i. Quota Server
It takes care of the enforcing the quota and maintains the context
specific to the brick. Since this doesn't have the complete picture of
the cluster, cluster wide usage is updated from the quota daemon. This
updated context is saved and used for the enforcement.
It updates its context by searching the QUOTA_UPDATE_KEY from the dict
in the setxattr call, and is updated from nowhere else.
The quota is always loaded in the server graph and is by passed if the
feature is not enabled.
Options specific to quota-server:
server-quota - Specifies whether the features is on/off. It is used
to by pass the quota if turned off.
deem-statfs - If set to on, it takes quota limits into
consideration while estimating fs size. (df command)
ii. Quota Daemon
This is the new xlator introduced with this patch. Its the
*gluster client* process with no mount point, started upon enabling
quota or restarting the volume. This is a single process for all the
volumes in the cluster. Its volfile stored in
GLUSTERD_DEFAULT_WORKI_DIR/quotad/quotad.vol.
It queries for the sizes on all the bricks, aggregates the size and
sends back the updated size, periodically. The timeout between
successive updation is configurable and typically/by default more for
below-soft-quota usage and less for above-soft-quota usage. It
maintains the timeout inside the limit structure based on the usage;
below soft limit and above soft limit.
There will be thread running per volume which iterates through the list
and decides whether the size to be queried in the current iteration
based on its timeout. It takes the next iteration time taking the least
of the timeouts in the list of entries.
Maintains a separate inode table for each volume in the quotad. In the
first iteration it builds the table for quota-dirs (dirs on which limit
is set) and its components.
Options specific to quotad:
hard-timeout - Timeout for updation of usage to the quota-server
when the usage is crosses the soft-limit.
soft-timeout - Timeout for the updation of usage to the
quota-server when the usage is below soft-limit.
alert-time - Frequency of logging after the usage reached
soft limit.
Options common to both:
default-soft-limit - This is used when individual paths are not
configured with soft-limit and default value of
this option is 90% of the hard-limit.
limit-set - String containing all the limits.
Thus in the current implementation we'll have 2 quota xlators: one in server
graph and one in trusted client (quota daemon) of which the sole
purpose will be to aggregate the quota size xattrs from all the bricks and
send the same to server quota xlator.
* Changes in glusterd and CLI
A single volfile is created for all the volumes, similar to nfs volfile.
All files related to quota client (volfile, pid etc) are stored in
GLUSTERD_DEFAULT_WORK_DIR/quotad/.
The new pattern of the quota limit stores in
limit-set = <single-dir-limit>[,<single-dir-limit>]
single-dir-limit = <abs-path>:<hard-limit>[:<soft-limit-in-percent>]
It also introduces new options:
volume quota <VOLNAME> {enable|disable|list [<path> ...]|remove <path>| default-soft-limit <percent>} |
volume quota <VOLNAME> {limit-usage <path> <size> |soft-limit <path> <percent>} |
volume quota <VOLNAME> {alert-time|soft-timeout|hard-timeout} {<time>}
Credit:
Raghavendra Bhat <rabhat@redhat.com>
Varun Shastry <vshastry@redhat.com>
Shishir Gowda <sgowda@redhat.com>
Kruthika Dhananjay <kdhananj@redhat.com>
Brian Foster <bfoster@redhat.com>
Krishnan Parthasarathi <kparthas@redhat.com>
Change-Id: I16ec5be0c2faaf42b14034b9ccaf17796adef082
BUG: 969461
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Diffstat (limited to 'xlators/mgmt')
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-handler.c | 23 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-quota.c | 720 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-utils.c | 216 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-utils.h | 22 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-volgen.c | 228 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-volgen.h | 2 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-volume-ops.c | 58 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd-volume-set.c | 68 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd.c | 19 | ||||
-rw-r--r-- | xlators/mgmt/glusterd/src/glusterd.h | 10 |
10 files changed, 1117 insertions, 249 deletions
diff --git a/xlators/mgmt/glusterd/src/glusterd-handler.c b/xlators/mgmt/glusterd/src/glusterd-handler.c index a74c5230..ac508f0c 100644 --- a/xlators/mgmt/glusterd/src/glusterd-handler.c +++ b/xlators/mgmt/glusterd/src/glusterd-handler.c @@ -54,6 +54,8 @@ #include <lvm2app.h> #endif +extern glusterd_op_info_t opinfo; + int glusterd_big_locked_notify (struct rpc_clnt *rpc, void *mydata, rpc_clnt_event_t event, void *data, rpc_clnt_notify_t notify_fn) @@ -307,10 +309,23 @@ _build_option_key (dict_t *d, char *k, data_t *v, void *tmp) char reconfig_key[256] = {0, }; struct args_pack *pack = NULL; int ret = -1; + xlator_t *this = NULL; + glusterd_conf_t *priv = NULL; + + this = THIS; + GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); pack = tmp; if (strcmp (k, GLUSTERD_GLOBAL_OPT_VERSION) == 0) return 0; + + if (priv->op_version > GD_OP_VERSION_MIN) { + if ((strcmp (k, "features.limit-usage") == 0) || + (strcmp (k, "features.soft-limit") == 0)) + return 0; + } snprintf (reconfig_key, 256, "volume%d.option.%s", pack->vol_count, k); ret = dict_set_str (pack->dict, reconfig_key, v->data); @@ -3776,6 +3791,7 @@ __glusterd_peer_rpc_notify (struct rpc_clnt *rpc, void *mydata, glusterd_peerinfo_t *peerinfo = NULL; glusterd_peerctx_t *peerctx = NULL; gf_boolean_t quorum_action = _gf_false; + uuid_t uuid; peerctx = mydata; if (!peerctx) @@ -3817,6 +3833,13 @@ __glusterd_peer_rpc_notify (struct rpc_clnt *rpc, void *mydata, glusterd_friend_remove_notify (peerctx); goto out; } + glusterd_get_lock_owner (&uuid); + if (!uuid_is_null (uuid) && + !uuid_compare (peerinfo->uuid, uuid)) { + glusterd_unlock (peerinfo->uuid); + if (opinfo.state.state != GD_OP_STATE_DEFAULT) + opinfo.state.state = GD_OP_STATE_DEFAULT; + } peerinfo->connected = 0; break; diff --git a/xlators/mgmt/glusterd/src/glusterd-quota.c b/xlators/mgmt/glusterd/src/glusterd-quota.c index 31826719..ec4a6eb8 100644 --- a/xlators/mgmt/glusterd/src/glusterd-quota.c +++ b/xlators/mgmt/glusterd/src/glusterd-quota.c @@ -24,6 +24,22 @@ #include <sys/wait.h> + +const char *gd_quota_op_list[GF_QUOTA_OPTION_TYPE_DEFAULT_SOFT_LIMIT+1] = { + [GF_QUOTA_OPTION_TYPE_NONE] = "none", + [GF_QUOTA_OPTION_TYPE_ENABLE] = "enable", + [GF_QUOTA_OPTION_TYPE_DISABLE] = "disable", + [GF_QUOTA_OPTION_TYPE_LIMIT_USAGE] = "limit-usage", + [GF_QUOTA_OPTION_TYPE_REMOVE] = "remove", + [GF_QUOTA_OPTION_TYPE_LIST] = "list", + [GF_QUOTA_OPTION_TYPE_VERSION] = "version", + [GF_QUOTA_OPTION_TYPE_SOFT_LIMIT] = "soft-limit", + [GF_QUOTA_OPTION_TYPE_ALERT_TIME] = "alert-time", + [GF_QUOTA_OPTION_TYPE_SOFT_TIMEOUT] = "soft-timeout", + [GF_QUOTA_OPTION_TYPE_HARD_TIMEOUT] = "hard-timeout", + [GF_QUOTA_OPTION_TYPE_DEFAULT_SOFT_LIMIT] = "default-soft-limit", +}; + int __glusterd_handle_quota (rpcsvc_request_t *req) { @@ -31,15 +47,17 @@ __glusterd_handle_quota (rpcsvc_request_t *req) gf_cli_req cli_req = {{0,}}; dict_t *dict = NULL; glusterd_op_t cli_op = GD_OP_QUOTA; - char operation[256] = {0, }; char *volname = NULL; int32_t type = 0; char msg[2048] = {0,}; xlator_t *this = NULL; + glusterd_conf_t *conf = NULL; GF_ASSERT (req); this = THIS; GF_ASSERT (this); + conf = this->private; + GF_ASSERT (conf); ret = xdr_to_generic (req->msg[0], &cli_req, (xdrproc_t)xdr_gf_cli_req); if (ret < 0) { @@ -82,23 +100,16 @@ __glusterd_handle_quota (rpcsvc_request_t *req) goto out; } - switch (type) { - case GF_QUOTA_OPTION_TYPE_ENABLE: - strncpy (operation, "enable", sizeof (operation)); - break; - - case GF_QUOTA_OPTION_TYPE_DISABLE: - strncpy (operation, "disable", sizeof (operation)); - break; - - case GF_QUOTA_OPTION_TYPE_LIMIT_USAGE: - strncpy (operation, "limit-usage", sizeof (operation)); - break; + if ((conf->op_version == GD_OP_VERSION_MIN) && + (type > GF_QUOTA_OPTION_TYPE_VERSION)) { + snprintf (msg, sizeof (msg), "Cannot execute command. The " + "cluster is operating at version 1. Executing command " + "%s is disallowed in this state", + gd_quota_op_list[type]); + ret = -1; + goto out; + } - case GF_QUOTA_OPTION_TYPE_REMOVE: - strncpy (operation, "remove", sizeof (operation)); - break; - } ret = glusterd_op_begin_synctask (req, GD_OP_QUOTA, dict); out: @@ -132,7 +143,6 @@ glusterd_check_if_quota_trans_enabled (glusterd_volinfo_t *volinfo) } if (flag == _gf_false) { - gf_log ("", GF_LOG_ERROR, "first enable the quota translator"); ret = -1; goto out; } @@ -141,13 +151,21 @@ out: return ret; } -/* At the end of the function, the variable found will be set +/* At the end of the function, the variable @found will be set * to true if the path to be removed was present in the limit-list, * else will be false. + * + * In addition, the function does the following things: + * + * a. places the path to be removed, if found, in @removed_path, + * b. places the new limit list formed after removing @path's entry, in + * @new_list. If @path is not found, the input limit string @quota_limits is + * dup'd as is and placed in @new_list. */ int32_t -_glusterd_quota_remove_limits (char **quota_limits, char *path, - gf_boolean_t *found) +_glusterd_quota_remove_limits (char *quota_limits, char *path, + gf_boolean_t *found, char **new_list, + char **removed_path) { int ret = 0; int i = 0; @@ -158,14 +176,15 @@ _glusterd_quota_remove_limits (char **quota_limits, char *path, int flag = 0; char *limits = NULL; char *qlimits = NULL; + char *rp = NULL; if (found != NULL) *found = _gf_false; - if (*quota_limits == NULL) + if (quota_limits == NULL) return -1; - qlimits = *quota_limits; + qlimits = quota_limits; pathlen = strlen (path); @@ -192,6 +211,15 @@ _glusterd_quota_remove_limits (char **quota_limits, char *path, } else { skiplen = size + 1; size = len - i - size; + if (removed_path) { + rp = GF_CALLOC (skiplen, sizeof (char), gf_gld_mt_char); + if (!rp) { + ret = -1; + goto out; + } + strncpy (rp, &qlimits[i], skiplen - 1); + *removed_path = rp; + } memcpy ((void *) &limits [i], (void *) &qlimits [i + skiplen], size); break; } @@ -200,42 +228,27 @@ _glusterd_quota_remove_limits (char **quota_limits, char *path, size = 0; } - if (!flag) { - ret = 1; - } else { - len = strlen (limits); - - if (len == 0) { - GF_FREE (qlimits); - - *quota_limits = NULL; - - goto out; - } - - if (limits[len - 1] == ',') { - limits[len - 1] = '\0'; - len --; - } - - GF_FREE (qlimits); - - qlimits = GF_CALLOC (len + 1, sizeof (char), gf_gld_mt_char); - - if (!qlimits) { - ret = -1; - goto out; - } - - memcpy ((void *) qlimits, (void *) limits, len + 1); + len = strlen (limits); + if (len == 0) + goto out; - *quota_limits = qlimits; + if (limits[len - 1] == ',') { + limits[len - 1] = '\0'; + len --; + } - ret = 0; + *new_list = GF_CALLOC (len + 1, sizeof (char), gf_gld_mt_char); + if (!*new_list) { + ret = -1; + goto out; } + memcpy ((void *) *new_list, (void *) limits, len + 1); + ret = 0; out: GF_FREE (limits); + if (ret != -1) + ret = flag ? 0 : 1; return ret; } @@ -381,22 +394,30 @@ _glusterd_quota_get_limit_usages (glusterd_volinfo_t *volinfo, int32_t glusterd_quota_get_limit_usages (glusterd_conf_t *priv, - glusterd_volinfo_t *volinfo, - char *volname, - dict_t *dict, - char **op_errstr, + glusterd_volinfo_t *volinfo, char *volname, + dict_t *dict, char **op_errstr, dict_t *rsp_dict) { - int32_t i = 0; - int32_t ret = 0; - int32_t count = 0; - char *path = NULL; - char cmd_str [1024] = {0, }; - char *ret_str = NULL; + int32_t i = 0; + int32_t ret = 0; + int32_t count = 0; + int entry_count = 0; + char *path = NULL; + char cmd_str [1024] = {0, }; + char *ret_str = NULL; + xlator_t *this = NULL; + glusterd_conf_t *conf = NULL; + char *default_limit = NULL; + char *val = NULL; if (rsp_dict == NULL) return 0; + this = THIS; + GF_ASSERT (this); + conf = this->private; + GF_ASSERT (conf); + ret = dict_get_int32 (dict, "count", &count); if (ret < 0) goto out; @@ -404,22 +425,60 @@ glusterd_quota_get_limit_usages (glusterd_conf_t *priv, if (count == 0) { ret_str = _glusterd_quota_get_limit_usages (volinfo, NULL, op_errstr); + ret = dict_set_dynstr (rsp_dict, "limit_list", ret_str); + if (ret) + goto out; } else { i = 0; while (count--) { - snprintf (cmd_str, 1024, "path%d", i++); + snprintf (cmd_str, sizeof (cmd_str), "path%d", i++); ret = dict_get_str (dict, cmd_str, &path); if (ret < 0) goto out; + ret = gf_canonicalize_path (path); + if (ret) { + goto out; + } - ret_str = _glusterd_quota_get_limit_usages (volinfo, path, op_errstr); + ret_str = _glusterd_quota_get_limit_usages (volinfo, + path, + op_errstr); + /* Despite quota limits being absent on @path, we go + * ahead and place it in @rsp_dict with + * value = "Not set". This is because after commit op, + * as part of aggregation of @rsp_dict with @op_ctx, + * when we copy the rsp_dict into op_ctx, op_ctx would + * still be containing the old key (same as @cmd_str) + * with the old value. In order to overwrite the old + * value, we replace it with "Not set", something that + * the cli can easily interpret as a case of quota + * limits not having been set on the given path. + */ + if (!ret_str) { + ret = dict_set_str (rsp_dict, cmd_str, + "Not set"); + } else { + ret = dict_set_dynstr (rsp_dict, cmd_str, + ret_str); + entry_count = entry_count + 1; + } } } + ret = dict_set_int32 (rsp_dict, "entry-count", entry_count); + if (ret) + goto out; - if (ret_str) { - ret = dict_set_dynstr (rsp_dict, "limit_list", ret_str); - } + ret = dict_set_uint32 (rsp_dict, "op-version", conf->op_version); + + ret = glusterd_volinfo_get (volinfo, "features.default-soft-limit", + &default_limit); + if (default_limit) + val = gf_strdup (default_limit); + else + val = gf_strdup ("90%"); + + ret = dict_set_dynstr (rsp_dict, "default-soft-limit", val); out: return ret; } @@ -430,10 +489,14 @@ glusterd_quota_enable (glusterd_volinfo_t *volinfo, char **op_errstr, { int32_t ret = -1; char *quota_status = NULL; + xlator_t *this = NULL; - GF_VALIDATE_OR_GOTO ("glusterd", volinfo, out); - GF_VALIDATE_OR_GOTO ("glusterd", crawl, out); - GF_VALIDATE_OR_GOTO ("glusterd", op_errstr, out); + this = THIS; + GF_ASSERT (this); + + GF_VALIDATE_OR_GOTO (this->name, volinfo, out); + GF_VALIDATE_OR_GOTO (this->name, crawl, out); + GF_VALIDATE_OR_GOTO (this->name, op_errstr, out); if (glusterd_is_volume_started (volinfo) == 0) { *op_errstr = gf_strdup ("Volume is stopped, start volume " @@ -449,15 +512,15 @@ glusterd_quota_enable (glusterd_volinfo_t *volinfo, char **op_errstr, quota_status = gf_strdup ("on"); if (!quota_status) { - gf_log ("", GF_LOG_ERROR, "memory allocation failed"); - *op_errstr = gf_strdup ("Enabling quota has been unsuccessful"); + gf_log (this->name, GF_LOG_ERROR, "memory allocation failed"); + ret = -1; goto out; } - ret = dict_set_dynstr (volinfo->dict, VKEY_FEATURES_QUOTA, quota_status); + ret = dict_set_dynstr (volinfo->dict, VKEY_FEATURES_QUOTA, + quota_status); if (ret) { - gf_log ("", GF_LOG_ERROR, "dict set failed"); - *op_errstr = gf_strdup ("Enabling quota has been unsuccessful"); + gf_log (this->name, GF_LOG_ERROR, "dict set failed"); goto out; } @@ -467,17 +530,34 @@ glusterd_quota_enable (glusterd_volinfo_t *volinfo, char **op_errstr, ret = 0; out: + if (ret && op_errstr && !*op_errstr) + gf_asprintf (op_errstr, "Enabling quota on volume %s has been " + "unsuccessful", volinfo->volname); return ret; } int32_t glusterd_quota_disable (glusterd_volinfo_t *volinfo, char **op_errstr) { - int32_t ret = -1; - char *quota_status = NULL, *quota_limits = NULL; + int32_t ret = -1; + int i = 0; + char *quota_status = NULL; + char *value = NULL; + xlator_t *this = NULL; + glusterd_conf_t *conf = NULL; + char *quota_options[] = {VKEY_FEATURES_LIMIT_USAGE, + "features.soft-timeout", + "features.hard-timeout", + "features.alert-time", + "features.default-soft-limit", NULL}; - GF_VALIDATE_OR_GOTO ("glusterd", volinfo, out); - GF_VALIDATE_OR_GOTO ("glusterd", op_errstr, out); + this = THIS; + GF_ASSERT (this); + conf = this->private; + GF_ASSERT (conf); + + GF_VALIDATE_OR_GOTO (this->name, volinfo, out); + GF_VALIDATE_OR_GOTO (this->name, op_errstr, out); ret = glusterd_check_if_quota_trans_enabled (volinfo); if (ret == -1) { @@ -487,47 +567,84 @@ glusterd_quota_disable (glusterd_volinfo_t *volinfo, char **op_errstr) quota_status = gf_strdup ("off"); if (!quota_status) { - gf_log ("", GF_LOG_ERROR, "memory allocation failed"); - *op_errstr = gf_strdup ("Disabling quota has been unsuccessful"); + gf_log (this->name, GF_LOG_ERROR, "memory allocation failed"); + ret = -1; goto out; } ret = dict_set_dynstr (volinfo->dict, VKEY_FEATURES_QUOTA, quota_status); if (ret) { - gf_log ("", GF_LOG_ERROR, "dict set failed"); - *op_errstr = gf_strdup ("Disabling quota has been unsuccessful"); + gf_log (this->name, GF_LOG_ERROR, "dict set failed"); goto out; } + for (i = 0; quota_options [i]; i++) { + ret = glusterd_volinfo_get (volinfo, quota_options[i], &value); + if (ret) { + gf_log (this->name, GF_LOG_INFO, "failed to get option" + " %s", + quota_options[i]); + } else { + dict_del (volinfo->dict, quota_options[i]); + } + if ((i == 0) && (conf->op_version == GD_OP_VERSION_MIN)) + break; + } + *op_errstr = gf_strdup ("Disabling quota has been successful"); + ret = 0; +out: + if (ret && op_errstr && !*op_errstr) + gf_asprintf (op_errstr, "Disabling quota on volume %s has been " + "unsuccessful", volinfo->volname); + return ret; +} - ret = glusterd_volinfo_get (volinfo, VKEY_FEATURES_LIMIT_USAGE, - "a_limits); - if (ret) { - gf_log ("", GF_LOG_WARNING, "failed to get the quota limits"); +static void +gd_quota_get_formatted_limit_list (char **value, char *existing_list, + char *path, char *hard_limit, + char *soft_limit) +{ + if (!existing_list) { + if (!soft_limit) + gf_asprintf (value, "%s:%s", path, hard_limit); + else + gf_asprintf (value, "%s:%s:%s", path, hard_limit, + soft_limit); } else { - GF_FREE (quota_limits); + if (!soft_limit) + gf_asprintf (value, "%s,%s:%s", existing_list, path, + hard_limit); + else + gf_asprintf (value, "%s,%s:%s:%s", existing_list, path, + hard_limit, soft_limit); } - - dict_del (volinfo->dict, VKEY_FEATURES_LIMIT_USAGE); - -out: - return ret; } int32_t -glusterd_quota_limit_usage (glusterd_volinfo_t *volinfo, dict_t *dict, char **op_errstr) +glusterd_quota_limit_usage (glusterd_volinfo_t *volinfo, dict_t *dict, + char **op_errstr) { - int32_t ret = -1; - char *path = NULL; - char *limit = NULL; - char *value = NULL; - char msg [1024] = {0,}; - char *quota_limits = NULL; + int32_t ret = -1; + char *path = NULL; + char *limit = NULL; + char *value = NULL; + char *sl = NULL; + char msg[5120] = {0,}; + char *quota_limits = NULL; + char *new_list = NULL; + xlator_t *this = NULL; + glusterd_conf_t *priv = NULL; + char *removed_path = NULL; - GF_VALIDATE_OR_GOTO ("glusterd", dict, out); - GF_VALIDATE_OR_GOTO ("glusterd", volinfo, out); - GF_VALIDATE_OR_GOTO ("glusterd", op_errstr, out); + this = THIS; + GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); + + GF_VALIDATE_OR_GOTO (this->name, dict, out); + GF_VALIDATE_OR_GOTO (this->name, volinfo, out); + GF_VALIDATE_OR_GOTO (this->name, op_errstr, out); ret = glusterd_check_if_quota_trans_enabled (volinfo); if (ret == -1) { @@ -539,109 +656,222 @@ glusterd_quota_limit_usage (glusterd_volinfo_t *volinfo, dict_t *dict, char **op ret = glusterd_volinfo_get (volinfo, VKEY_FEATURES_LIMIT_USAGE, "a_limits); if (ret) { - gf_log ("", GF_LOG_ERROR, "failed to get the quota limits"); - *op_errstr = gf_strdup ("failed to set limit"); + gf_log (this->name, GF_LOG_ERROR, "failed to get quota limits"); goto out; } ret = dict_get_str (dict, "path", &path); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to fetch quota limits" ); - *op_errstr = gf_strdup ("failed to set limit"); + gf_log (this->name, GF_LOG_ERROR, "Unable to fetch path"); goto out; } + ret = gf_canonicalize_path (path); + if (ret) + goto out; ret = dict_get_str (dict, "limit", &limit); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to fetch quota limits" ); - *op_errstr = gf_strdup ("failed to set limit"); + gf_log (this->name, GF_LOG_ERROR, "Unable to fetch limit"); goto out; } - if (quota_limits) { - ret = _glusterd_quota_remove_limits ("a_limits, path, NULL); - if (ret == -1) { - gf_log ("", GF_LOG_ERROR, "Unable to allocate memory"); - *op_errstr = gf_strdup ("failed to set limit"); - goto out; - } - } - if (quota_limits == NULL) { - ret = gf_asprintf (&value, "%s:%s", path, limit); - if (ret == -1) { - gf_log ("", GF_LOG_ERROR, "Unable to allocate memory"); - *op_errstr = gf_strdup ("failed to set limit"); - goto out; - } + ; //do nothing and go past the else block } else { - ret = gf_asprintf (&value, "%s,%s:%s", - quota_limits, path, limit); + ret = _glusterd_quota_remove_limits (quota_limits, path, NULL, + &new_list, &removed_path); if (ret == -1) { - gf_log ("", GF_LOG_ERROR, "Unable to allocate memory"); - *op_errstr = gf_strdup ("failed to set limit"); + gf_log (this->name, GF_LOG_ERROR, "Unable to remove " + "limit"); goto out; } - GF_FREE (quota_limits); + if (removed_path && (priv->op_version > GD_OP_VERSION_MIN)) { + ret = gf_get_soft_limit (removed_path, &sl); + if (ret == -1) + goto out; + } } - - quota_limits = value; + gd_quota_get_formatted_limit_list (&value, new_list, path, limit, sl); ret = dict_set_str (volinfo->dict, VKEY_FEATURES_LIMIT_USAGE, - quota_limits); + value); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to set quota limits" ); - *op_errstr = gf_strdup ("failed to set limit"); + gf_log (this->name, GF_LOG_ERROR, "Unable to set quota limits"); goto out; } - snprintf (msg, 1024, "limit set on %s", path); - *op_errstr = gf_strdup (msg); + snprintf (msg, sizeof (msg), "hard limit set on %s", path); + *op_errstr = gf_strdup (msg); ret = 0; out: + GF_FREE (sl); + GF_FREE (removed_path); + GF_FREE (new_list); + + if (ret && op_errstr && !*op_errstr) + gf_asprintf (op_errstr, "Failed to set hard limit on path %s " + "for volume %s", path, volinfo->volname); + return ret; +} + +int +glusterd_quota_soft_limit (glusterd_volinfo_t *volinfo, dict_t *dict, + char **op_errstr) +{ + int ret = 0; + char *path = NULL; + char *limit = NULL; + char *quota_limits = NULL; + char *new_list = NULL; + char *removed_path = NULL; + char msg[1024] = {0,}; + char *hl = NULL; + char *value = NULL; + xlator_t *this = NULL; + + this = THIS; + GF_ASSERT (this); + + if (!volinfo || !dict || !op_errstr) { + ret = -1; + goto out; + } + + ret = glusterd_check_if_quota_trans_enabled (volinfo); + if (ret == -1) { + *op_errstr = gf_strdup ("Quota is disabled, please enable " + "quota"); + goto out; + } + + ret = glusterd_volinfo_get (volinfo, VKEY_FEATURES_LIMIT_USAGE, + "a_limits); + if (ret) { + gf_log (this->name, GF_LOG_ERROR, "failed to get quota limits"); + goto out; + } + + ret = dict_get_str (dict, "path", &path); + if (ret) { + gf_log (this->name, GF_LOG_ERROR, "Unable to fetch path"); + goto out; + } + + ret = gf_canonicalize_path (path); + if (ret) + goto out; + + ret = dict_get_str (dict, "limit", &limit); + if (ret) { + gf_log (this->name, GF_LOG_ERROR, "Unable to fetch limit"); + goto out; + } + + if (quota_limits == NULL) { + gf_asprintf (op_errstr, "Soft-limit cannot be set on path %s, " + "without setting hard-limit on it first.", path); + ret = -1; + goto out; + } else { + ret = _glusterd_quota_remove_limits (quota_limits, path, NULL, + &new_list, &removed_path); + if (ret == -1) { + gf_log (this->name, GF_LOG_ERROR, "Failed to remove " + "limit"); + goto out; + } + + if (!removed_path) { + gf_asprintf (op_errstr, "Soft-limit cannot be set on " + "path %s without setting hard-limit on it " + "first.", path); + ret = -1; + goto out; + } else { + ret = gf_get_hard_limit (removed_path, &hl); + if (ret) + goto out; + } + } + + gd_quota_get_formatted_limit_list (&value, new_list, path, hl, limit); + + ret = dict_set_str (volinfo->dict, VKEY_FEATURES_LIMIT_USAGE, + value); + if (ret) { + gf_log (this->name, GF_LOG_ERROR, "Unable to set quota limits"); + goto out; + } + + snprintf (msg, sizeof (msg), "soft limit set on %s", path); + *op_errstr = gf_strdup (msg); + ret = 0; + +out: + GF_FREE (hl); + GF_FREE (removed_path); + GF_FREE (new_list); + + if (ret && op_errstr && !*op_errstr) + gf_asprintf (op_errstr, "Failed to set soft limit for path %s " + "on volume %s", path, volinfo->volname); return ret; } int32_t -glusterd_quota_remove_limits (glusterd_volinfo_t *volinfo, dict_t *dict, char **op_errstr) +glusterd_quota_remove_limits (glusterd_volinfo_t *volinfo, dict_t *dict, + char **op_errstr) { int32_t ret = -1; char str [PATH_MAX + 1024] = {0,}; char *quota_limits = NULL; + char *new_list = NULL; + char *value = NULL; char *path = NULL; gf_boolean_t flag = _gf_false; + xlator_t *this = NULL; - GF_VALIDATE_OR_GOTO ("glusterd", dict, out); - GF_VALIDATE_OR_GOTO ("glusterd", volinfo, out); - GF_VALIDATE_OR_GOTO ("glusterd", op_errstr, out); + this = THIS; + GF_ASSERT (this); + + GF_VALIDATE_OR_GOTO (this->name, dict, out); + GF_VALIDATE_OR_GOTO (this->name, volinfo, out); + GF_VALIDATE_OR_GOTO (this->name, op_errstr, out); ret = glusterd_check_if_quota_trans_enabled (volinfo); if (ret == -1) { - *op_errstr = gf_strdup ("Quota is disabled, please enable quota"); + *op_errstr = gf_strdup ("Quota is disabled, please enable " + "quota"); goto out; } ret = glusterd_volinfo_get (volinfo, VKEY_FEATURES_LIMIT_USAGE, "a_limits); if (ret) { - gf_log ("", GF_LOG_ERROR, "failed to get the quota limits"); + gf_log (this->name, GF_LOG_ERROR, "failed to get quota limits"); goto out; } ret = dict_get_str (dict, "path", &path); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to fetch quota limits" ); + gf_log (this->name, GF_LOG_ERROR, "Unable to fetch path"); goto out; } - ret = _glusterd_quota_remove_limits ("a_limits, path, &flag); + ret = gf_canonicalize_path (path); + if (ret) + goto out; + + ret = _glusterd_quota_remove_limits (quota_limits, path, &flag, + &new_list, NULL); if (ret == -1) { if (flag == _gf_true) snprintf (str, sizeof (str), "Removing limit on %s has " "been unsuccessful", path); else - snprintf (str, sizeof (str), "%s has no limit set", path); + snprintf (str, sizeof (str), "%s has no limit set", + path); *op_errstr = gf_strdup (str); goto out; } else { @@ -654,11 +884,13 @@ glusterd_quota_remove_limits (glusterd_volinfo_t *volinfo, dict_t *dict, char ** *op_errstr = gf_strdup (str); } - if (quota_limits) { + if (new_list) { + value = gf_strdup (new_list); ret = dict_set_str (volinfo->dict, VKEY_FEATURES_LIMIT_USAGE, - quota_limits); + value); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to set quota limits" ); + gf_log (this->name, GF_LOG_ERROR, "Unable to set quota " + "limits"); goto out; } } else { @@ -668,9 +900,46 @@ glusterd_quota_remove_limits (glusterd_volinfo_t *volinfo, dict_t *dict, char ** ret = 0; out: + GF_FREE (new_list); return ret; } +int +glusterd_set_quota_option (glusterd_volinfo_t *volinfo, dict_t *dict, + char *key, char **op_errstr) +{ + int ret = 0; + char *value = NULL; + xlator_t *this = NULL; + char *option = NULL; + + this = THIS; + GF_ASSERT (this); + + ret = glusterd_check_if_quota_trans_enabled (volinfo); + if (ret == -1) { + gf_asprintf (op_errstr, "Cannot set %s. Quota on volume %s is " + "disabled", key, volinfo->volname); + return -1; + } + + ret = dict_get_str (dict, "value", &value); + if(ret) { + gf_log (this->name, GF_LOG_ERROR, "Option value absent."); + return -1; + } + + option = gf_strdup (value); + ret = dict_set_dynstr (volinfo->dict, key, option); + if(ret) { + gf_log (this->name, GF_LOG_ERROR, "Failed to set option %s", + key); + return -1; + } + gf_asprintf (op_errstr, "%s on volume %s set", key, volinfo->volname); + + return 0; +} int glusterd_op_quota (dict_t *dict, char **op_errstr, dict_t *rsp_dict) @@ -681,26 +950,41 @@ glusterd_op_quota (dict_t *dict, char **op_errstr, dict_t *rsp_dict) int type = -1; gf_boolean_t start_crawl = _gf_false; glusterd_conf_t *priv = NULL; + xlator_t *this = NULL; GF_ASSERT (dict); GF_ASSERT (op_errstr); - priv = THIS->private; + this = THIS; + GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); ret = dict_get_str (dict, "volname", &volname); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to get volume name " ); + gf_log (this->name, GF_LOG_ERROR, "Unable to get volume name"); goto out; } ret = glusterd_volinfo_find (volname, &volinfo); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to allocate memory"); + gf_asprintf (op_errstr, FMTSTR_CHECK_VOL_EXISTS, volname); goto out; } ret = dict_get_int32 (dict, "type", &type); + if ((priv->op_version == GD_OP_VERSION_MIN) && + (type > GF_QUOTA_OPTION_TYPE_VERSION)) { + gf_asprintf (op_errstr, "Volume quota failed. The cluster is " + "operating at version %d. Option %s " + "is disallowed in this state.", + priv->op_version, + gd_quota_op_list[type]); + ret = -1; + goto out; + } + if (type == GF_QUOTA_OPTION_TYPE_ENABLE) { ret = glusterd_quota_enable (volinfo, op_errstr, &start_crawl); if (ret < 0) @@ -708,7 +992,6 @@ glusterd_op_quota (dict_t *dict, char **op_errstr, dict_t *rsp_dict) goto create_vol; } - if (type == GF_QUOTA_OPTION_TYPE_DISABLE) { ret = glusterd_quota_disable (volinfo, op_errstr); if (ret < 0) @@ -742,15 +1025,59 @@ glusterd_op_quota (dict_t *dict, char **op_errstr, dict_t *rsp_dict) } ret = glusterd_quota_get_limit_usages (priv, volinfo, volname, - dict, op_errstr, rsp_dict); + dict, op_errstr, + rsp_dict); goto out; } + + if (type == GF_QUOTA_OPTION_TYPE_SOFT_LIMIT) { + ret = glusterd_quota_soft_limit (volinfo, dict, op_errstr); + if (ret < 0) + goto out; + goto create_vol; + } + + if (type == GF_QUOTA_OPTION_TYPE_SOFT_TIMEOUT) { + ret = glusterd_set_quota_option (volinfo, dict, + "features.soft-timeout", + op_errstr); + if (ret) + goto out; + goto create_vol; + } + + if (type == GF_QUOTA_OPTION_TYPE_HARD_TIMEOUT) { + ret = glusterd_set_quota_option (volinfo, dict, + "features.hard-timeout", + op_errstr); + if (ret) + goto out; + goto create_vol; + } + + if (type == GF_QUOTA_OPTION_TYPE_ALERT_TIME) { + ret = glusterd_set_quota_option (volinfo, dict, + "features.alert-time", + op_errstr); + if (ret) + goto out; + goto create_vol; + } + if (type == GF_QUOTA_OPTION_TYPE_DEFAULT_SOFT_LIMIT) { + ret = glusterd_set_quota_option (volinfo, dict, + "features.default-soft-limit", + op_errstr); + if (ret) + goto out; + goto create_vol; + } + create_vol: ret = glusterd_create_volfiles_and_notify_services (volinfo); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to re-create volfile for" - " 'quota'"); + gf_log (this->name, GF_LOG_ERROR, "Unable to re-create " + "volfiles"); ret = -1; goto out; } @@ -759,20 +1086,31 @@ create_vol: if (ret) goto out; - if (GLUSTERD_STATUS_STARTED == volinfo->status) - ret = glusterd_check_generate_start_nfs (); - + if (GLUSTERD_STATUS_STARTED == volinfo->status) { + if (priv->op_version == GD_OP_VERSION_MIN) + ret = glusterd_check_generate_start_nfs (); + } ret = 0; out: if (rsp_dict && start_crawl == _gf_true) glusterd_quota_initiate_fs_crawl (priv, volname); + if (priv->op_version > GD_OP_VERSION_MIN && + is_origin_glusterd ()) { + if (type != GF_QUOTA_OPTION_TYPE_LIST) { + if (glusterd_all_volumes_with_quota_stopped ()) + ret = glusterd_quotad_stop (); + else + ret = glusterd_check_generate_start_quotad (); + } + } + if (rsp_dict && *op_errstr) { ret = dict_set_dynstr (rsp_dict, "errstr", *op_errstr); if (ret) { GF_FREE (*op_errstr); - gf_log ("", GF_LOG_DEBUG, + gf_log (this->name, GF_LOG_DEBUG, "failed to set error message in ctx"); } *op_errstr = NULL; @@ -789,51 +1127,63 @@ glusterd_op_stage_quota (dict_t *dict, char **op_errstr) gf_boolean_t exists = _gf_false; int type = 0; dict_t *ctx = NULL; + xlator_t *this = NULL; + glusterd_conf_t *priv = NULL; + + this = THIS; + GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); GF_ASSERT (dict); GF_ASSERT (op_errstr); ret = dict_get_str (dict, "volname", &volname); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to get volume name"); + gf_log (this->name, GF_LOG_ERROR, "Unable to get volume name"); goto out; } exists = glusterd_check_volume_exists (volname); if (!exists) { - gf_log ("", GF_LOG_ERROR, "Volume with name: %s " - "does not exist", - volname); - *op_errstr = gf_strdup ("Invalid volume name"); + gf_asprintf (op_errstr, FMTSTR_CHECK_VOL_EXISTS, volname); ret = -1; goto out; } ret = dict_get_int32 (dict, "type", &type); if (ret) { - gf_log ("", GF_LOG_ERROR, "Unable to get 'type' for quota op"); - *op_errstr = gf_strdup ("Volume quota failed, internal error " - ", unable to get type of operation"); + *op_errstr = gf_strdup ("Volume quota failed, internal error, " + "unable to get type of operation"); goto out; } - - ctx = glusterd_op_get_ctx(); - if (ctx && (type == GF_QUOTA_OPTION_TYPE_ENABLE - || type == GF_QUOTA_OPTION_TYPE_LIST)) { - /* Fuse mount req. only for enable & list-usage options*/ - if (!glusterd_is_fuse_available ()) { - gf_log ("glusterd", GF_LOG_ERROR, "Unable to open /dev/" - "fuse (%s), quota command failed", - strerror (errno)); - *op_errstr = gf_strdup ("Fuse unavailable"); - ret = -1; - goto out; - } + if ((priv->op_version == GD_OP_VERSION_MIN) && + (type > GF_QUOTA_OPTION_TYPE_VERSION)) { + gf_asprintf (op_errstr, "Volume quota failed. The cluster is " + "operating at version %d. Option %s " + "is disallowed in this state.", + priv->op_version, + gd_quota_op_list[type]); + ret = -1; + goto out; } -out: - gf_log ("", GF_LOG_DEBUG, "Returning %d", ret); - - return ret; + ctx = glusterd_op_get_ctx(); + if (ctx && (type == GF_QUOTA_OPTION_TYPE_ENABLE + || type == GF_QUOTA_OPTION_TYPE_LIST)) { + /* Fuse mount req. only for enable & list-usage options*/ + if (!glusterd_is_fuse_available ()) { + *op_errstr = gf_strdup ("Fuse unavailable"); + ret = -1; + goto out; + } + } + + out: + if (ret && op_errstr && *op_errstr) + gf_log (this->name, GF_LOG_ERROR, "%s", *op_errstr); + gf_log (this->name, GF_LOG_DEBUG, "Returning %d", ret); + + return ret; } diff --git a/xlators/mgmt/glusterd/src/glusterd-utils.c b/xlators/mgmt/glusterd/src/glusterd-utils.c index bde5b9b5..4a2b9454 100644 --- a/xlators/mgmt/glusterd/src/glusterd-utils.c +++ b/xlators/mgmt/glusterd/src/glusterd-utils.c @@ -3037,6 +3037,7 @@ glusterd_compare_friend_data (dict_t *vols, int32_t *status, char *hostname) gf_boolean_t update = _gf_false; gf_boolean_t stale_nfs = _gf_false; gf_boolean_t stale_shd = _gf_false; + gf_boolean_t stale_qd = _gf_false; GF_ASSERT (vols); GF_ASSERT (status); @@ -3066,6 +3067,8 @@ glusterd_compare_friend_data (dict_t *vols, int32_t *status, char *hostname) stale_nfs = _gf_true; if (glusterd_is_nodesvc_running ("glustershd")) stale_shd = _gf_true; + if (glusterd_is_nodesvc_running ("quotad")) + stale_qd = _gf_true; ret = glusterd_import_global_opts (vols); if (ret) goto out; @@ -3079,6 +3082,8 @@ glusterd_compare_friend_data (dict_t *vols, int32_t *status, char *hostname) glusterd_nfs_server_stop (); if (stale_shd) glusterd_shd_stop (); + if (stale_qd) + glusterd_quotad_stop (); } } @@ -3161,7 +3166,10 @@ glusterd_get_nodesvc_volfile (char *server, char *workdir, GF_ASSERT (len == PATH_MAX); glusterd_get_nodesvc_dir (server, workdir, dir, sizeof (dir)); - snprintf (volfile, len, "%s/%s-server.vol", dir, server); + if (strcmp ("quotad", server) != 0) + snprintf (volfile, len, "%s/%s-server.vol", dir, server); + else + snprintf (volfile, len, "%s/%s.vol", dir, server); } void @@ -3174,11 +3182,14 @@ glusterd_nodesvc_set_online_status (char *server, gf_boolean_t status) GF_ASSERT (priv); GF_ASSERT (priv->shd); GF_ASSERT (priv->nfs); + GF_ASSERT (priv->quotad); if (!strcmp("glustershd", server)) priv->shd->online = status; else if (!strcmp ("nfs", server)) priv->nfs->online = status; + else if (!strcmp ("quotad", server)) + priv->quotad->online = status; } gf_boolean_t @@ -3192,11 +3203,14 @@ glusterd_is_nodesvc_online (char *server) GF_ASSERT (conf); GF_ASSERT (conf->shd); GF_ASSERT (conf->nfs); + GF_ASSERT (conf->quotad); if (!strcmp (server, "glustershd")) online = conf->shd->online; else if (!strcmp (server, "nfs")) online = conf->nfs->online; + else if (!strcmp (server, "quotad")) + online = conf->quotad->online; return online; } @@ -3262,11 +3276,14 @@ glusterd_nodesvc_get_rpc (char *server) GF_ASSERT (priv); GF_ASSERT (priv->shd); GF_ASSERT (priv->nfs); + GF_ASSERT (priv->quotad); if (!strcmp (server, "glustershd")) rpc = priv->shd->rpc; else if (!strcmp (server, "nfs")) rpc = priv->nfs->rpc; + else if (!strcmp (server, "quotad")) + rpc = priv->quotad->rpc; return rpc; } @@ -3284,11 +3301,14 @@ glusterd_nodesvc_set_rpc (char *server, struct rpc_clnt *rpc) GF_ASSERT (priv); GF_ASSERT (priv->shd); GF_ASSERT (priv->nfs); + GF_ASSERT (priv->quotad); if (!strcmp ("glustershd", server)) priv->shd->rpc = rpc; else if (!strcmp ("nfs", server)) priv->nfs->rpc = rpc; + else if (!strcmp ("quotad", server)) + priv->quotad->rpc = rpc; return ret; } @@ -3415,6 +3435,14 @@ glusterd_nodesvc_start (char *server) runner_add_args (&runner, "--xlator-option", glusterd_uuid_option, NULL); } + if (!strcmp (server, "quotad")) { + runner_add_args (&runner, "--xlator-option", + "*replicate*.data-self-heal=off", + "--xlator-option", + "*replicate*.metadata-self-heal=off", + "--xlator-option", + "*replicate*.entry-self-heal=off", NULL); + } runner_log (&runner, "", GF_LOG_DEBUG, "Starting the nfs/glustershd services"); @@ -3438,6 +3466,12 @@ glusterd_shd_start () return glusterd_nodesvc_start ("glustershd"); } +int +glusterd_quotad_start () +{ + return glusterd_nodesvc_start ("quotad"); +} + gf_boolean_t glusterd_is_nodesvc_running (char *server) { @@ -3556,6 +3590,12 @@ glusterd_shd_stop () } int +glusterd_quotad_stop () +{ + return glusterd_nodesvc_stop ("quotad", SIGTERM); +} + +int glusterd_add_node_to_dict (char *server, dict_t *dict, int count, dict_t *vol_opts) { @@ -3705,6 +3745,12 @@ glusterd_reconfigure_shd () } int +glusterd_reconfigure_quotad () +{ + return glusterd_reconfigure_nodesvc (glusterd_create_quotad_volfile); +} + +int glusterd_reconfigure_nfs () { int ret = -1; @@ -3751,21 +3797,54 @@ glusterd_check_generate_start_shd () } int -glusterd_nodesvcs_batch_op (glusterd_volinfo_t *volinfo, - int (*nfs_op) (), int (*shd_op) ()) +glusterd_check_generate_start_quotad () { + int ret = 0; + + ret = glusterd_check_generate_start_service (glusterd_create_quotad_volfile, + glusterd_quotad_stop, + glusterd_quotad_start); + if (ret == -EINVAL) + ret = 0; + return ret; +} + +int +glusterd_nodesvcs_batch_op (glusterd_volinfo_t *volinfo, int (*nfs_op) (), + int (*shd_op) (), int (*qd_op) ()) + { int ret = 0; + xlator_t *this = THIS; + glusterd_conf_t *conf = NULL; + + GF_ASSERT (this); + conf = this->private; + GF_ASSERT (conf); ret = nfs_op (); if (ret) goto out; if (volinfo && !glusterd_is_volume_replicate (volinfo)) - goto out; + goto quotad_op; ret = shd_op (); if (ret) goto out; + +quotad_op: + + if (conf->op_version == GD_OP_VERSION_MIN) + goto out; + + if (is_origin_glusterd ()) { + if (volinfo && !glusterd_is_volume_quota_enabled (volinfo)) + goto out; + ret = qd_op (); + if (ret) + goto out; + } + out: return ret; } @@ -3775,7 +3854,8 @@ glusterd_nodesvcs_start (glusterd_volinfo_t *volinfo) { return glusterd_nodesvcs_batch_op (volinfo, glusterd_nfs_server_start, - glusterd_shd_start); + glusterd_shd_start, + glusterd_quotad_start); } int @@ -3783,7 +3863,8 @@ glusterd_nodesvcs_stop (glusterd_volinfo_t *volinfo) { return glusterd_nodesvcs_batch_op (volinfo, glusterd_nfs_server_stop, - glusterd_shd_stop); + glusterd_shd_stop, + glusterd_quotad_stop); } gf_boolean_t @@ -3829,21 +3910,53 @@ glusterd_all_replicate_volumes_stopped () return _gf_true; } +gf_boolean_t +glusterd_all_volumes_with_quota_stopped () +{ + glusterd_conf_t *priv = NULL; + xlator_t *this = NULL; + glusterd_volinfo_t *voliter = NULL; + + this = THIS; + GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); + + list_for_each_entry (voliter, &priv->volumes, vol_list) { + if (!glusterd_is_volume_quota_enabled (voliter)) + continue; + if (voliter->status == GLUSTERD_STATUS_STARTED) + return _gf_false; + } + + return _gf_true; +} + + int glusterd_nodesvcs_handle_graph_change (glusterd_volinfo_t *volinfo) { int (*shd_op) () = NULL; int (*nfs_op) () = NULL; + int (*qd_op) () = NULL; shd_op = glusterd_check_generate_start_shd; nfs_op = glusterd_check_generate_start_nfs; + qd_op = glusterd_check_generate_start_quotad; if (glusterd_are_all_volumes_stopped ()) { shd_op = glusterd_shd_stop; nfs_op = glusterd_nfs_server_stop; - } else if (glusterd_all_replicate_volumes_stopped()) { - shd_op = glusterd_shd_stop; + qd_op = glusterd_quotad_stop; + } else { + if (glusterd_all_replicate_volumes_stopped()) { + shd_op = glusterd_shd_stop; + } + if (glusterd_all_volumes_with_quota_stopped ()) { + qd_op = glusterd_quotad_stop; + } } - return glusterd_nodesvcs_batch_op (volinfo, nfs_op, shd_op); + + return glusterd_nodesvcs_batch_op (volinfo, nfs_op, shd_op, qd_op); } int @@ -3851,7 +3964,8 @@ glusterd_nodesvcs_handle_reconfigure (glusterd_volinfo_t *volinfo) { return glusterd_nodesvcs_batch_op (volinfo, glusterd_reconfigure_nfs, - glusterd_reconfigure_shd); + glusterd_reconfigure_shd, + glusterd_reconfigure_quotad); } int @@ -5878,6 +5992,82 @@ out: return ret; } +int +glusterd_quotad_statedump (char *options, int option_cnt, char **op_errstr) +{ + int ret = -1; + xlator_t *this = NULL; + glusterd_conf_t *conf = NULL; + char pidfile_path[PATH_MAX] = {0,}; + char path[PATH_MAX] = {0,}; + FILE *pidfile = NULL; + pid_t pid = -1; + char dumpoptions_path[PATH_MAX] = {0,}; + char *option = NULL; + char *tmpptr = NULL; + char *dup_options = NULL; + char msg[256] = {0,}; + + this = THIS; + GF_ASSERT (this); + conf = this->private; + GF_ASSERT (conf); + + dup_options = gf_strdup (options); + option = strtok_r (dup_options, " ", &tmpptr); + if (strcmp (option, "quotad")) { + snprintf (msg, sizeof (msg), "for quotad statedump, options " + "should be after the key 'quotad'"); + *op_errstr = gf_strdup (msg); + ret = -1; + goto out; + } + + GLUSTERD_GET_QUOTAD_DIR (path, conf); + GLUSTERD_GET_QUOTAD_PIDFILE (pidfile_path, path); + + pidfile = fopen (pidfile_path, "r"); + if (!pidfile) { + gf_log (this->name, GF_LOG_ERROR, "Unable to open pidfile: %s", + pidfile_path); + ret = -1; + goto out; + } + + ret = fscanf (pidfile, "%d", &pid); + if (ret <= 0) { + gf_log (this->name, GF_LOG_ERROR, "Unable to get pid of quotad " + "process"); + ret = -1; + goto out; + } + + snprintf (dumpoptions_path, sizeof (dumpoptions_path), + DEFAULT_VAR_RUN_DIRECTORY"/glusterdump.%d.options", pid); + ret = glusterd_set_dump_options (dumpoptions_path, options, option_cnt); + if (ret < 0) { + gf_log (this->name, GF_LOG_ERROR, "error while parsing " + "statedump options"); + ret = -1; + goto out; + } + + gf_log (this->name, GF_LOG_INFO, "Performing statedump on quotad with " + "pid %d", pid); + + kill (pid, SIGUSR1); + + sleep (1); + + ret = 0; +out: + if (pidfile) + fclose (pidfile); + unlink (dumpoptions_path); + GF_FREE (dup_options); + return ret; +} + /* Checks if the given peer contains all the bricks belonging to the * given volume. Returns true if it does else returns false */ @@ -7619,3 +7809,9 @@ out: gf_log ("", GF_LOG_DEBUG, "Returning %d", ret); return ret; } + +int +glusterd_is_volume_quota_enabled (glusterd_volinfo_t *volinfo) +{ + return (glusterd_volinfo_get_boolean (volinfo, VKEY_FEATURES_QUOTA)); +} diff --git a/xlators/mgmt/glusterd/src/glusterd-utils.h b/xlators/mgmt/glusterd/src/glusterd-utils.h index 4d1dde7c..a5590db0 100644 --- a/xlators/mgmt/glusterd/src/glusterd-utils.h +++ b/xlators/mgmt/glusterd/src/glusterd-utils.h @@ -188,6 +188,12 @@ glusterd_shd_start (); int32_t glusterd_shd_stop (); +int32_t +glusterd_quotad_start (); + +int32_t +glusterd_quotad_stop (); + void glusterd_set_socket_filepath (char *sock_filepath, char *sockpath, size_t len); @@ -228,6 +234,9 @@ int glusterd_check_generate_start_shd (void); int +glusterd_check_generate_start_quotad (void); + +int glusterd_nodesvcs_handle_graph_change (glusterd_volinfo_t *volinfo); int @@ -401,8 +410,13 @@ glusterd_brick_statedump (glusterd_volinfo_t *volinfo, char *options, int option_cnt, char **op_errstr); int glusterd_nfs_statedump (char *options, int option_cnt, char **op_errstr); + +int +glusterd_quotad_statedump (char *options, int option_cnt, char **op_errstr); + gf_boolean_t glusterd_is_volume_replicate (glusterd_volinfo_t *volinfo); + gf_boolean_t glusterd_is_brick_decommissioned (glusterd_volinfo_t *volinfo, char *hostname, char *path); @@ -565,4 +579,12 @@ glusterd_check_gsync_running_local (char *master, char *slave, char *conf_path, gf_boolean_t *is_run); +int +glusterd_is_volume_quota_enabled (glusterd_volinfo_t *volinfo); + +gf_boolean_t +glusterd_all_volumes_with_quota_stopped (); + +int +glusterd_reconfigure_quotad (); #endif diff --git a/xlators/mgmt/glusterd/src/glusterd-volgen.c b/xlators/mgmt/glusterd/src/glusterd-volgen.c index 06a2d37d..d0f3d5fa 100644 --- a/xlators/mgmt/glusterd/src/glusterd-volgen.c +++ b/xlators/mgmt/glusterd/src/glusterd-volgen.c @@ -1423,6 +1423,7 @@ server_graph_builder (volgen_graph_t *graph, glusterd_volinfo_t *volinfo, char *vg = NULL; glusterd_brickinfo_t *brickinfo = NULL; char changelog_basepath[PATH_MAX] = {0,}; + char *value = NULL; brickinfo = param; path = brickinfo->path; @@ -1607,7 +1608,18 @@ server_graph_builder (volgen_graph_t *graph, glusterd_volinfo_t *volinfo, if (ret) return -1; - if (dict_get_str_boolean (set_dict, "features.read-only", 0) && + xl = volgen_graph_add (graph, "features/quota", volname); + if (!xl) + return -1; + + ret = glusterd_volinfo_get (volinfo, VKEY_FEATURES_QUOTA, &value); + if (value) { + ret = xlator_set_option (xl, "server-quota", value); + if (ret) + return -1; + } + + if (dict_get_str_boolean (set_dict, "features.read-only", 0) && dict_get_str_boolean (set_dict, "features.worm",0)) { gf_log (THIS->name, GF_LOG_ERROR, "read-only and worm cannot be set together"); @@ -2310,13 +2322,15 @@ out: static int volgen_graph_build_dht_cluster (volgen_graph_t *graph, - glusterd_volinfo_t *volinfo, size_t child_count) + glusterd_volinfo_t *volinfo, size_t child_count, + gf_boolean_t is_quotad) { int32_t clusters = 0; int ret = -1; char *decommissioned_children = NULL; xlator_t *dht = NULL; char *voltype = "cluster/distribute"; + char *name_fmt = NULL; /* NUFA and Switch section */ if (dict_get_str_boolean (volinfo->dict, "cluster.nufa", 0) && @@ -2335,9 +2349,14 @@ volgen_graph_build_dht_cluster (volgen_graph_t *graph, if (dict_get_str_boolean (volinfo->dict, "cluster.switch", 0)) voltype = "cluster/switch"; + if (is_quotad) + name_fmt = "%s"; + else + name_fmt = "%s-dht"; + clusters = volgen_graph_build_clusters (graph, volinfo, voltype, - "%s-dht", + name_fmt, child_count, child_count); if (clusters < 0) @@ -2362,7 +2381,8 @@ out: static int volume_volgen_graph_build_clusters (volgen_graph_t *graph, - glusterd_volinfo_t *volinfo) + glusterd_volinfo_t *volinfo, + gf_boolean_t is_quotad) { char *replicate_args[] = {"cluster/replicate", "%s-replicate-%d"}; @@ -2435,8 +2455,8 @@ build_distribute: } ret = volgen_graph_build_dht_cluster (graph, volinfo, - dist_count); - if (ret == -1) + dist_count, is_quotad); + if (ret) goto out; ret = 0; @@ -2452,25 +2472,30 @@ client_graph_builder (volgen_graph_t *graph, glusterd_volinfo_t *volinfo, xlator_t *xl = NULL; char *volname = NULL; data_t *tmp_data = NULL; + glusterd_conf_t *conf = THIS->private; + GF_ASSERT (conf); volname = volinfo->volname; ret = volgen_graph_build_clients (graph, volinfo, set_dict, param); if (ret) goto out; - ret = volume_volgen_graph_build_clusters (graph, volinfo); - if (ret == -1) - goto out; - - ret = glusterd_volinfo_get_boolean (volinfo, VKEY_FEATURES_QUOTA); - if (ret == -1) + ret = volume_volgen_graph_build_clusters (graph, volinfo, _gf_false); + if (ret) goto out; - if (ret) { - xl = volgen_graph_add (graph, "features/quota", volname); - if (!xl) { - ret = -1; + if (conf->op_version == GD_OP_VERSION_MIN) { + ret = glusterd_volinfo_get_boolean (volinfo, + VKEY_FEATURES_QUOTA); + if (ret == -1) goto out; + if (ret) { + xl = volgen_graph_add (graph, "features/quota", + volname); + if (!xl) { + ret = -1; + goto out; + } } } @@ -2740,6 +2765,60 @@ nfs_option_handler (volgen_graph_t *graph, } static int +set_quotad_xlator_option (char *fmt_str, char *opt_name, char *volname, + xlator_t *xl, char *value) +{ + int ret = -1; + char *opt_str = NULL; + + ret = gf_asprintf (&opt_str, fmt_str, volname, opt_name); + if (opt_str == NULL) { + ret = -1; + goto out; + } + ret = xlator_set_option (xl, opt_str, value); + +out: + GF_FREE (opt_str); + return ret; +} + +int +quotad_option_handler (volgen_graph_t *graph, struct volopt_map_entry *vme, + void *param) +{ + xlator_t *xl = NULL; + char *opt_str = NULL; + int ret = 0; + glusterd_volinfo_t *volinfo = NULL; + int i = 0; + char *ptr = NULL; + char *options[] = {"!*.soft-timeout", "!*.hard-timeout", + "limit-set", "!*.alert-time", + "default-soft-limit", NULL}; + + volinfo = param; + xl = first_of (graph); + + for (i = 0; options[i]; i++) { + if (!strcmp (vme->option, options[i])) { + ptr = strchr (options[i], '.'); + if (!ptr) + opt_str = options[i]; + else + opt_str = ptr + 1; + ret = set_quotad_xlator_option ("%s.%s", opt_str, + volinfo->volname, xl, + vme->value); + if (ret) + return -1; + } + } + + return 0; +} + +static int volgen_graph_set_iam_shd (volgen_graph_t *graph) { xlator_t *trav; @@ -3024,9 +3103,6 @@ build_nfs_graph (volgen_graph_t *graph, dict_t *mod_dict) return ret; } - - - /**************************** * * Volume generation interface @@ -3113,7 +3189,109 @@ glusterd_generate_brick_volfile (glusterd_volinfo_t *volinfo, return ret; } +static int +build_quotad_graph (volgen_graph_t *graph, dict_t *mod_dict) +{ + volgen_graph_t cgraph = {0}; + glusterd_volinfo_t *voliter = NULL; + xlator_t *this = NULL; + glusterd_conf_t *priv = NULL; + dict_t *set_dict = NULL; + int ret = 0; + xlator_t *quotad_xl = NULL; + + this = THIS; + priv = this->private; + + set_dict = dict_new (); + if (!set_dict) { + ret = -ENOMEM; + goto out; + } + + quotad_xl = volgen_graph_add_as (graph, "features/quotad", "quotad"); + if (!quotad_xl) { + ret = -1; + goto out; + } + + list_for_each_entry (voliter, &priv->volumes, vol_list) { + if (voliter->status != GLUSTERD_STATUS_STARTED) + continue; + + if (1 != glusterd_is_volume_quota_enabled (voliter)) + continue; + + ret = dict_set_uint32 (set_dict, "trusted-client", + GF_CLIENT_TRUSTED); + if (ret) + goto out; + + dict_copy (voliter->dict, set_dict); + if (mod_dict) + dict_copy (mod_dict, set_dict); + + memset (&cgraph, 0, sizeof (cgraph)); + ret = volgen_graph_build_clients (&cgraph, voliter, set_dict, + NULL); + if (ret) + goto out; + + ret = volume_volgen_graph_build_clusters (&cgraph, voliter, + _gf_true); + if (ret) { + ret = -1; + goto out; + } + + if (mod_dict) { + dict_copy (mod_dict, set_dict); + ret = volgen_graph_set_options_generic (&cgraph, set_dict, + voliter, + basic_option_handler); + } else { + ret = volgen_graph_set_options_generic (&cgraph, + voliter->dict, + voliter, + basic_option_handler); + } + if (ret) + goto out; + + ret = volgen_graph_merge_sub (graph, &cgraph, 1); + if (ret) + goto out; + + ret = dict_reset (set_dict); + if (ret) + goto out; + } + + list_for_each_entry (voliter, &priv->volumes, vol_list) { + if (voliter->status != GLUSTERD_STATUS_STARTED) + continue; + + if (1 != glusterd_is_volume_quota_enabled (voliter)) + continue; + if (mod_dict) { + ret = volgen_graph_set_options_generic (graph, mod_dict, + voliter, + quotad_option_handler); + } else { + ret = volgen_graph_set_options_generic (graph, + voliter->dict, voliter, + quotad_option_handler); + } + if (ret) + gf_log (THIS->name, GF_LOG_ERROR, "Failed to set " + "options for the volume %s", voliter->volname); + } +out: + if (set_dict) + dict_unref (set_dict); + return ret; +} static void get_vol_tstamp_file (char *filename, glusterd_volinfo_t *volinfo) @@ -3381,6 +3559,18 @@ out: } int +glusterd_create_quotad_volfile () +{ + char filepath[PATH_MAX] = {0,}; + glusterd_conf_t *conf = THIS->private; + + glusterd_get_nodesvc_volfile ("quotad", conf->workdir, + filepath, sizeof (filepath)); + return glusterd_create_global_volfile (build_quotad_graph, + filepath, NULL); +} + +int glusterd_check_nfs_volfile_identical (gf_boolean_t *identical) { char nfsvol[PATH_MAX] = {0,}; diff --git a/xlators/mgmt/glusterd/src/glusterd-volgen.h b/xlators/mgmt/glusterd/src/glusterd-volgen.h index 4ff899f4..303acd90 100644 --- a/xlators/mgmt/glusterd/src/glusterd-volgen.h +++ b/xlators/mgmt/glusterd/src/glusterd-volgen.h @@ -22,6 +22,7 @@ #define VKEY_DIAG_CNT_FOP_HITS "diagnostics.count-fop-hits" #define VKEY_DIAG_LAT_MEASUREMENT "diagnostics.latency-measurement" #define VKEY_FEATURES_LIMIT_USAGE "features.limit-usage" +#define VKEY_FEATURES_SOFT_LIMIT "features.soft-limit" #define VKEY_MARKER_XTIME GEOREP".indexing" #define VKEY_CHANGELOG "changelog.changelog" #define VKEY_FEATURES_QUOTA "features.quota" @@ -119,6 +120,7 @@ void glusterd_get_shd_filepath (char *filename); int glusterd_create_nfs_volfile (); int glusterd_create_shd_volfile (); +int glusterd_create_quotad_volfile (); int glusterd_delete_volfile (glusterd_volinfo_t *volinfo, glusterd_brickinfo_t *brickinfo); diff --git a/xlators/mgmt/glusterd/src/glusterd-volume-ops.c b/xlators/mgmt/glusterd/src/glusterd-volume-ops.c index 7cac938c..1b860df2 100644 --- a/xlators/mgmt/glusterd/src/glusterd-volume-ops.c +++ b/xlators/mgmt/glusterd/src/glusterd-volume-ops.c @@ -526,9 +526,12 @@ __glusterd_handle_cli_statedump_volume (rpcsvc_request_t *req) glusterd_op_t cli_op = GD_OP_STATEDUMP_VOLUME; char err_str[2048] = {0,}; xlator_t *this = NULL; + glusterd_conf_t *priv = NULL; this = THIS; GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); GF_ASSERT (req); @@ -577,6 +580,14 @@ __glusterd_handle_cli_statedump_volume (rpcsvc_request_t *req) goto out; } + if (priv->op_version == GD_OP_VERSION_MIN && + strstr (options, "quotad")) { + snprintf (err_str, sizeof (err_str), "The cluster is operating " + "at op-version 1. Taking quotad's statedump is " + "disallowed in this state"); + ret = -1; + goto out; + } gf_log (this->name, GF_LOG_INFO, "Received statedump request for " "volume %s with options %s", volname, options); @@ -1236,6 +1247,13 @@ glusterd_op_stage_statedump_volume (dict_t *dict, char **op_errstr) gf_boolean_t is_running = _gf_false; glusterd_volinfo_t *volinfo = NULL; char msg[2408] = {0,}; + xlator_t *this = NULL; + glusterd_conf_t *priv = NULL; + + this = THIS; + GF_ASSERT (this); + priv = this->private; + GF_ASSERT (priv); ret = glusterd_op_statedump_volume_args_get (dict, &volname, &options, &option_cnt); @@ -1244,10 +1262,7 @@ glusterd_op_stage_statedump_volume (dict_t *dict, char **op_errstr) ret = glusterd_volinfo_find (volname, &volinfo); if (ret) { - snprintf (msg, sizeof(msg), "Volume %s does not exist", - volname); - gf_log ("", GF_LOG_ERROR, "%s", msg); - *op_errstr = gf_strdup (msg); + snprintf (msg, sizeof(msg), FMTSTR_CHECK_VOL_EXISTS, volname); goto out; } @@ -1257,16 +1272,31 @@ glusterd_op_stage_statedump_volume (dict_t *dict, char **op_errstr) is_running = glusterd_is_volume_started (volinfo); if (!is_running) { - snprintf (msg, sizeof(msg), "Volume %s is not in a started" + snprintf (msg, sizeof(msg), "Volume %s is not in the started" " state", volname); - gf_log ("", GF_LOG_ERROR, "%s", msg); - *op_errstr = gf_strdup (msg); ret = -1; goto out; } + if (priv->op_version == GD_OP_VERSION_MIN && + strstr (options, "quotad")) { + snprintf (msg, sizeof (msg), "The cluster is operating " + "at op-version 1. Taking quotad's statedump is " + "disallowed in this state"); + ret = -1; + goto out; + } + if ((strstr (options, "quotad")) && + (!glusterd_is_volume_quota_enabled (volinfo))) { + snprintf (msg, sizeof (msg), "Quota is not enabled on " + "volume %s", volname); + ret = -1; + goto out; + } out: - gf_log ("", GF_LOG_DEBUG, "Returning %d", ret); + if (ret && msg[0] != '\0') + *op_errstr = gf_strdup (msg); + gf_log (this->name, GF_LOG_DEBUG, "Returning %d", ret); return ret; } @@ -1828,6 +1858,18 @@ glusterd_op_statedump_volume (dict_t *dict, char **op_errstr) ret = glusterd_nfs_statedump (options, option_cnt, op_errstr); if (ret) goto out; + + } else if (strstr (options, "quotad")) { + if (is_origin_glusterd()) { + ret = glusterd_quotad_statedump (options, option_cnt, + op_errstr); + if (ret) + goto out; + } else { + ret = 0; + goto out; + } + } else { list_for_each_entry (brickinfo, &volinfo->bricks, brick_list) { diff --git a/xlators/mgmt/glusterd/src/glusterd-volume-set.c b/xlators/mgmt/glusterd/src/glusterd-volume-set.c index b6dd09a6..d59b81d3 100644 --- a/xlators/mgmt/glusterd/src/glusterd-volume-set.c +++ b/xlators/mgmt/glusterd/src/glusterd-volume-set.c @@ -945,29 +945,51 @@ struct volopt_map_entry glusterd_volopt_map[] = { }, /* Quota xlator options */ - { .key = VKEY_FEATURES_LIMIT_USAGE, - .voltype = "features/quota", - .option = "limit-set", - .type = NO_DOC, - .op_version = 1, - .flags = OPT_FLAG_CLIENT_OPT - }, - { .key = "features.quota-timeout", - .voltype = "features/quota", - .option = "timeout", - .value = "0", - .op_version = 1, - .validate_fn = validate_quota, - .flags = OPT_FLAG_CLIENT_OPT - }, - { .key = "features.quota-deem-statfs", - .voltype = "features/quota", - .option = "deem-statfs", - .value = "off", - .type = DOC, - .op_version = 3, - .validate_fn = validate_quota, - .flags = OPT_FLAG_CLIENT_OPT + { .key = VKEY_FEATURES_LIMIT_USAGE, + .voltype = "features/quota", + .option = "limit-set", + .type = NO_DOC, + .op_version = 1, + }, + { + .key = "features.quota-timeout", + .voltype = "features/quota", + .option = "timeout", + .value = "0", + .op_version = 1, + .validate_fn = validate_quota, + }, + { .key = "features.default-soft-limit", + .voltype = "features/quota", + .option = "default-soft-limit", + .type = NO_DOC, + .op_version = 3, + }, + { .key = "features.soft-timeout", + .voltype = "features/quotad", + .option = "!*.soft-timeout", + .type = NO_DOC, + .op_version = 3, + }, + { .key = "features.hard-timeout", + .voltype = "features/quotad", + .option = "!*.hard-timeout", + .type = NO_DOC, + .op_version = 3, + }, + { .key = "features.alert-time", + .voltype = "features/quotad", + .option = "!*.alert-time", + .type = NO_DOC, + .op_version = 3, + }, + { .key = "features.quota-deem-statfs", + .voltype = "features/quota", + .option = "deem-statfs", + .value = "off", + .type = DOC, + .op_version = 2, + .validate_fn = validate_quota, }, /* Marker xlator options */ diff --git a/xlators/mgmt/glusterd/src/glusterd.c b/xlators/mgmt/glusterd/src/glusterd.c index a43c8d54..291b6d0a 100644 --- a/xlators/mgmt/glusterd/src/glusterd.c +++ b/xlators/mgmt/glusterd/src/glusterd.c @@ -1061,6 +1061,15 @@ init (xlator_t *this) exit (1); } + snprintf (storedir, PATH_MAX, "%s/quotad", workdir); + ret = mkdir (storedir, 0777); + if ((-1 == ret) && (errno != EEXIST)) { + gf_log (this->name, GF_LOG_CRITICAL, + "Unable to create quotad directory %s" + " ,errno = %d", storedir, errno); + exit (1); + } + snprintf (storedir, PATH_MAX, "%s/groups", workdir); ret = mkdir (storedir, 0777); if ((-1 == ret) && (errno != EEXIST)) { @@ -1114,12 +1123,14 @@ init (xlator_t *this) conf = GF_CALLOC (1, sizeof (glusterd_conf_t), gf_gld_mt_glusterd_conf_t); GF_VALIDATE_OR_GOTO(this->name, conf, out); - conf->shd = GF_CALLOC (1, sizeof (nodesrv_t), - gf_gld_mt_nodesrv_t); + + conf->shd = GF_CALLOC (1, sizeof (nodesrv_t), gf_gld_mt_nodesrv_t); GF_VALIDATE_OR_GOTO(this->name, conf->shd, out); - conf->nfs = GF_CALLOC (1, sizeof (nodesrv_t), - gf_gld_mt_nodesrv_t); + conf->nfs = GF_CALLOC (1, sizeof (nodesrv_t), gf_gld_mt_nodesrv_t); GF_VALIDATE_OR_GOTO(this->name, conf->nfs, out); + conf->quotad = GF_CALLOC (1, sizeof (nodesrv_t), + gf_gld_mt_nodesrv_t); + GF_VALIDATE_OR_GOTO(this->name, conf->quotad, out); INIT_LIST_HEAD (&conf->peers); INIT_LIST_HEAD (&conf->volumes); diff --git a/xlators/mgmt/glusterd/src/glusterd.h b/xlators/mgmt/glusterd/src/glusterd.h index 8c043a9a..ac58e2a6 100644 --- a/xlators/mgmt/glusterd/src/glusterd.h +++ b/xlators/mgmt/glusterd/src/glusterd.h @@ -131,6 +131,7 @@ typedef struct { rpcsvc_t *rpc; nodesrv_t *shd; nodesrv_t *nfs; + nodesrv_t *quotad; struct pmap_registry *pmap; struct list_head volumes; pthread_mutex_t xprt_lock; @@ -311,6 +312,7 @@ typedef enum gd_node_type_ { GD_NODE_SHD, GD_NODE_REBALANCE, GD_NODE_NFS, + GD_NODE_QUOTAD, } gd_node_type; typedef struct glusterd_pending_node_ { @@ -366,6 +368,9 @@ typedef ssize_t (*gd_serialize_t) (struct iovec outmsg, void *args); #define GLUSTERD_GET_NFS_DIR(path, priv) \ snprintf (path, PATH_MAX, "%s/nfs", priv->workdir); +#define GLUSTERD_GET_QUOTAD_DIR(path, priv) \ + snprintf (path, PATH_MAX, "%s/quotad", priv->workdir); + #define GLUSTERD_REMOVE_SLASH_FROM_PATH(path,string) do { \ int i = 0; \ for (i = 1; i < strlen (path); i++) { \ @@ -389,6 +394,11 @@ typedef ssize_t (*gd_serialize_t) (struct iovec outmsg, void *args); nfspath); \ } +#define GLUSTERD_GET_QUOTAD_PIDFILE(pidfile,quotadpath) { \ + snprintf (pidfile, PATH_MAX, "%s/run/quotad.pid", \ + quotadpath); \ + } + #define GLUSTERD_STACK_DESTROY(frame) do {\ frame->local = NULL; \ STACK_DESTROY (frame->root);\ |