tier/dht: unlink fails after lookup in a directory

unlink fails with invalid argument for files that
are being present on cold tier, before attaching.

All of the fops will be hashed to hot_tier after
attach-tier (unless explicitly set the "rule"
option). Lookups sent to directory, will eventually
search the directory using readdirp, and will
populate inode_ctx for the inodes based on the output,
in respective dht_xlators. So the readdirp will
populate inodes_ctx for the files (that is already
present in volume before attaching) in cold-dht
only because it got the entries from the cold-tier.

So when an unlink comes on such an inode, the lookup
associated with the unlink will be send as a
re validate request to cold-tier only, since
already a lookup was performed on the inode,
and the new lookup will succeed. So from the
unlink of dht, it will hash to cold-tier but the
cached_subvol will be cold, since there is a
mismatch in hash and cach , it chose hashed
subvolume and will sent the fop to hot dht,
and the fops fail with EINVAL from the hot-dht
since it does not have inode_ctx stored for
that inode (because, no lookup was performed
from hot-dht).

Change-Id: Ib7c14a9297a22d615f7a890a060be4809b5a745a
BUG: 1236032
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/11675
Tested-by: NetBSD Build System <jenkins@build.gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>

1 files changed, 17 insertions, 14 deletions

diff --git a/xlators/cluster/dht/src/dht-common.c b/xlators/cluster/dht/src/dht-common.c
index 4a55905f36a..ef1776beb30 100644
--- a/xlators/cluster/dht/src/dht-common.c
+++ b/xlators/cluster/dht/src/dht-common.c
@@ -4485,6 +4485,7 @@ dht_readdirp_cbk (call_frame_t *frame, void *cookie, xlator_t *this, int op_ret,
         xlator_t     *hashed_subvol = 0;
         int           ret    = 0;
         int           readdir_optimize = 0;
+        dht_inode_ctx_t *ctx = NULL;
 
         INIT_LIST_HEAD (&entries.list);
         prev = cookie;
@@ -4578,20 +4579,22 @@ list:
                 if (orig_entry->dict)
                         entry->dict = dict_ref (orig_entry->dict);
 
-                /* making sure we set the inode ctx right with layout,
-                   currently possible only for non-directories, so for
-                   directories don't set entry inodes */
-                if (!IA_ISDIR(entry->d_stat.ia_type) && orig_entry->inode) {
-                        ret = dht_layout_preset (this, prev->this,
-                                                 orig_entry->inode);
-                        if (ret)
-                                gf_msg (this->name, GF_LOG_WARNING, 0,
-                                        DHT_MSG_LAYOUT_SET_FAILED,
-                                        "failed to link the layout in inode");
-                        entry->inode = inode_ref (orig_entry->inode);
-                } else if (orig_entry->inode) {
-                        dht_inode_ctx_time_update (orig_entry->inode, this,
-                                                   &entry->d_stat, 1);
+                /* For non-directories don't set inode ctx from readdirp cbk,
+                 * let them populate on first lookup, for directories
+                 * don't set entry inodes */
+                if (orig_entry->inode) {
+                        ret = dht_inode_ctx_get (orig_entry->inode, this, &ctx);
+                            if (ret == -1) {
+                                 entry->inode = NULL;
+                            } else {
+                                   entry->inode = inode_ref (orig_entry->inode);
+                                if (IA_ISDIR (entry->d_stat.ia_type)) {
+                                   dht_inode_ctx_time_update (orig_entry->inode,
+                                                       this, &entry->d_stat, 1);
+
+                                }
+
+                            }
                 }
 
                 list_add_tail (&entry->list, &entries.list);