performance/write-behind: better invalidation in readdirp

Current invalidation of stats in wb_readdirp_cbk is prone to races. As
the deleted comment explains,

<snip>
We cannot guarantee integrity of entry->d_stat as there are cached
writes. The stat is most likely stale as it doesn't account the cached
writes. However, checking for non-empty liability list here is not a
fool-proof solution as there can be races like,
  1. readdirp is successful on posix
  2. sync of cached write is successful on posix
  3. write-behind received sync response and removed the request from
     liability queue
  4. readdirp response is processed at write-behind.

In the above scenario, stat for the file is sent back in readdirp
response but it is stale.
</snip>

Change-Id: I6ce170985cc6ce3df2382ec038dd5415beefded5
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Updates: bz#1512691

1 files changed, 28 insertions, 23 deletions

diff --git a/xlators/performance/write-behind/src/write-behind.c b/xlators/performance/write-behind/src/write-behind.c
index bb07cb53c4b..15dd8fa44d0 100644
--- a/xlators/performance/write-behind/src/write-behind.c
+++ b/xlators/performance/write-behind/src/write-behind.c
@@ -115,6 +115,8 @@ typedef struct wb_inode {
                                 * error during fulfill.
                                 */
 
+        int invalidate_stat;
+
 } wb_inode_t;
 
 
@@ -2461,29 +2463,7 @@ wb_readdirp_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
 
                 LOCK (&wb_inode->lock);
                 {
-                        if (!list_empty (&wb_inode->liability)) {
-                                /* We cannot guarantee integrity of
-                                   entry->d_stat as there are cached writes.
-                                   The stat is most likely stale as it doesn't
-                                   account the cached writes. However, checking
-                                   for non-empty liability list here is not a
-                                   fool-proof solution as there can be races
-                                   like,
-                                   1. readdirp is successful on posix
-                                   2. sync of cached write is successful on
-                                      posix
-                                   3. write-behind received sync response and
-                                      removed the request from liability queue
-                                   4. readdirp response is processed at
-                                      write-behind
-
-                                   In the above scenario, stat for the file is
-                                   sent back in readdirp response but it is
-                                   stale.
-
-                                   For lack of better solutions I am sticking
-                                   with current solution.
-                                */
+                        if (wb_inode->invalidate_stat) {
                                 inode = entry->inode;
 
                                 entry->inode = NULL;
@@ -2491,6 +2471,7 @@ wb_readdirp_cbk (call_frame_t *frame, void *cookie, xlator_t *this,
                                         sizeof (entry->d_stat));
 
                                 inode_unref (inode);
+                                wb_inode->invalidate_stat = 0;
                         }
                 }
                 UNLOCK (&wb_inode->lock);
@@ -2507,6 +2488,30 @@ int32_t
 wb_readdirp (call_frame_t *frame, xlator_t *this, fd_t *fd, size_t size,
              off_t off, dict_t *xdata)
 {
+        dentry_t     *child    = NULL;
+        wb_inode_t   *wb_inode = NULL;
+        wb_request_t *each     = NULL;
+
+        pthread_mutex_lock (&fd->inode->table->lock);
+        {
+                list_for_each_entry (child, &fd->inode->children, parent_list) {
+                        wb_inode = wb_inode_ctx_get (this, child->inode);
+                        if (!wb_inode)
+                                continue;
+
+                        LOCK (&wb_inode->lock);
+                        {
+                                list_for_each_entry (each, &wb_inode->liability,
+                                                     lie) {
+                                        if (each->gen < wb_inode->gen)
+                                                wb_inode->invalidate_stat = 1;
+                                }
+                        }
+                        UNLOCK (&wb_inode->lock);
+                }
+        }
+        pthread_mutex_unlock (&fd->inode->table->lock);
+
         STACK_WIND (frame, wb_readdirp_cbk, FIRST_CHILD(this),
                     FIRST_CHILD(this)->fops->readdirp,
                     fd, size, off, xdata);