storage/posix: implement batched fsync in a single thread

Because of the extra fsync()s issued by AFR transaction, they
could potentially "clog" all the io-threads denying unrelated
operations from making progress.

This patch assigns a dedicated thread to issues fsyncs, as
an experimental feature to understand performance characteristics
with the approach.

As a basis, incoming individual fsync requests are grouped into
batches, falling in the same @batch-fsync-delay-usec window of
time. These windows can extend in practice, as processing of
the previous batch can take longer than @batch-fsync-delay-usec
while new requests are getting batched.

The feature support three modes (similar to the -S modes of fs_mark)

- syncfs: In this mode one syncfs() is issued per batch, instead
  of N fsync()s (one per file.)

- syncfs-single-fsync: In this mode one syncfs() is issued per
  batch (which, on Linux, guarantees the completion of write-out
  of dirty pages in the filesystem up to that point) and one single
  fsync() to synchronize or flush the controller/drive cache. This
  corresponds to -S 2 of fsmark.

- syncfs-reverse-fsync: In this mode, one syncfs() is issued per
  batch, and all the open files in that batch are fsync()'ed in
  the reverse order of the queue. This corresponds to -S 4 of
  fsmark.

- reverse-fsync: In this mode, no syncfs() is issued and all the
  files in the batch are fsync()'ed in the reverse order. This
  corresponds to -S 3 of fsmark.

Change-Id: Ia1e170a810c780c8d80e02cf910accc4170c4cd4
BUG: 927146
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4746
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>

1 files changed, 19 insertions, 0 deletions

diff --git a/xlators/storage/posix/src/posix.h b/xlators/storage/posix/src/posix.h
index 22340370e..c834b29d9 100644
--- a/xlators/storage/posix/src/posix.h
+++ b/xlators/storage/posix/src/posix.h
@@ -43,6 +43,7 @@
 #include "timer.h"
 #include "posix-mem-types.h"
 #include "posix-handle.h"
+#include "call-stub.h"
 
 #ifdef HAVE_LIBAIO
 #include <libaio.h>
@@ -128,6 +129,22 @@ struct posix_private {
         /* node-uuid in pathinfo xattr */
         gf_boolean_t  node_uuid_pathinfo;
 
+	pthread_t         fsyncer;
+	struct list_head  fsyncs;
+	pthread_mutex_t   fsync_mutex;
+	pthread_cond_t    fsync_cond;
+	int               fsync_queue_count;
+
+	enum {
+		BATCH_NONE = 0,
+		BATCH_SYNCFS,
+		BATCH_SYNCFS_SINGLE_FSYNC,
+		BATCH_REVERSE_FSYNC,
+		BATCH_SYNCFS_REVERSE_FSYNC
+	}               batch_fsync_mode;
+
+	uint32_t        batch_fsync_delay_usec;
+
         /* seconds to sleep between health checks */
         uint32_t        health_check_interval;
         pthread_t       health_check;
@@ -184,4 +201,6 @@ void
 __posix_fd_set_odirect (fd_t *fd, struct posix_fd *pfd, int opflags,
 			off_t offset, size_t size);
 void posix_spawn_health_check_thread (xlator_t *this);
+
+void *posix_fsyncer (void *);
 #endif /* _POSIX_H */