glusterfs-quota.git/xlators, branch test

features/changelog: fixes when enabling changelog

2013-07-25T11:40:17+00:00

Other enhancements being:
      * ignore fops made by rebalance
      * ignore internally triggered fops

BUG: 987734
Change-Id: I7dd164ae3c209fdb8ec43a27e67b8846f937c93b
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/5380
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System

write-behind: preserve error returned as-is

2013-07-25T01:39:13+00:00

Change-Id: Ib766403774c1323e0bbddafedeaa47e7fa3a59fa
Signed-off-by: Amar Tumballi 
BUG: 987415
Reviewed-on: http://review.gluster.org/5296
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core: increase the auxillary group limit to 65536

2013-07-24T18:36:46+00:00

Make the allocation of groups dynamic and increase the limit
to 65536.

Change-Id: I702364ff460e3a982e44ccbcb3e337cac9c2df51
BUG: 953694
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5111
Reviewed-by: Xavier Hernandez 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/marker: pass xdata in marker_unlink()

2013-07-24T17:41:30+00:00

Change-Id: Ia310af96b25f29351f3adc4bbc97aea271df7673
BUG: 987747
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/5379
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

storage/posix: Fix conditional compiling for syncfs

2013-07-24T16:46:06+00:00

Change-Id: Ief22e1c0f2b5074060752d70da41ae93f1028d62
BUG: 927146
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5381
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: mark linkfile creation/deletion as internal fop

2013-07-24T11:27:29+00:00

Currently dht creates/deletes linkfiles for various ops like
rename/linking and when layout changes. dht_linkfile_create
already sends a key GLUSTERFS_INTERNAL_FOP_KEY in dict to
identify this as an internal fop and not user based op.

Enhancing rename related links/unlinks to send this key too.
Marker/changelog or other xlators can now identify these as
internal fops and handle them accordingly

Change-Id: Ib1ca789e6dbce48703c55ad3f4f88f7cd2df3d06
BUG: 987428
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/5335
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

storage/posix: implement batched fsync in a single thread

2013-07-23T13:11:12+00:00

Because of the extra fsync()s issued by AFR transaction, they
could potentially "clog" all the io-threads denying unrelated
operations from making progress.

This patch assigns a dedicated thread to issues fsyncs, as
an experimental feature to understand performance characteristics
with the approach.

As a basis, incoming individual fsync requests are grouped into
batches, falling in the same @batch-fsync-delay-usec window of
time. These windows can extend in practice, as processing of
the previous batch can take longer than @batch-fsync-delay-usec
while new requests are getting batched.

The feature support three modes (similar to the -S modes of fs_mark)

- syncfs: In this mode one syncfs() is issued per batch, instead
  of N fsync()s (one per file.)

- syncfs-single-fsync: In this mode one syncfs() is issued per
  batch (which, on Linux, guarantees the completion of write-out
  of dirty pages in the filesystem up to that point) and one single
  fsync() to synchronize or flush the controller/drive cache. This
  corresponds to -S 2 of fsmark.

- syncfs-reverse-fsync: In this mode, one syncfs() is issued per
  batch, and all the open files in that batch are fsync()'ed in
  the reverse order of the queue. This corresponds to -S 4 of
  fsmark.

- reverse-fsync: In this mode, no syncfs() is issued and all the
  files in the batch are fsync()'ed in the reverse order. This
  corresponds to -S 3 of fsmark.

Change-Id: Ia1e170a810c780c8d80e02cf910accc4170c4cd4
BUG: 927146
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/4746
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Handle parallel hardlinks self-heal

2013-07-23T07:33:39+00:00

Change-Id: Ieda11870c65edae500140b6c061f15a7b3f264f3
BUG: 986905
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5370
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

move 'xlators/marker/utils/' to 'geo-replication/' directory

2013-07-22T08:53:03+00:00

Change-Id: Ibd0faefecc15b6713eda28bc96794ae58aff45aa
BUG: 847839
Original Author: Amar Tumballi 
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/5133
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

features/changelog: changelog translator

2013-07-22T08:52:24+00:00

This is the initial version of the Changelog Translator.

What is it
-----------
Goal is to capture changes performed on a GlusterFS volume.
The translator needs to be loaded on the server (bricks) and
captures changes in a plain text file inside a configured
directory path (controlled by "changelog-dir", should be
somewhere in /.glusterfs/changelog by default).

Changes are classified into 3 types:
    - Data:     : TYPE-I
    - Metadata  : TYPE-II
    - Entry     : TYPE-III

Changelog file is rolled over after a certain time interval
(defauls to 60 seconds) after which a changelog is started.
The thing to be noted here is that for a time interval
(time slice) multiple changes for an inode are recorded only
once (ie. say for 100+ writes on an inode that happens within
the time slice has only a single corresponding entry in the
changelog file). That way we do not bloat up the changelog
and also save lots of writes.

Changelog Format
-----------------
TYPE-I and TYPE-II changes have the gfid on the entity on
which the operation happened. TYPE-III being a entry op
requires the parent gfid and the basename. Changelog format
has been kept to a minimal and it's upto the consumers to
do the heavy loading of figuring out deletes, renames etc..
A single changelog file records all three types of changes,
with each change starting with an identifier ("D": DATA,
"M": METADATA and "E": ENTRY). Option is provided for the
encoding type (See TUNABLES).

Consumers
----------
The only consumer as of today would be geo-replication, although
backup utilities, self-heal, bit-rot detection could be possible
consumers in the future.

CLI
----
By default, change-logging is disabled (the translator is present
in the server graph but does nothing). When enabled (via cli) each
brick starts to log the changes. There are a set of tunable that
can be used to change the translators behaviour:

- enable/disable changelog (disabled by default)
  gluster volume set  changelog {on|off}

- set the logging directory (/.glusterfs/changelogs is the
  default)
  gluster volume set  changelog-dir /path/to/dir

- select encoding type (binary (default) or ascii)
  gluster volume set  encoding {binary|ascii}

- change the rollover time for the logs (60 secs by default)
  gluster volume set  rollover-time 

- when secs > 0, changelog file is not open()'d with O_SYNC flag
- and fsync is trigerred periodically every  seconds.
  gluster volume set  fsync-interval 

features/changelog: changelog consumer library (libgfchangelog)

A shared library is provided for the consumer of the changelogs
for easy acess via APIs. Application can link against this library
and request for changelog updates. Conversion of binary logs to
human-readable ascii format is also taken care by the library which
keeps a copy of the changelog in application provided working
directory.

Change-Id: I75575fb7f1c53d2bec3dba1a329ea7bb3c628497
BUG: 847839
Original Author: Venky Shankar 
Signed-off-by: Avra Sengupta 
Reviewed-on: http://review.gluster.org/5127
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur