glusterfs.git/xlators/features/changelog, branch v3.6.0beta1

features/changelog: Removal of redundant fop color count while draining.

2014-09-09T08:09:52+00:00

BUG: 1138952
Change-Id: I594be0d09c6af2e4a34da3e819d1ab6fd85e34c4
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/8542
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System 
Reviewed-on: http://review.gluster.org/8647

features/changelog: barrier all entry creation fops

2014-09-09T08:09:38+00:00

when a snapshot is taken, there are chances of entry creation fops
not being recorded either in changelog or through the recursive
ancestry xtime updation by marker. This causes consumers of changelog
(primarily geo-replication as of today) to not be aware of these entries
after a snapshot is restored. This can lead to inconsistencies. This patch
is an interim workaround to barrier creates till changelog becomes completely
crash consistent.

BUG: 1138952
Change-Id: Idd5e690a05fe2c7c5d32d1541a0d9b5132881ea7
Signed-off-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8517
Tested-by: Gluster Build System 
Reviewed-by: ajeet jha 
Reviewed-by: Aravinda VK 
Reviewed-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/8646

geo-rep/libgfchangelog: Support of symlinks while creation of working dir.

2014-09-08T17:42:39+00:00

In gf_changelog_register, enable symlink support while creating
working directory if its not already created.

BUG: 1138952
Change-Id: I8fec52a5768fae46ce30a2331f30f1d8d5e2e173
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8409
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Tested-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8641
Reviewed-by: Vijay Bellur

geo-rep/libgfchangelog: Create working dir during changelog_register if not present.

2014-09-08T17:42:25+00:00

Earlier, xysnc's register was being called first,
which was creating working directory before calling
changelog_register. Now it is history crawl first.
Hence working directory would not have been created.
Create it in gf_changelog_register itself if it is
not already created.

BUG: 1138952
Change-Id: Ie39b9fd8c1ef7385f76a9b67d0acc3c1c2fd2bb2
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8399
Tested-by: Gluster Build System 
Reviewed-by: Aravinda VK 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8640

features/changelog: Capture "correct" internal FOPs

2014-09-08T17:21:17+00:00

This patch fixes changelog capturing internal FOPs in a cascaded
setup, where the intermediate master would record internal FOPs
(generated by DHT on link()/rename()). This is due to I/O happening
on the intermediate slave on geo-replication's auxillary mount with
client-pid -1. Currently, the internal FOP capturing logic depends
on client pid being non-negative and the presence of a special key
in dictionary. Due to this, internal FOPs on an inter-mediate master
would be recorded in the changelog. Checking client-pid being
non-negative was introduced to capture AFR self-heal traffic in
changelog, thereby breaking cascading setups. By coincidence,
AFR self-heal daemon uses -1 as frame->root->pid thereby making
is hard to differentiate b/w geo-rep's auxillary mount and self-heal
daemon.

BUG: 1138952
Change-Id: Ia08a2cfa3b02bb785f343794f5b2695d44398c4c
Original-Author: Venky Shankar 
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8347
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8638

features/changelog: Crash consistency of changelog wrt snapshot

2014-09-08T14:12:47+00:00

This patch introduces call-path fop details logging for data operations
in CHANGELOG.SNAP. This feature is enabled with barrier-enable
notification and disabled with barrier-disable notification.

BUG: 1138952
Change-Id: Ic418dd70b0a0b369202c5b79a6f7f96512821065
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/8533
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-on: http://review.gluster.org/8648

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-09-03T18:01:45+00:00

Backport from master branch - http://review.gluster.org/#/c/8246/

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I06e684ac4c26d1e74c9daf76753403ad15f79276
BUG: 1130308
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8486
Tested-by: Gluster Build System

changelog: variable 'writen' is changed

2014-07-14T14:02:43+00:00

This patch change the variable name to 'written' in changelog, along
with few other typo fixing.

Change-Id: I1d3cbea5c89fc0a15233bb6e61b4fbd4bfdd3a67
BUG: 1075417
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/8267
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/changelog: prevent deadlock on thread cancellation

2014-07-11T07:08:57+00:00

helper threads (fsync, rollover) wake up periodically and perform
their respective operation under a lock (crt->lock). These threads
are also subjected to cancellation under some circumstance such as
disabling changelog. This is inherently dangerous when funtions
which are cancellation points for pthread_cancel(3) are used
in the locked region.

Consider this

         pthread_mutex_lock(&mutex);
         {
                /* ... */
                ret = fsync (fd);   <-- cancellation point
                /* ... */
         }
         pthread_mutex_unlock(&mutex);

A pthread_cancel(3) by another thread just before fsync(3) but
after pthread_mutex_lock(3) would result in the thread getting
cancelled when fsync(3) is invoked, thereby never unlocking the
mutex. Moreover, in case of changelog translator, the locked
region (under crt->lock in changelog-rt.c) is also the code
path for fop changelog updation. Therefore, unlocking the
mutex in thread cleanup handler (pthread_cleanup_pop(3)) might
prematurely release the mutex during fop updation path.

This patch fixes such problems existing in fsync and rollover
threads. Fix is to enter the locked region with cancellation
disabled and enable it after mutex unlock. Also, test for a
cancellation request early on in case none of the functions
are cancellation points.

Change-Id: I1795627a12827609c1da659d07fc1457ffa033de
BUG: 1110917
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8106
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System

feature/changelog: Fix for missing changelogs at backend.

2014-06-30T11:25:48+00:00

Problem:
       A few changelog files are missing at the backend
       during snapshot with changelog enabled.

Cause:
       Race between actual rollover and explicit rollover.

       Changelog rollover can happen either due to actual
       or the explict rollover due to snapshot. Actual
       rollover is controlled by tuneable called rollover-time.
       The minimum granularity for rollover-time is 1 second
       Explicit rollover is asynchronous in nature and happens
       during snapshot.

       Basically, rollover renames the current CHANGELOG file
       to CHANGELOG.TIMESTAMP after rollover-time. Let's assume,
       at time 't1', actual and explicit rollover raced against
       each other and actual rollover won the race renaming the
       CHANGELOG file to CHANGELOG.t1 and opens a new
       CHANGELOG file. An immediate explicit rollover at time
       't1' happened with in the same second to rename
       CHANGELOG file to CHANGELOG.t1 resulting in purging the
       earlier CHANGELOG.t1 file created by actual rollover.

Solution:
       Adding a delay of 1 sec guarantees unique CHANGELOG.TIMESTAMP
       during explicit rollover.

Thanks Venky, for the all the help in root causing the issue.

Change-Id: I8958824e107e16f61be9f09a11d95f8645ecf34d
BUG: 1109692
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8196
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar