glusterfs.git/xlators/features/changelog/src, branch v3.7dev

changelog: variable 'writen' is changed

2014-07-14T14:02:43+00:00

This patch change the variable name to 'written' in changelog, along
with few other typo fixing.

Change-Id: I1d3cbea5c89fc0a15233bb6e61b4fbd4bfdd3a67
BUG: 1075417
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/8267
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/changelog: prevent deadlock on thread cancellation

2014-07-11T07:08:57+00:00

helper threads (fsync, rollover) wake up periodically and perform
their respective operation under a lock (crt->lock). These threads
are also subjected to cancellation under some circumstance such as
disabling changelog. This is inherently dangerous when funtions
which are cancellation points for pthread_cancel(3) are used
in the locked region.

Consider this

         pthread_mutex_lock(&mutex);
         {
                /* ... */
                ret = fsync (fd);   <-- cancellation point
                /* ... */
         }
         pthread_mutex_unlock(&mutex);

A pthread_cancel(3) by another thread just before fsync(3) but
after pthread_mutex_lock(3) would result in the thread getting
cancelled when fsync(3) is invoked, thereby never unlocking the
mutex. Moreover, in case of changelog translator, the locked
region (under crt->lock in changelog-rt.c) is also the code
path for fop changelog updation. Therefore, unlocking the
mutex in thread cleanup handler (pthread_cleanup_pop(3)) might
prematurely release the mutex during fop updation path.

This patch fixes such problems existing in fsync and rollover
threads. Fix is to enter the locked region with cancellation
disabled and enable it after mutex unlock. Also, test for a
cancellation request early on in case none of the functions
are cancellation points.

Change-Id: I1795627a12827609c1da659d07fc1457ffa033de
BUG: 1110917
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/8106
Reviewed-by: Kotresh HR 
Reviewed-by: Vijay Bellur 
Tested-by: Gluster Build System

feature/changelog: Fix for missing changelogs at backend.

2014-06-30T11:25:48+00:00

Problem:
       A few changelog files are missing at the backend
       during snapshot with changelog enabled.

Cause:
       Race between actual rollover and explicit rollover.

       Changelog rollover can happen either due to actual
       or the explict rollover due to snapshot. Actual
       rollover is controlled by tuneable called rollover-time.
       The minimum granularity for rollover-time is 1 second
       Explicit rollover is asynchronous in nature and happens
       during snapshot.

       Basically, rollover renames the current CHANGELOG file
       to CHANGELOG.TIMESTAMP after rollover-time. Let's assume,
       at time 't1', actual and explicit rollover raced against
       each other and actual rollover won the race renaming the
       CHANGELOG file to CHANGELOG.t1 and opens a new
       CHANGELOG file. An immediate explicit rollover at time
       't1' happened with in the same second to rename
       CHANGELOG file to CHANGELOG.t1 resulting in purging the
       earlier CHANGELOG.t1 file created by actual rollover.

Solution:
       Adding a delay of 1 sec guarantees unique CHANGELOG.TIMESTAMP
       during explicit rollover.

Thanks Venky, for the all the help in root causing the issue.

Change-Id: I8958824e107e16f61be9f09a11d95f8645ecf34d
BUG: 1109692
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8196
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/changelog: Do not ignore self-heal fops in changelog

2014-06-16T11:23:37+00:00

Problem: Geo-rep fails to sync some files to slave as the
changelog entries are missing for those files.

Cause: Fops happened when the active brick is down and
self- healed later when it came up.

Solution: Capture self-heal fops as well in changelog so
those entries are not missed.

Change-Id: Ibc288779421b5156dd1695e529aba0b602a530e0
BUG: 1109692
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8070
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

feature/changelog: Fix compilation warning 'set but not used'

2014-06-12T16:58:07+00:00

Change-Id: I37de39aec341fbde9516aa8f8c963b70bd808bac
BUG: 1108034
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8035
Tested-by: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Pranith Kumar Karampuri

Core: Fix issues reported by Cppcheck

2014-06-12T11:20:34+00:00

Fixed in this patch:

[glusterfs/extras/geo-rep/gsync-sync-gfid.c:105]: (error) Resource leak: fp
[glusterfs/libglusterfs/src/xlator.c:651]: (error) Uninitialized variable: gfid
[glusterfs/libglusterfs/src/xlator.c:652]: (error) Uninitialized variable: gfid
[glusterfs/xlators/cluster/ha/src/ha.c:2699]: (error) Possible null pointer dereference: priv
[glusterfs/xlators/features/changelog/src/changelog.c:1464]: (error) Possible null pointer dereference: priv
[glusterfs/xlators/mgmt/glusterd/src/glusterd-mgmt-handler.c:865]: (error) Possible null pointer dereference: ctx
[glusterfs/xlators/mgmt/glusterd/src/glusterd-mgmt-handler.c:194]: (error) Possible null pointer dereference: ctx
[glusterfs/xlators/mgmt/glusterd/src/glusterd-syncop.c:1408]: (error) Possible null pointer dereference: this
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:7002]: (error) Possible null pointer dereference: path_tokens

Fixed in 3.4 and 3.5 branch (http://review.gluster.org/#/c/7583/ ,
http://review.gluster.org/#/c/7605/ will be backported in a separate patch)

[glusterfs/xlators/mount/fuse/src/fuse-bridge.c:4688]: (error) Uninitialized variable: finh
[glusterfs/xlators/mount/fuse/src/fuse-bridge.c:3081]: (error) Possible null pointer dereference: state
[glusterfs/xlators/cluster/dht/src/dht-rebalance.c:1719]: (error) Possible null pointer dereference: ctx
[glusterfs/xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local
[glusterfs/xlators/mgmt/glusterd/src/glusterd-replace-brick.c:915]: (error) Resource leak: file
[glusterfs/xlators/mgmt/glusterd/src/glusterd-replace-brick.c:999]: (error) Resource leak: file
[glusterfs/xlators/mgmt/glusterd/src/glusterd-sm.c:248]: (error) Possible null pointer dereference: new_ev_ctx
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:5297]: (error) Possible null pointer dereference: this
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:6273]: (error) Possible null pointer dereference: this
[glusterfs/xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf
[glusterfs/xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it).

False positives

[glusterfs/geo-replication/src/gsyncd.c:99]: (error) Memory leak: str
[glusterfs/geo-replication/src/gsyncd.c:395]: (error) Memory leak: argv
[glusterfs/xlators/nfs/server/src/nlm4.c:1199]: (error) Possible null pointer dereference: fde
[glusterfs/xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1659]: (error) Possible null pointer dereference: command
[glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:7001]: (error) Possible null pointer dereference: path_tokens

Insignificant/Don't care

[glusterfs/contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'.
[glusterfs/contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'.
[glusterfs/extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments.
[glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr
[glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr
[glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr

Change-Id: I7696ed1a2a9553b79f9714e10210a8d563a5abd8
BUG: 1091677
Signed-off-by: Lalatendu Mohanty 
Reviewed-on: http://review.gluster.org/7693
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

features/changelog: skipping empty changelogs in history

2014-06-10T13:37:57+00:00

Empty changelgs are unlinked and htime-update is bypassed
while publishing partially written changelog.

Change-Id: I1533ad431e4fbf592d2f38837fce782932d01e9c
BUG: 1105466
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/7949
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

feature/changelog: Add timeout to changelog barrier.

2014-06-10T10:35:59+00:00

This patch introduces configurable 'changelog-barrier-timeout'
in changelog. The default changelog-barrier-timeout is set to
120 sec which is same as barrier-timeout of barrier translator.
On this timeout, all the barriered fops are unbarriered. It is
safe to have this internal timeout for changelog barrier to
prevent it from indefinite barrierring of fops if barrier
disable notification does not come from snapshot for some
reason.

Change-Id: Iad07807935f88222cdefcd46312b4b6579186dce
BUG: 1094226
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/7669
Reviewed-by: Kaushal M 
Reviewed-by: Venky Shankar 
Tested-by: Venky Shankar

features/changelog : historical journal consumption.

2014-05-14T12:10:15+00:00

Facilitates Glusterfs with the ability to detect file-operations
happened in past by scanning the back-end(brick-level) glusterfs
journal (changelog).

Design:
  * List of changelogs produces in one perfectly running session are
    stored in htime file which also holds necessary information about
    the session start and end time.
  * Involves fixed sized seeks to identify N'th changelog in the list.
  * Requires O(log n), (where n is number of changelogs in the list),
    time to identify the end changelog for the given start-end time
    interval.

Currently the background processing of changelogs is sub optimal. BZ
1097041 tracks the development effort.

For complete design, refer the below link:
http://lists.nongnu.org/archive/html/gluster-devel/2014-02/msg00206.html

Change-Id: I27e49f75e492e843084d0ecaf9130224d08462a0
BUG: 1091961
Signed-off-by: Ajeet Jha 
Signed-off-by: Venky Shankar 
Signed-off-by: Ajeet Jha 
Reviewed-on: http://review.gluster.org/6930
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System

changelog: in C99 `inline` shouldn't be part of function definition

2014-05-04T02:40:11+00:00

- C99 standard dictates that `inline` should be used only to hint
  function implementation not as a definition
          - http://clang.llvm.org/compatibility.html#inline
  This also leads to run time issues as 'undefined symbols'

- Remember to use adding explicit headers when using an external
  function

Change-Id: I66a9030da7ac38ded6e0b6a977182be977382fa2
BUG: 1089172
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/7655
Reviewed-by: Anand Avati 
Tested-by: Anand Avati