summaryrefslogtreecommitdiffstats
path: root/xlators/features/changelog/src/changelog.c
Commit message (Collapse)AuthorAgeFilesLines
* features/changelog: RPC'fy {libgf}changelogVenky Shankar2015-03-181-180/+293
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces RPC based communication between the changelog translator and libgfchangelog. It replaces the old pathetic stream based interaction that existed earlier (due to time constraints :-/). Changelog, upon initialization starts a RPC server (rpcsvc) allowing clients to invoke a probe API as a bootup mechanism to request for event notifications. During probe, clients can choose an event filter specifying the type(s) of events they are interested in. As of now there is no way to change the event notification set once the probe RPC call is made, but that is easier to implement. The actual event notifications is done on a separate RPC session. The client (libgfchangelog) itself starts and RPC server which the changelog translator "connects back" during probe. Notifications are dispatched by a bunch of threads from the server (translator) and the client optionally orders them if ordered notifications are requried. FOPs fill in their respective event details in a buffer (rot-buffs to be particular) and a bunch of threads (consumers) swap the buffers out of roatation and dispatch them via RPC. To avoid writer starvation, then number of dispatcher threads is one less than the number of buffer list in rot-buffs.x libgfchangelog becomes purely callback based -- upon event notification from the server (and re-ordering them if required) invoke a callback routine specified by consumer(s). A major part of the patch is also aimed at providing backward compatibility for geo-replication, which was one of the main consumer of the stream based API. Also, this patch does not\ "turn on" event notifications for all fops, just a bunch which is currently in requirement. Another pain point is that the server does not filter events before dispatching it to the clients. That load is taken up by the client itself (although it's done at the library layer rather than making it hard on the callback implementor). This needs improvement and care needs to be taken to not load the server up with expensive filtering mechanisms. Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9708 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* feature/changelog: Logging entry as well for explicit sync virtual xattr.Kotresh HR2015-01-091-7/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an improvement over the patch 'http://review.gluster.org/9337' to trigger explicit geo-rep sync on regular files even if entry is not present on the slave. An attempt is made to find the pargfid and if available captures CREATE along with DATA in changelog. CREATE is captured with default file permissions. Setting this virtual setxattr on directories captures MKDIR in changelog. The value of setxattr can be as follows. If value = "1" : Both CREATE and DATA is captured in changelog if pargfid is available, else on DATA is captured. value = "any other: ENOTSUP is returned. Usage: setfattr -n glusterfs.geo-rep.trigger-sync -v "1" <file-path> NOTE: This patch supports explicit record of entries only for directories and regular files. Change-Id: Iedde8b2c8bc3b78db524050d8c866ff664811d01 BUG: 1176934 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/9370 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* feature/changelog: Virtual xattr to trigger explicit sync in geo-rep.Kotresh HR2014-12-291-0/+11
| | | | | | | | | | | | | | | | | | | A virtual xattr "glusterfs.geo-rep.trigger-sync" is provided in glusterfs through changelog translator. Geo-rep triggers a explicit data sync on setting this xattr on a file. Changelog captures a DATA entry on file's gfid on setting this virtual xattr on a file. This is supported only for files. It doesn't support directories. Usage: setfattr -n glusterfs.geo-rep.trigger-sync <file-path> Change-Id: Ia689326ac2dcb31035ffbecad2c548eda4eb9245 BUG: 1176934 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/9337 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* feature/changelog: Fix changelog missing SETATTR entries.Kotresh HR2014-10-291-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | Problem: Valid SETATTR entries are missing in changelog when more than one metadata operation happen on same inode within changelog roll-over time. Cause: Metadata entries with fop num being GF_FOP_NULL are logged in changelog which is of no use. Since slice version checking is done for metadata entries to avoid logging of subsequent entries of same inode falling into same changelog, if the entry with GF_FOP_NULL is logged first, subsequent valid ones will be missed. Solution: Have a boundary condition to log only those fops whose fop number falls between GF_FOP_NULL and GF_FOP_MAXVALUE. Change-Id: Iff585ea573ac5e521a361541c6646225943f0b2d BUG: 1104954 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8964 Reviewed-by: Aravinda VK <avishwan@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Removal of redundant fop color count while draining.Ajeet Jha2014-08-261-5/+0
| | | | | | | | | Change-Id: Ie5581579df63513e636fa3dd92dd90699ea7d999 BUG: 1128093 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/8542 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/changelog: Crash consistency of changelog wrt snapshotAjeet Jha2014-08-261-0/+52
| | | | | | | | | | | | | This patch introduces call-path fop details logging for data operations in CHANGELOG.SNAP. This feature is enabled with barrier-enable notification and disabled with barrier-disable notification. Change-Id: Ib548d34203eb99cea478a6baff402e82251c73a4 BUG: 1128093 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/8533 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: barrier all entry creation fopsVijay Bellur2014-08-251-27/+308
| | | | | | | | | | | | | | | | | | | when a snapshot is taken, there are chances of entry creation fops not being recorded either in changelog or through the recursive ancestry xtime updation by marker. This causes consumers of changelog (primarily geo-replication as of today) to not be aware of these entries after a snapshot is restored. This can lead to inconsistencies. This patch is an interim workaround to barrier creates till changelog becomes completely crash consistent. BUG: 1128093 Change-Id: Ie0dbfd74beecb88df5c2ddf9fc680af91547c3f3 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/8517 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: ajeet jha <ajha@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com>
* feature/changelog: Fix for missing changelogs at backend.Kotresh H R2014-06-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: A few changelog files are missing at the backend during snapshot with changelog enabled. Cause: Race between actual rollover and explicit rollover. Changelog rollover can happen either due to actual or the explict rollover due to snapshot. Actual rollover is controlled by tuneable called rollover-time. The minimum granularity for rollover-time is 1 second Explicit rollover is asynchronous in nature and happens during snapshot. Basically, rollover renames the current CHANGELOG file to CHANGELOG.TIMESTAMP after rollover-time. Let's assume, at time 't1', actual and explicit rollover raced against each other and actual rollover won the race renaming the CHANGELOG file to CHANGELOG.t1 and opens a new CHANGELOG file. An immediate explicit rollover at time 't1' happened with in the same second to rename CHANGELOG file to CHANGELOG.t1 resulting in purging the earlier CHANGELOG.t1 file created by actual rollover. Solution: Adding a delay of 1 sec guarantees unique CHANGELOG.TIMESTAMP during explicit rollover. Thanks Venky, for the all the help in root causing the issue. Change-Id: I8958824e107e16f61be9f09a11d95f8645ecf34d BUG: 1109692 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8196 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: Do not ignore self-heal fops in changelogKotresh H R2014-06-161-2/+3
| | | | | | | | | | | | | | | | | | Problem: Geo-rep fails to sync some files to slave as the changelog entries are missing for those files. Cause: Fops happened when the active brick is down and self- healed later when it came up. Solution: Capture self-heal fops as well in changelog so those entries are not missed. Change-Id: Ibc288779421b5156dd1695e529aba0b602a530e0 BUG: 1109692 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/8070 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Core: Fix issues reported by CppcheckLalatendu Mohanty2014-06-121-11/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed in this patch: [glusterfs/extras/geo-rep/gsync-sync-gfid.c:105]: (error) Resource leak: fp [glusterfs/libglusterfs/src/xlator.c:651]: (error) Uninitialized variable: gfid [glusterfs/libglusterfs/src/xlator.c:652]: (error) Uninitialized variable: gfid [glusterfs/xlators/cluster/ha/src/ha.c:2699]: (error) Possible null pointer dereference: priv [glusterfs/xlators/features/changelog/src/changelog.c:1464]: (error) Possible null pointer dereference: priv [glusterfs/xlators/mgmt/glusterd/src/glusterd-mgmt-handler.c:865]: (error) Possible null pointer dereference: ctx [glusterfs/xlators/mgmt/glusterd/src/glusterd-mgmt-handler.c:194]: (error) Possible null pointer dereference: ctx [glusterfs/xlators/mgmt/glusterd/src/glusterd-syncop.c:1408]: (error) Possible null pointer dereference: this [glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:7002]: (error) Possible null pointer dereference: path_tokens Fixed in 3.4 and 3.5 branch (http://review.gluster.org/#/c/7583/ , http://review.gluster.org/#/c/7605/ will be backported in a separate patch) [glusterfs/xlators/mount/fuse/src/fuse-bridge.c:4688]: (error) Uninitialized variable: finh [glusterfs/xlators/mount/fuse/src/fuse-bridge.c:3081]: (error) Possible null pointer dereference: state [glusterfs/xlators/cluster/dht/src/dht-rebalance.c:1719]: (error) Possible null pointer dereference: ctx [glusterfs/xlators/cluster/stripe/src/stripe.c:4940]: (error) Possible null pointer dereference: local [glusterfs/xlators/mgmt/glusterd/src/glusterd-replace-brick.c:915]: (error) Resource leak: file [glusterfs/xlators/mgmt/glusterd/src/glusterd-replace-brick.c:999]: (error) Resource leak: file [glusterfs/xlators/mgmt/glusterd/src/glusterd-sm.c:248]: (error) Possible null pointer dereference: new_ev_ctx [glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:5297]: (error) Possible null pointer dereference: this [glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:6273]: (error) Possible null pointer dereference: this [glusterfs/xlators/performance/quick-read/src/quick-read.c:586]: (error) Possible null pointer dereference: iobuf [glusterfs/xlators/nfs/server/src/nfs-common.c:89]: (error) Dangerous usage of 'volname' (strncpy doesn't always null-terminate it). False positives [glusterfs/geo-replication/src/gsyncd.c:99]: (error) Memory leak: str [glusterfs/geo-replication/src/gsyncd.c:395]: (error) Memory leak: argv [glusterfs/xlators/nfs/server/src/nlm4.c:1199]: (error) Possible null pointer dereference: fde [glusterfs/xlators/mgmt/glusterd/src/glusterd-geo-rep.c:1659]: (error) Possible null pointer dereference: command [glusterfs/xlators/mgmt/glusterd/src/glusterd-utils.c:7001]: (error) Possible null pointer dereference: path_tokens Insignificant/Don't care [glusterfs/contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 2) requires 'long *' but the argument type is 'unsigned long *'. [glusterfs/contrib/uuid/gen_uuid.c:369]: (warning) %ld in format string (no. 3) requires 'long *' but the argument type is 'unsigned long *'. [glusterfs/extras/test/test-ffop.c:27]: (error) Buffer overrun possible for long command line arguments. [glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:138]: (error) Possible null pointer dereference: __ptr [glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:140]: (error) Possible null pointer dereference: __ptr [glusterfs/xlators/cluster/afr/src/afr-self-heal-common.c:331]: (error) Possible null pointer dereference: __ptr Change-Id: I7696ed1a2a9553b79f9714e10210a8d563a5abd8 BUG: 1091677 Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com> Reviewed-on: http://review.gluster.org/7693 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* feature/changelog: Add timeout to changelog barrier.Kotresh H R2014-06-101-4/+32
| | | | | | | | | | | | | | | | | | | This patch introduces configurable 'changelog-barrier-timeout' in changelog. The default changelog-barrier-timeout is set to 120 sec which is same as barrier-timeout of barrier translator. On this timeout, all the barriered fops are unbarriered. It is safe to have this internal timeout for changelog barrier to prevent it from indefinite barrierring of fops if barrier disable notification does not come from snapshot for some reason. Change-Id: Iad07807935f88222cdefcd46312b4b6579186dce BUG: 1094226 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/7669 Reviewed-by: Kaushal M <kaushal@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog : historical journal consumption.Venky Shankar2014-05-141-4/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Facilitates Glusterfs with the ability to detect file-operations happened in past by scanning the back-end(brick-level) glusterfs journal (changelog). Design: * List of changelogs produces in one perfectly running session are stored in htime file which also holds necessary information about the session start and end time. * Involves fixed sized seeks to identify N'th changelog in the list. * Requires O(log n), (where n is number of changelogs in the list), time to identify the end changelog for the given start-end time interval. Currently the background processing of changelogs is sub optimal. BZ 1097041 tracks the development effort. For complete design, refer the below link: http://lists.nongnu.org/archive/html/gluster-devel/2014-02/msg00206.html Change-Id: I27e49f75e492e843084d0ecaf9130224d08462a0 BUG: 1091961 Signed-off-by: Ajeet Jha <ajha@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6930 Reviewed-by: Kotresh HR <khiremat@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* changelog: in C99 `inline` shouldn't be part of function definitionHarshavardhana2014-05-031-0/+1
| | | | | | | | | | | | | | | | | - C99 standard dictates that `inline` should be used only to hint function implementation not as a definition - http://clang.llvm.org/compatibility.html#inline This also leads to run time issues as 'undefined symbols' - Remember to use adding explicit headers when using an external function Change-Id: I66a9030da7ac38ded6e0b6a977182be977382fa2 BUG: 1089172 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/7655 Reviewed-by: Anand Avati <avati@redhat.com> Tested-by: Anand Avati <avati@redhat.com>
* features/changelog: Barrier in changelog during snapshot.Kotresh H R2014-05-011-23/+217
| | | | | | | | | | | | | | | | | | Changelog barriers unlink, rename, rmdir fops on barrier 'on' notification from glusterfsd mgmt layer and unbarriers the same on barrier 'off' notification during snapshot. Please see the following link for more details. http://www.gluster.org/community/documentation/index.php/Changelog_Design_changes_for_snapshot Signed-off-by: Kotresh H R <khiremat@redhat.com> Change-Id: Iea9c62fafc86242f9404e03679b1941aa9c88c9a Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/7415 Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Varun Shastry <vshastry@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* feature/changelog: Draining of in-transit fops in changelog.Kotresh H R2014-05-011-3/+343
| | | | | | | | | | | | | | | | | | | | This is required for Geo-rep to work with snapshots. Following things are done in this patch. 1. Draining of in-transit fops during changelog rollover. 2. Explicit rollover of changelog when snapshot barrier notification comes. During this, intransit fops are drained and changelog is rolled over. For more details on the purpose of the patch. Please visit following link. http://www.gluster.org/community/documentation/index.php/Changelog_Design_changes_for_snapshot Change-Id: I22690131e19d3027f6d8957178bdc3431b9062f6 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/7216 Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Varun Shastry <vshastry@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* feature/changelog: Change default rollover and fsync-interval timeKotresh H R2014-04-251-2/+2
| | | | | | | | | | | | This will change the following default configurables. 1. rollover-time: from 60 to 15. 2. fsync-interval: from 0 to 5. Change-Id: I9c8db01376967c4f19547ec87f54833f8b139d29 Signed-off-by: Kotresh H R <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/7545 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Venky Shankar <vshankar@redhat.com>
* features/changelog: more changelog fixes.Ajeet Jha2013-12-121-19/+110
| | | | | | | | | | | | | | -> log additional records. -> include FOP number for metadata. -> prevent crash if inode is not found in a fop. Change-Id: I9edd4b71819ebd68c6a2b4150ae279c471d129da BUG: 1036536 Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/6403 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@gmail.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* features/changelog : Improvement in changelog "encoding-change".ajha2013-09-291-16/+1
| | | | | | | | | | | | | | | | | change in encoding method of changelog was critical section for "fop dispatch thread", "roll-over thread" and "reconfigure dispatch thread". In this patch the "encoding-method" is changed by the reconfigure dispatch thread lazily during handle_change, which solves the concurrency among the racing threads. BUG: 1002940 Change-Id: I78c3e8887efa46d0fcc60755cdf4243031cfa3eb Signed-off-by: Ajeet Jha <ajha@redhat.com> Reviewed-on: http://review.gluster.org/5844 Reviewed-by: Venky Shankar <vshankar@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
* core: block unused signals in created threadsAnand Avati2013-09-251-6/+6
| | | | | | | | | | | | | | | Block all signal except those which are set for explicit handling in glusterfs_signals_setup(). Since thread spawning code in libglusterfs and xlators can get called from application threads when used through libgfapi, it is necessary to do this blocking. Change-Id: Ia320f80521a83d2edcda50b9ad414583a0175281 BUG: 1011662 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/5995 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* gsyncd / geo-rep: Introduce basic crawl instrumentationVenky Shankar2013-09-041-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch extends the persistent instrumentation work done by Aravinda (@avishwa), by introducing a handfull of instrumentation variables for crawl. These variables are "pulled up" by glusterd in the event of a geo-replication status cli command and looks something like below: "Uptime=00:21:10;FilesSyned=2982;FilesPending=0;BytesPending=0;DeletesPending=0;" "FilesPending", "BytesPending" and "DeletesPending" are short-lived variables that are non-zero when a changelog is being processes (ie. when an active sync in ongoing). After a successfull changelog process "FilesPending" is summed up into "FilesSynced". The three short-lived variabled are then reset to zero and the data is persisted Additionally this patch also reverts some of the changes made for BZ #986929 (those were not needed). Change-Id: I948f1a0884ca71bc5e5bcfdc017d16c8c54fc30b BUG: 990420 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5441 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* glusterd: Try to start all bricks on 'start force'Kaushal M2013-08-181-4/+6
| | | | | | | | | | | | | | | | | | | A volume would fail to start if any one of the bricks fails staging or fails to start, even with the 'force' option. With this patch, when the 'force' option is given for a volume start, glusterd will continue and start other bricks even if one fails staging or starting. Also did a small fix in changelog, to prevent it crashing when it fails to init. Change-Id: I7efbd9ab13d12d69b0335ae54143fa17586f8f98 BUG: 994375 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/5510 Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: fixes when enabling changelogVenky Shankar2013-07-251-28/+31
| | | | | | | | | | | | | Other enhancements being: * ignore fops made by rebalance * ignore internally triggered fops BUG: 987734 Change-Id: I7dd164ae3c209fdb8ec43a27e67b8846f937c93b Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5380 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/changelog: changelog translatorAvra Sengupta2013-07-221-0/+1487
This is the initial version of the Changelog Translator. What is it ----------- Goal is to capture changes performed on a GlusterFS volume. The translator needs to be loaded on the server (bricks) and captures changes in a plain text file inside a configured directory path (controlled by "changelog-dir", should be somewhere in <export>/.glusterfs/changelog by default). Changes are classified into 3 types: - Data: : TYPE-I - Metadata : TYPE-II - Entry : TYPE-III Changelog file is rolled over after a certain time interval (defauls to 60 seconds) after which a changelog is started. The thing to be noted here is that for a time interval (time slice) multiple changes for an inode are recorded only once (ie. say for 100+ writes on an inode that happens within the time slice has only a single corresponding entry in the changelog file). That way we do not bloat up the changelog and also save lots of writes. Changelog Format ----------------- TYPE-I and TYPE-II changes have the gfid on the entity on which the operation happened. TYPE-III being a entry op requires the parent gfid and the basename. Changelog format has been kept to a minimal and it's upto the consumers to do the heavy loading of figuring out deletes, renames etc.. A single changelog file records all three types of changes, with each change starting with an identifier ("D": DATA, "M": METADATA and "E": ENTRY). Option is provided for the encoding type (See TUNABLES). Consumers ---------- The only consumer as of today would be geo-replication, although backup utilities, self-heal, bit-rot detection could be possible consumers in the future. CLI ---- By default, change-logging is disabled (the translator is present in the server graph but does nothing). When enabled (via cli) each brick starts to log the changes. There are a set of tunable that can be used to change the translators behaviour: - enable/disable changelog (disabled by default) gluster volume set <volume> changelog {on|off} - set the logging directory (<brick>/.glusterfs/changelogs is the default) gluster volume set <volume> changelog-dir /path/to/dir - select encoding type (binary (default) or ascii) gluster volume set <volume> encoding {binary|ascii} - change the rollover time for the logs (60 secs by default) gluster volume set <volume> rollover-time <secs> - when secs > 0, changelog file is not open()'d with O_SYNC flag - and fsync is trigerred periodically every <secs> seconds. gluster volume set <volume> fsync-interval <secs> features/changelog: changelog consumer library (libgfchangelog) A shared library is provided for the consumer of the changelogs for easy acess via APIs. Application can link against this library and request for changelog updates. Conversion of binary logs to human-readable ascii format is also taken care by the library which keeps a copy of the changelog in application provided working directory. Change-Id: I75575fb7f1c53d2bec3dba1a329ea7bb3c628497 BUG: 847839 Original Author: Venky Shankar <vshankar@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5127 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2025-06-29 15:08:22 +0000