summaryrefslogtreecommitdiffstats
path: root/xlators/features/changetimerecorder
Commit message (Collapse)AuthorAgeFilesLines
* features/ctr: Check for NULL localN Balachandran2016-07-273-2/+509
| | | | | | | | | | | | | | | | | | | | | This is a defensive fix to prevent a crash reported during a rename operation. This is not reproducible under normal circumstances. This patch also moves ctr-messages.h to the src dir of the changetimerecorder xlator. Backported from master: http://review.gluster.org/#/c/14964/ Change-Id: I2aac2d4da5752f6a0b45a70e0d97a4d506532ff4 BUG: 1360125 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/15007 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Milind Changire <mchangir@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tiering/ctr: Fix strcpy coverityAvra Sengupta2016-06-141-2/+3
| | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/14696/ Change-Id: I6cffba3e09a023f105dbf2975cc9a3ae1a965c31 BUG: 1346132 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/14696 Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit 3474aa85e399a92d3ee9159a2d066d4bd73611a0) Reviewed-on: http://review.gluster.org/14720 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Rename enum _gf_client_pid to _gf_special_pidAnoop C S2016-05-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Till now _gf_client_pid enum has been used to define special PIDs used by glusterfs clients like shd, quotad etc. In order to have this enum capable of holding all other special PIDs including the one used by trash translator, _gf_client_pid is being renamed to _gf_special_pid. > Change-Id: Id123127771f18aa55d39f335801a54810848d7bc > BUG: 1330616 > Reviewed-on: http://review.gluster.org/14083 > Reviewed-by: Joseph Fernandes > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> (cherry picked from commit 2a6c6de35130328cfb6f95a6d7794dcb01b4004d) Change-Id: Id123127771f18aa55d39f335801a54810848d7bc Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/14097 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* cluster/ec: Mark self-heal fops as internalPranith Kumar K2016-03-041-1/+1
| | | | | | | | | | | | | | | | | | >Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36 >BUG: 1282761 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/12598 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> BUG: 1283757 Change-Id: Ic20d4ee031265305db1a6ed2cf591ce94b7d0749 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/12668 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* all: reduce "inline" usageKaleb S KEITHLEY2016-01-182-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | There are three kinds of inline functions: plain inline, extern inline, and static inline. All three have been removed from .c files, except those in "contrib" which aren't our problem. Inlines in .h files, which are overwhelmingly "static inline" already, have generally been left alone. Over time we should be able to "lower" these into .c files, but that has to be done in a case-by-case fashion requiring more manual effort. This part was easy to do automatically without (as far as I can tell) any ill effect. In the process, several pieces of dead code were flagged by the compiler, and were removed. backport of Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155, http://review.gluster.org/11769, BUG: 1245331 Change-Id: Iba1efb0bc578ea4a5e9bf76b7bd93dc1be9eba44 BUG: 1283302 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12646 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* ctr/sql: Providing for vol set for sqlcachesize and sqlWALsize and skip ↵Joseph Fernandes2015-12-223-29/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | recording path 1. Providing vol set option for cache size and wal autocheck point so that performance can be tuned. 2. Removed recording of file path in the db. Trimming database columns. Path need not be stored in the db, as PARGFID, GFID, Basename is suffice to derive the path during migration. Backport of http://review.gluster.org/12972 > Change-Id: I2cb590451a6d244bc91fe66c6dbffe2c2059dfb8 > BUG: 1293034 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12972 > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ia1c109983ec6ce75ed27b8c08f454f5b6283c31d BUG: 1293659 Reviewed-on: http://review.gluster.org/13067 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Check filename in ctr_lookup for nameless lookupJoseph Fernandes2015-12-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Check filename in ctr_lookup for nameless lookup Backport of http://review.gluster.org/12760 > Change-Id: I1ab3f658e3b9fee708f6986d1990f16db920d2fb > BUG: 1285663 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12760 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I2dccf8fe556c42623dc0aa89459dd2aa2a038380 BUG: 1290295 Reviewed-on: http://review.gluster.org/12935 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/glusterd : Validation for frequency thresholds and record-countersJoseph Fernandes2015-12-023-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) if record-counters is set to off check if both the frequency thresholds are non-zero, then pop an error message, with volume set failed. 2) if record-counters is set to on check if both the frequency thresholds are zero, then pop an note, but volume set is not failed. 3) If any of the frequency thresholds are set to a non-zero value, switch record-counters on, if not already on 4) If both the frequency thresholds are set to zero, switch record-counters off, if not already off NOTE: In this fix we have 1) removed unnecessary ctr vol set options. 2) changed ctr_hardlink_heal_expire_period to ctr_lookupheal_link_timeout Backport of http://review.gluster.org/12780 > Change-Id: Ie7ccfd3f6e021056905a79de5a3d8f199312f315 > BUG: 1286346 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/12780 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I35621ed3cc1ad20a75ef16e0b95453b8c6120852 BUG: 1287560 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12849 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Correcting rename logicJoseph Fernandes2015-11-263-22/+313
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When a file with old_file_name and GFID_1 is renamed with a new_file_name which already exists and with GFID_2, this is what happens in linux internaly. a. "new_file_name" is unlinked for GFID_2 b. a hardlink "new_file_name" is created to GFID_1 c. "old_file_name" hardlink is unlinked for GFID_2. Well this is all internal to linux, and gluster just issues a rename system call at POSIX layer. But CTR Xlator doesn't delete the entries corresponding to the "new_file_name" and GFID_2. Thus leaving the stale entry in the DB. The following are the implications. a. Promotion are tried on these stale entries which will fail and show false results in the status of migration, b. GFID_2 Files with 2 hardlinks, which will have only one hardlink after the rename will not be promoted or demoted as the DB shows 2 entries. Solution: Delete the older database entry for the replaced hardlink Backport of http://review.gluster.org/12711 > Change-Id: I4eafa0872253e29ff1f0bec4283bcfc579ecf0e2 > BUG: 1284090 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12711 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ic35348303ec21f9bd19f20a48f3141449349668b BUG: 1285688 Reviewed-on: http://review.gluster.org/12762 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Providing option to record or ignore metadata heatJoseph Fernandes2015-11-163-54/+235
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we heat up a file for both data and metadata write. Here we provide a ctr xlator option called "ctr-record-metadata-heat" were the admin can decide on recording metadata heat i.e heatup a file on metadata writes or not. Metadata data operation are a. setattr: explicit changing of atime/mtime using utimes, changing of posix permissions of the file b. rename: Renaming a file, c. unlink, link: adding or deleting hardlinks d. xattrs: setting or removal of xattrs. NOTE: atime, mtime and ctime change through writev, readv, truncate, mknod and create will not be considered here as these fops are data and primary metadata fops. Defaultly "ctr-record-metadata-heat" is off. Admin can switch it on using gluster volume set command. Backport of http://review.gluster.org/12540 > Change-Id: I91157509255dd5cb429cda2b6d4f64582e155e7b > BUG: 1279166 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12540 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I986c319f0cc337b0692a1dd02f71254e786afac4 BUG: 1282315 Reviewed-on: http://review.gluster.org/12582 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: ignoring bitrot scrubber fopsJoseph Fernandes2015-11-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | We had missed adding GF_CLIENT_PID_SCRUB to the internal fops list of bitrot. Doing that in this fix backport of http://review.gluster.org/#/c/12555/ > Change-Id: I0c7de37e2bf625fb577a32a599a885ee95f5d3bd > BUG: 1278326 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12555 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Venky Shankar <vshankar@redhat.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I30384e970531d2f23d32a512ac2986ca6661af7c BUG: 1278640 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12568 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* marker: do remove xattr only for last linkvmallika2015-11-091-8/+8
| | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/12033/ With unlink, rename, rmdir, contribution xattrs are removed. If the file is a last link then remove_xattr will fail with ENOENT. So it better to perform remove_xattr only if there are more links to the file > Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d > BUG: 1257694 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: Icf5fdd86bbb8eef0adeb9518e89e5b612e9e0705 BUG: 1279331 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12549 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/libgfdb: Replacing ASCII query file with binaryJoseph Fernandes2015-11-082-11/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, when the database was queried we used to save all the queried records in an ASCII format in the query file. This caused issues like filename having ASCII delimiter and used to take a lot of space. The tier.c file also had a lot of parsing code. Here we changed the format of the query file to binary. All the logic of serialization and formating of query record is done by libgfdb. Libgfdb provides API, gfdb_write_query_record() and gfdb_read_query_record(), which the user i.e tier migrator and CTR xlator can use to write to and read from query file. With this binary format we save on disk space i.e reduce to 50% atleast as we are saving GFID's in binary format 16 bytes and not the string format which takes 36 bytes + We are not saving path of the file + we are also saving on ASCII delimiters. The on disk format of query record is as follows, +---------------------------------------------------------------------------+ | Length of serialized query record | Serialized Query Record | +---------------------------------------------------------------------------+ 4 bytes Length of serialized query record | | -------------------------------------------------| | | V Serialized Query Record Format: +---------------------------------------------------------------------------+ | GFID | Link count | <LINK INFO> |..... | FOOTER | +---------------------------------------------------------------------------+ 16 B 4 B Link Length 4 B | | | | -----------------------------| | | | | | V | Each <Link Info> will be serialized as | +-----------------------------------------------+ | | PGID | BASE_NAME_LENGTH | BASE_NAME | | +-----------------------------------------------+ | 16 B 4 B BASE_NAME_LENGTH | | | ------------------------------------------------------------------------| | | V FOOTER is a magic number 0xBAADF00D indicating the end of the record. This also serves as a serialized schema validator. Backport of http://review.gluster.org/#/c/12354/ > Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a > BUG: 1272207 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12354 > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I170c579027f2594a58706f826e3ddf89e34022f4 BUG: 1263619 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12535 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Ignore bitrot related fopsJoseph Fernandes2015-11-071-0/+10
| | | | | | | | | | | | | | Ignore bitrot related fops since they are internal fops. Backport of http://review.gluster.org/#/c/12512/ Change-Id: I5b676fe450d266b95bbd25aeca0d54436e098917 BUG: 1278640 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12533 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Correcting the internal fop calculationJoseph Fernandes2015-11-062-17/+28
| | | | | | | | | | | | | | Correcting the internal fop calculation method, as it had wrong logic. backport of http://review.gluster.org/#/c/12418/ Change-Id: I1deeae8b67dd967159853b494e89a3f46572c962 BUG: 1275483 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12423 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* features/ctr: Reduce the log-level for ctr-disabled messageAnoop C S2015-10-281-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/11592/ Since by default CTR translator is disabled, the following log message is being displayed in brick logs. [ctr-helper.c:256:extract_ctr_options] 0-gfdbdatastore: CTR Xlator is disabled. Therefore this change is to reduce the log-level to INFO. >Change-Id: I3b82d6b0dc0445286f91490fb497167a36914a2b >BUG: 1241379 >Signed-off-by: Anoop C S <anoopcs@redhat.com> >Reviewed-on: http://review.gluster.org/11592 >Reviewed-by: Joseph Fernandes >Tested-by: Joseph Fernandes >Tested-by: NetBSD Build System <jenkins@build.gluster.org> BUG: 1275157 Signed-off-by: Anoop C S <anoopcs@redhat.com> (cherry picked from commit b6a7a44079c724cbf6fb24c7ac83892f551dc5f6) Change-Id: I1bcea5258da525b955606f2353f749b2e37449cc Reviewed-on: http://review.gluster.org/12424 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: CTR DB named lookup heal of cold tier during attach tierJoseph Fernandes2015-10-102-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Heal hardlink in the db for already existing data in the cold tier during attach tier. i.e during fix layout do lookup to files in the cold tier. CTR xlator on the brick/server side does db update/insert of the hardlink on a namelookup. Currently the namedlookup is done synchronous to the fixlayout that is triggered by attach tier. This is not performant, adding more time to fixlayout. The performant approach is record the hardlinks on a compressed datastore and then do the namelookup asynchronously later, giving the ctr db eventual consistency master patch : http://review.gluster.org/#/c/11828/ >>Change-Id: I4ffc337fffe7d447804786851a9183a51b5044a9 >>BUG: 1252586 >>Signed-off-by: Joseph Fernandes <josferna@redhat.com> >>Reviewed-on: http://review.gluster.org/11828 >>Tested-by: Gluster Build System <jenkins@build.gluster.com> >>Reviewed-by: Dan Lambright <dlambrig@redhat.com> >>Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I61b185a54ae4e8c1d82804b95a278bfbea870987 BUG: 1261146 Reviewed-on: http://review.gluster.org/12331 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Solution for db locks for tier migrator and ctr using sqlite ↵Joseph Fernandes2015-10-092-37/+302
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | version less than 3.7 i.e rhel 6.7 Problem: On RHEL 6.7, we have sqlite version 3.6.2 which doesnt support WAL journaling mode, as this journaling mode is only available in sqlite 3.7 and above. As a result we cannot have to progreses concurrently accessing sqlite, without running into db locks! Well WAL is also need for performace on CTR side. Solution: This solution is to use CTR db connection for doing queries when WAL mode is absent. i,e tier migrator will send sync_op ipc calls to CTR, which in turn will do the query and create/update the query file suggested by tier migrator. Pending: Well this solution will stop the db locks but the performance is still an issue for CTR. We are developing an in-Memory Transaction Log (iMeTaL) which will help boost the CTR performance by doing in memory udpates on the IO path and later flush the updates to the db in a batch/segment flush. Master patch: http://review.gluster.org/#/c/12191 >> Change-Id: Ie3149643ded159234b5cc6aa6cf93b9022c2f124 >> BUG: 1240577 >> Signed-off-by: Joseph Fernandes <josferna@redhat.com> >> Signed-off-by: Dan Lambright <dlambrig@redhat.com> >> Signed-off-by: Joseph Fernandes <josferna@redhat.com> >> Reviewed-on: http://review.gluster.org/12191 >> Tested-by: Gluster Build System <jenkins@build.gluster.com> >> Reviewed-by: Luis Pabon <lpabon@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ie8c7a7e9566244c104531b579126bb57fbc6e32b BUG: 1270123 Reviewed-on: http://review.gluster.org/12325 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Solving DB Lock issue due to write contention from db connectionsJoseph Fernandes2015-09-112-1/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of 12031. > Problem: The DB on the brick is been accessed by CTR, for write and > tier migrator, for read and write. The write from tier migrator is reseting > the heat counters after a cycle. Since we are using sqlite, two connections > trying to write would cause a db lock contention. As a result CTR used to fail > to update the db. > Solution: Using the same db connection of CTR for reseting the heat counters. > 1) Introducted a new IPC FOP for CTR > 2) After the query do a ipc syncop to the underlying client xlator associated > to the brick. > 3) CTR in brick will catch the IPC FOP and cleat the heat counters. > Change-Id: I53306bfc08dcdba479deb4ccc154896521336150 > BUG: 1260730 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12031 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Conflicts: xlators/cluster/dht/src/tier.c Change-Id: I88aa289cdf21e216b42c3d8ccfb4e7e828b43772 BUG: 1262341 Reviewed-on: http://review.gluster.org/12161 Reviewed-by: Joseph Fernandes Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* ctr:Avoid assertion failure in lookupMohammed Rafi KC2015-08-192-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | in ctr_lookup, the loc variable need not be comes with pargfid, though there is a parent for the inode. The same for loc->name also. From this patch, we will generate loc->name from loc->path Back port of : >Change-Id: I24a79554748139504ec09f77930f8208d3805977 >BUG: 1236128 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/11459 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Joseph Fernandes (cherry picked from commit 2e44d1580497eb75f325ad3104249a425ddf592a) Change-Id: Ia926bee45059ec97bbe5120f19999b9d5a71aa88 BUG: 1254439 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/11947 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* libgfdb/ctr: Fixing rename or update of linkJoseph Fernandes2015-08-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11584 to 3.7.2 Fixing rename or updating of link in 1) libgfdb : handling return from updatelink 2) ctr xlator: correcting freeing of oldbase name in hardlink list in the update link >Change-Id: I1789c492b81876123cd456051a9ae495e876ad46 >BUG: 1241054 >Signed-off-by: Joseph Fernandes <josferna@redhat.com> >Reviewed-on: http://review.gluster.org/11584 >Tested-by: Joseph Fernandes >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I6da86c06e390f62fe8a44ade7b1efda5771e9ca8 BUG: 1242536 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/11649 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Ignore creation of T file and Ctr Lookup heal improvememntsDan Lambright2015-06-275-22/+256
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a back port of 11334 1) Ignore creation of T file in ctr_mknod 2) Ignore lookup for T file in ctr_lookup 3) Ctr_lookup: a. If the gfid and pgfid in empty dont record b. Decreased log level for multiple heal attempts c. Inode/File heal happens after an expiry period, which is configurable. d. Hardlink heal happens after an expiry period, which is configurable. > Change-Id: Id8eb5092e78beaec22d05f5283645081619e2452 > BUG: 1235269 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/11334 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia28a5cf975e41d318906f707deca447aaa35630f BUG: 1236288 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/11446 Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com>
* changetimerecorder : port log messages to a new frameworkMohamed Ashiq2015-06-174-125/+198
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/10938/ Cherry picked from 00f9a61fe8884062c141edd662424625d349a377 >Change-Id: I66e7ccc5e62482c3ecf0aab302568e6c9ecdc05d >BUG: 1194640 >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> >Reviewed-on: http://review.gluster.org/10938 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Joseph Fernandes >Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Change-Id: I66e7ccc5e62482c3ecf0aab302568e6c9ecdc05d BUG: 1217722 Signed-off-by: Mohamed Ashiq <ashiq333@gmail.com> Reviewed-on: http://review.gluster.org/11195 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes
* CTR/Libgfdb: Log typo fixJoseph Fernandes2015-05-101-1/+1
| | | | | | | | | | | Log typo fix for CTR Xlator and Libgfdb Change-Id: Ia39069a5ce9c48bbee937f1b5c5d749a30c9ac56 BUG: 1220100 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10742 Reviewed-by: N Balachandran <nbalacha@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* ctr/xlator: Named lookup heal of pre-existing files, before ctr was ON.Joseph Fernandes2015-05-086-4/+945
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The CTR xlator records file meta (heat/hardlinks) into the data. This works fine for files which are created after ctr xlator is switched ON. But for files which were created before CTR xlator is ON, CTR xlator is not able to record either of the meta i.e heat or hardlinks. Thus making those files immune to promotions/demotions. Solution: The solution that is implemented in this patch is do ctr-db heal of all those pre-existent files, using named lookup. For this purpose we use the inode-xlator context variable option in gluster. The inode-xlator context variable for ctr xlator will have the following, a. A Lock for the context variable b. A hardlink list: This list represents the successful looked up hardlinks. These are the scenarios when the hardlink list is updated: 1) Named-Lookup: Whenever a named lookup happens on a file, in the wind path we copy all required hardlink and inode information to ctr_db_record structure, which resides in the frame->local variable. We dont update the database in wind. During the unwind, we read the information from the ctr_db_record and , Check if the inode context variable is created, if not we create it. Check if the hard link is there in the hardlink list. If its not there we add it to the list and send a update to the database using libgfdb. Please note: The database transaction can fail(and we ignore) as there already might be a record in the db. This update to the db is to heal if its not there. If its there in the list we ignore it. 2) Inode Forget: Whenever an inode forget hits we clear the hardlink list in the inode context variable and delete the inode context variable. Please note: An inode forget may happen for two reason, a. when the inode is delete. b. the in-memory inode is evicted from the inode table due to cache limits. 3) create: whenever a create happens we create the inode context variable and add the hardlink. The database updation is done as usual by ctr. 4) link: whenever a hardlink is created for the inode, we create the inode context variable, if not present, and add the hardlink to the list. 5) unlink: whenever a unlink happens we delete the hardlink from the list. 6) mknod: same as create. 7) rename: whenever a rename happens we update the hardlink in list. if the hardlink was not present for updation, we add the hardlink to the list. What is pending: 1) This solution will only work for named lookups. 2) We dont track afr-self-heal/dht-rebalancer traffic for healing. > http://review.gluster.org/#/c/10370/ > Cherry picked from commit cb11dd91a6cc296e4a3808364077f4eacb810e48 > Change-Id: Ia4bbaf84128ad6ce8c3ddd70bcfa82894c79585f > BUG: 1212037 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/10370 > Reviewed-by: Jeff Darcy <jdarcy@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: I367aa46c3f4b8f912248fb8be75866507f2538df BUG: 1219075 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/10370 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10615 Tested-by: Vijay Bellur <vbellur@redhat.com>
* ctr/libgfdb: Performance enhancer for unlink/create/rename/link fopsJoseph Fernandes2015-05-073-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1) ctr_link_consistency option for ctr xaltor is provided so that the user can choose to switch it on or off. /* For link consistency we do a double update i.e mark the link * during the wind and during the unwind we update/delete the link. * This has a performance hit. We give a choice here whether we need * link consistency to be spoton or not using link_consistency flag. * This will have only one link update */ 2) In delete the wind time recording is moved to unwind path. /* Special performance case: * Updating wind time in unwind for delete. This is done here * as in the wind path we will not know whether its the last * link or not. For a last link there is not use to update any * wind or unwind time!*/ > http://review.gluster.org/#/c/10170/ > Cherry picked from commit 606d9734543208542afcf9df982bf2d560235ef6 > Change-Id: I209472fb816f939db4a868b97ba053b028f17ea6 > BUG: 1217786 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/10170 > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I4a89ef80875f36cff91520f712e1f47fde258a63 BUG: 1219066 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10170 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10614 Tested-by: NetBSD Build System Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/trash : Notify CTR translator if an unlink happens to a fileJiffin Tony Thottan2015-05-041-13/+9
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/9989/ This implementation is same as the posix_unlink_cbk() where CTR sends a request during a unlink to send the number of links to the inode and posix obliges sending it using the unwind xdata dict. For Trash xlator a unlink is stat + mkdir(if parent is not present) + rename. And hence this is handled in trash_unlink_rename_cbk(). Change-Id: I402e83567b88e3c9fe171379693c82937af567f9 BUG: 1218032 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Anoop C S <achiraya@redhat.com> Reviewed-on: http://review.gluster.org/10513 Reviewed-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* ctr : Fix for heating of files during promotion/demotionJoseph Fernandes2015-04-083-44/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fix will solve the heating of the files during the promotion or demotion. Promotion: ~~~~~~~~~ When a file gets promoted it get the current time stamp during creation only, but following writes or reads during the migration wont heat the file. Demotion: ~~~~~~~~ When a file gets demoted it get the wind/unwind time stamp is set to zero. The following writes or reads during the migration wont heat the file. What is remaining ? ~~~~~~~~~~~~~~~~~ Bug 1209129 ( https://bugzilla.redhat.com/show_bug.cgi?id=1209129 ) Inspite of this fix there is still a issue remaining, i.e the heat of the file is not keep intact during a internal rebalance activity i.e a rebalance within a tier. Change-Id: I01e82dc226355599732d40e699062cee7960b0a5 BUG: 1207867 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/10080 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cluster/dht: fix tier.c problems found prior to feature freezeDan Lambright2015-04-062-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch resolves tiering translator issues taken from the list in bug 1203776. These issues have been selected to be fixed first. The rest will be fixed in a subsequent patch (or are not a problem). 3. Replace hardcoded #defines of promote/demote file names 6. Use loc_wipe() in migrate_using_query_file() 9. Only promote/demote files on the same node on which they reside. 14. Replace calloc with GF_CALLOC in tier.c and ensure freeing done properly. 15. Handle if parse_query_str fails 22. Only load gfdb library on server side, remove SQL references from client. Change-Id: I6563b11e58ab2e4c6b1ce44db755781ad6d930fb BUG: 1203776 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/9987 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-043-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* features/ctr: Removing rpc-lib, rpc-xdr and api from CPPFLAGSAnoop C S2015-03-251-6/+1
| | | | | | | | | | | | | Changetimerecorder doesn't seem to use rpc-lib, rpc-xdr and gfapi in the source. So removing those from Makefile.am Change-Id: I21c71db6212c10ba3821c6c456958a45c5312d41 BUG: 1198849 Signed-off-by: Anoop C S <achiraya@redhat.com> Reviewed-on: http://review.gluster.org/9997 Reviewed-by: Joseph Fernandes <josferna@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* tiering/CTR:build CTR only if tiering is enabledMohammed Rafi KC2015-03-201-1/+5
| | | | | | | | | | | | Thanks to Niels for this fix Change-Id: I9a13c3de3ed5d4eb06c6af61a2519bf27f1b6259 BUG: 1194753 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/9957 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* build: pass the correct CFLAGS and LDFLAGS when linking (to) libgfdbNiels de Vos2015-03-191-3/+2
| | | | | | | | | Change-Id: Id9a7d0f457d9759ab7d0a52a4000b5ae36d211f8 BUG: 1194753 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/9946 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* Adding ChangeTimeRecorder(CTR) Xlator to GlusterFSJoseph Fernandes2015-03-197-0/+1959
********************************************************************** ChangeTimeRecorder(CTR) Xlator | ********************************************************************** ChangeTimeRecorder(CTR) is server side xlator(translator) which sits just above posix xlator. The main role of this xlator is to record the access/write patterns on a file residing the brick. It records the read(only data) and write(data and metadata) times and also count on how many times a file is read or written. This xlator also captures the hard links to a file(as its required by data tiering to move files). CTR Xlator is the consumer of libgfdb. To Enable/Disable CTR Xlator: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ gluster volume set <volume-name> features.ctr-enabled {on/off} To Enable/Disable Frequency Counter Recording in CTR Xlator: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ gluster volume set <volume-name> features.record-counters {on/off} Change-Id: I5d3cf056af61ac8e3f8250321a27cb240a214ac2 BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9935 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-25 17:10:22 +0000