summaryrefslogtreecommitdiffstats
path: root/libglusterfs
Commit message (Collapse)AuthorAgeFilesLines
* storage/posix: Implement .unlink directoryAshish Pandey2016-01-252-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: For EC volume, If a file descriptor is open and file has been unlinked, any further write on that fd will fail. When a write request comes, EC internally reads some blocks using anonymous fd. This read will fail as the file has already been unlinked. Solution: To solve this issue, we are using .unlink directory to keep track of unlinked file. If a file is to be unlinked while its fd is open, move this to .unlink directory and unlink it from .glusterfs and real path. Once all the fd will be closed, remove this entry form .unlink directory. master - http://review.gluster.org/#/c/12816/ Change-Id: I8344edb0d340bdb883dc46458c16edbc336916b9 BUG: 1291557 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Reviewed-on: http://review.gluster.org/12968 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* protocol/server: Fix memory corruption during client-table-expand.Raghavendra G2016-01-191-0/+1
| | | | | | | | | | | | | | | | | | gf_client_clienttable_expand frees up old entries after expanding. So, cliententry should be reassigned to a free slot in new array of cliententries. Earlier it used to point to a slot in oldentries resulting in a use-after-free bug. Thanks to Pranith for the assistance provided. Change-Id: Iabe40c7df475471a7df7bccb302aef496ded3f1c BUG: 1299314 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/13249 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* all: reduce "inline" usageKaleb S KEITHLEY2016-01-188-45/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | There are three kinds of inline functions: plain inline, extern inline, and static inline. All three have been removed from .c files, except those in "contrib" which aren't our problem. Inlines in .h files, which are overwhelmingly "static inline" already, have generally been left alone. Over time we should be able to "lower" these into .c files, but that has to be done in a case-by-case fashion requiring more manual effort. This part was easy to do automatically without (as far as I can tell) any ill effect. In the process, several pieces of dead code were flagged by the compiler, and were removed. backport of Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155, http://review.gluster.org/11769, BUG: 1245331 Change-Id: Iba1efb0bc578ea4a5e9bf76b7bd93dc1be9eba44 BUG: 1283302 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12646 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* core : Use correct path in dlopen for socket.soAtin Mukherjee2015-12-272-1/+7
| | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/12656 This patch fixes the path for socket.so file while loading the so dynamically. Also for config.memory-accounting & config.transport voltype is changed to glusterd to fix the warning message coming from xlator_volopt_dynload Change-Id: I0f7964814586f2018d4922b23c683f4e1eb3098e BUG: 1283833 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/12656 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/12670 Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* ctr/sql: Providing for vol set for sqlcachesize and sqlWALsize and skip ↵Joseph Fernandes2015-12-226-142/+211
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | recording path 1. Providing vol set option for cache size and wal autocheck point so that performance can be tuned. 2. Removed recording of file path in the db. Trimming database columns. Path need not be stored in the db, as PARGFID, GFID, Basename is suffice to derive the path during migration. Backport of http://review.gluster.org/12972 > Change-Id: I2cb590451a6d244bc91fe66c6dbffe2c2059dfb8 > BUG: 1293034 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12972 > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ia1c109983ec6ce75ed27b8c08f454f5b6283c31d BUG: 1293659 Reviewed-on: http://review.gluster.org/13067 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* glusterd/afr : Readdirp performance improvementAnuradha Talur2015-12-211-0/+2
| | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/12467/ Add xlator options to index xlator with xattrs that it needs to keep track of. Change-Id: If818673be5e626f77e65cc3a340f8cdd624179c2 BUG: 1287531 Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/12467 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Signed-off-by: Anuradha Talur <atalur@redhat.com> Reviewed-on: http://review.gluster.org/12848
* tier/libgfdb/sql: Correcting logic in sql query for readJoseph Fernandes2015-12-041-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Correcting query in the where clause from "a & b | c" to "a & (b | c)" where "a" is the condition to join the gf_file_tb and gf_flink_tb through gfids "b" is the condition for the write heat and "c" is the condition for read heat Backport of http://review.gluster.org/12823 > Change-Id: I99226d82b0efb68fbef3a40f02b215bb2b4370d6 > BUG: 1286656 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/12823 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I70533c8589a6cd67e8737e110af0a23abed9013d BUG: 1287583 Reviewed-on: http://review.gluster.org/12868 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Correcting rename logicJoseph Fernandes2015-11-261-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: When a file with old_file_name and GFID_1 is renamed with a new_file_name which already exists and with GFID_2, this is what happens in linux internaly. a. "new_file_name" is unlinked for GFID_2 b. a hardlink "new_file_name" is created to GFID_1 c. "old_file_name" hardlink is unlinked for GFID_2. Well this is all internal to linux, and gluster just issues a rename system call at POSIX layer. But CTR Xlator doesn't delete the entries corresponding to the "new_file_name" and GFID_2. Thus leaving the stale entry in the DB. The following are the implications. a. Promotion are tried on these stale entries which will fail and show false results in the status of migration, b. GFID_2 Files with 2 hardlinks, which will have only one hardlink after the rename will not be promoted or demoted as the DB shows 2 entries. Solution: Delete the older database entry for the replaced hardlink Backport of http://review.gluster.org/12711 > Change-Id: I4eafa0872253e29ff1f0bec4283bcfc579ecf0e2 > BUG: 1284090 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12711 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ic35348303ec21f9bd19f20a48f3141449349668b BUG: 1285688 Reviewed-on: http://review.gluster.org/12762 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* features/changelog: Capture FXATTROP and XATTROP in changelogKotresh HR2015-11-241-0/+4
| | | | | | | | | | | | | | | | | GEO-REP INTEROP WITH SHARD FEATURE shard xlator updates size of the file using FXATTROP or XATTROP. Hence record the same in changelog. BUG: 1284453 Change-Id: I2f14b6075f863c0bed3ee2a159085b9b536a756d Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: http://review.gluster.org/12225 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/12732 Reviewed-by: Aravinda VK <avishwan@redhat.com>
* cluster/ec: Allow read fops to be processed in parallelXavier Hernandez2015-11-241-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently ec only sends a single read request at a time for a given inode. Since reads do not interfere between them, this patch allows multiple concurrent read requests to be sent in parallel. This is a backport of these patches: > Change-Id: If853430482a71767823f39ea70ff89797019d46b > BUG: 1245689 > Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> > Reviewed-on: http://review.gluster.org/11742 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > > Change-Id: I6042129f09082497b80782b5704a52c35c78f44d > BUG: 1276031 > Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Change-Id: I1b1146d1fd1828b12bfc566cd76e5ea110f8909b BUG: 1251467 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/12447 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* glusterd: cli command implementation for bitrot scrub statusGaurav Kumar Garg2015-11-222-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch is backport of: http://review.gluster.org/10231 CLI command for bitrot scrub status will be : gluster volume bitrot <volname> scrub status Above command will show the statistics of bitrot scrubber. Upon execution of this command it will show some common scrubber tunable value of volume <VOLNAME> followed by statistics of scrubber statistics of individual nodes. sample ouput for single node: Volume name : <VOLNAME> State of scrub: Active Scrub frequency: biweekly Bitrot error log location: /var/log/glusterfs/bitd.log Scrubber error log location: /var/log/glusterfs/scrub.log ========================================================= Node name: Number of Scrubbed files: Number of Unsigned files: Last completed scrub time: Duration of last scrub: Error count: ========================================================= This is just infrastructure. list of bad file, last scrub time, error count value will be taken care by http://review.gluster.org/#/c/12503/ and http://review.gluster.org/#/c/12654/ patches. >> Change-Id: I3ed3c7057c9d0c894233f4079a7f185d90c202d1 >> BUG: 1207627 >> Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> >> Reviewed-on: http://review.gluster.org/10231 >> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> >> Tested-by: NetBSD Build System <jenkins@build.gluster.org> >> Tested-by: Gluster Build System <jenkins@build.gluster.com> Change-Id: I45ed94e5e0e78a1e007c30eb0b252f74cf3c9187 BUG: 1283881 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/12704 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* tier/ctr: Providing option to record or ignore metadata heatJoseph Fernandes2015-11-161-3/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we heat up a file for both data and metadata write. Here we provide a ctr xlator option called "ctr-record-metadata-heat" were the admin can decide on recording metadata heat i.e heatup a file on metadata writes or not. Metadata data operation are a. setattr: explicit changing of atime/mtime using utimes, changing of posix permissions of the file b. rename: Renaming a file, c. unlink, link: adding or deleting hardlinks d. xattrs: setting or removal of xattrs. NOTE: atime, mtime and ctime change through writev, readv, truncate, mknod and create will not be considered here as these fops are data and primary metadata fops. Defaultly "ctr-record-metadata-heat" is off. Admin can switch it on using gluster volume set command. Backport of http://review.gluster.org/12540 > Change-Id: I91157509255dd5cb429cda2b6d4f64582e155e7b > BUG: 1279166 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12540 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I986c319f0cc337b0692a1dd02f71254e786afac4 BUG: 1282315 Reviewed-on: http://review.gluster.org/12582 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/libgfdb: Extending log level flexibity in libgfdbJoseph Fernandes2015-11-112-86/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | Extending log level flexibity to relevant fops and operations This is an extension to http://review.gluster.org/#/c/12491/ Backport of http://review.gluster.org/#/c/12567/ > Change-Id: I33b2f7732f89f52569fb99baa692c7e3bb4c7ab1 > BUG: 1277352 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12567 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Idcde04a65775ce98367967faf8a53b0f07ff022e BUG: 1279059 Reviewed-on: http://review.gluster.org/12569 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* gfapi: xattr key length check to avoid brick crashMilind Changire2015-11-091-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | Added check to test if xattr key length > max allowed for OS distribution and return: EINVAL if xattr name pointer is NULL or 0 length ENAMETOOLONG if xattr name length > max allowed for distribution Function exit path for invalid input is via label "out:" for mandatory __GLFS_EXIT_FS. Typically the VFS does this in the kernel for us. But since we are bypassing the VFS by providing the libgfapi to talk directly to the brick process, we need to add such checks. Change-Id: I610a8440871200ae4640351902b752777a3ec0c2 BUG: 1272926 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: http://review.gluster.org/12207 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> (cherry picked from commit 47d8d2fc9c88c95dfcae2c5c06e6eb3b1ce03a92) Reviewed-on: http://review.gluster.org/12387 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* marker: do remove xattr only for last linkvmallika2015-11-091-3/+3
| | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/12033/ With unlink, rename, rmdir, contribution xattrs are removed. If the file is a last link then remove_xattr will fail with ENOENT. So it better to perform remove_xattr only if there are more links to the file > Change-Id: Ifc1e7fda4d310fd87f6f28a635c9ea78b8f3929d > BUG: 1257694 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: Icf5fdd86bbb8eef0adeb9518e89e5b612e9e0705 BUG: 1279331 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12549 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/libgfdb: Replacing ASCII query file with binaryJoseph Fernandes2015-11-0810-279/+1039
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Earlier, when the database was queried we used to save all the queried records in an ASCII format in the query file. This caused issues like filename having ASCII delimiter and used to take a lot of space. The tier.c file also had a lot of parsing code. Here we changed the format of the query file to binary. All the logic of serialization and formating of query record is done by libgfdb. Libgfdb provides API, gfdb_write_query_record() and gfdb_read_query_record(), which the user i.e tier migrator and CTR xlator can use to write to and read from query file. With this binary format we save on disk space i.e reduce to 50% atleast as we are saving GFID's in binary format 16 bytes and not the string format which takes 36 bytes + We are not saving path of the file + we are also saving on ASCII delimiters. The on disk format of query record is as follows, +---------------------------------------------------------------------------+ | Length of serialized query record | Serialized Query Record | +---------------------------------------------------------------------------+ 4 bytes Length of serialized query record | | -------------------------------------------------| | | V Serialized Query Record Format: +---------------------------------------------------------------------------+ | GFID | Link count | <LINK INFO> |..... | FOOTER | +---------------------------------------------------------------------------+ 16 B 4 B Link Length 4 B | | | | -----------------------------| | | | | | V | Each <Link Info> will be serialized as | +-----------------------------------------------+ | | PGID | BASE_NAME_LENGTH | BASE_NAME | | +-----------------------------------------------+ | 16 B 4 B BASE_NAME_LENGTH | | | ------------------------------------------------------------------------| | | V FOOTER is a magic number 0xBAADF00D indicating the end of the record. This also serves as a serialized schema validator. Backport of http://review.gluster.org/#/c/12354/ > Change-Id: I9db7416fd421e118dd44eafab8b535caafe50d5a > BUG: 1272207 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12354 > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I170c579027f2594a58706f826e3ddf89e34022f4 BUG: 1263619 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12535 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Ignore CTR Lookup heal insert errorsJoseph Fernandes2015-11-073-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | CTR doesnt read from the DB, so to make sure that file records are created it does a heal during a lookup. It remembers the decision in the inode context cache and retrys periodically. When the volume is restarted it looses all the inode cache from the previous time and CTR lookup heals tries the heal again, but this time it finds that the records are already there from sql and logs this error, and remembers this until the volume is restarted or inode is flushed out of inode cache of the brick. Solution: the log levels should be reduced to trace for this case and customers need not see this. Backport of http://review.gluster.org/#/c/12491/ Change-Id: I5f11e83c306ae912a90f12220f62bb7563b8eb96 BUG: 1279059 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/12534 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* quota: add version to quota xattrsvmallika2015-11-021-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/12386/ When a quota is disable and the clean-up process terminated without completely cleaning-up the quota xattrs. Now when quota is enabled again, this can mess-up the accounting A version number is suffixed for all quota xattrs and this version number is specific to marker xaltor, i.e when quota xattrs are requested by quotad/client marker will remove the version suffix in the key before sending the response > Change-Id: I1ca2c11460645edba0f6b68db70d476d8d26e1eb > BUG: 1272411 > Signed-off-by: vmallika <vmallika@redhat.com> > Reviewed-on: http://review.gluster.org/12386 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I67b1b930b28411d76b2d476a4e5250c52aa495a0 BUG: 1277080 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12487 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* cli : 'gluster volume help' output sorted alphabeticallyMohamed Ashiq2015-11-021-2/+2
| | | | | | | | | | | | | | | | | | | | | 'gluster volume help' output is not sorted alphabetically. This makes little harder for the user to search or get to know of few gluster volume commands usage just from gluster cli. Backport of http://review.gluster.org/11663 Cherry picked from 323e71617fee5020324540776d0d4469577f0afe Change-Id: I855da2e4748a5c2ff3be319c50fa9548d676ee8a BUG: 1276208 Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com> >Reviewed-on: http://review.gluster.org/11663 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> >Reviewed-by: Anand Nekkunti <anekkunt@redhat.com> Reviewed-on: http://review.gluster.org/12452 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Nekkunti <anekkunt@redhat.com>
* logging : GF_LOG_NONE logs alwaysMohamed Ashiq2015-10-281-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | Shouldn't GF_LOG_NONE mean "Never log this"? If so, it's not being tested for and is, instead, treated as a higher priority than CRITICAL thus is always logged. Backport of http://review.gluster.org/11797 Cherry picked from 7b390983768e833a8df15b994038155c53d71bdd >Change-Id: Icad1e02a720a05ff21bd54ebf19c0032e6d5ce03 >BUG: 1246794 >Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com> >Reviewed-on: http://review.gluster.org/11797 >Reviewed-by: Raghavendra Talur <rtalur@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> Change-Id: Icad1e02a720a05ff21bd54ebf19c0032e6d5ce03 BUG: 1259167 Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com> Reviewed-on: http://review.gluster.org/12087 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* tier/ctr: CTR DB named lookup heal of cold tier during attach tierJoseph Fernandes2015-10-101-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Heal hardlink in the db for already existing data in the cold tier during attach tier. i.e during fix layout do lookup to files in the cold tier. CTR xlator on the brick/server side does db update/insert of the hardlink on a namelookup. Currently the namedlookup is done synchronous to the fixlayout that is triggered by attach tier. This is not performant, adding more time to fixlayout. The performant approach is record the hardlinks on a compressed datastore and then do the namelookup asynchronously later, giving the ctr db eventual consistency master patch : http://review.gluster.org/#/c/11828/ >>Change-Id: I4ffc337fffe7d447804786851a9183a51b5044a9 >>BUG: 1252586 >>Signed-off-by: Joseph Fernandes <josferna@redhat.com> >>Reviewed-on: http://review.gluster.org/11828 >>Tested-by: Gluster Build System <jenkins@build.gluster.com> >>Reviewed-by: Dan Lambright <dlambrig@redhat.com> >>Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I61b185a54ae4e8c1d82804b95a278bfbea870987 BUG: 1261146 Reviewed-on: http://review.gluster.org/12331 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: add watermarks and policy driverDan Lambright2015-10-101-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport fix 12039 This fix introduces infrastructure to support different policies for promotion and demotion. Currently the tier feature automatically promotes and demotes files periodically based on access. This is good for testing but too stringent for most real workloads. It makes it difficult to fully utilize a hot tier- data will be demoted before it is touched- its unlikely a 100GB hot SSD will have all its data touched in a window of time. A new parameter "mode" allows the user to pick promotion/demotion polcies. The "test mode" will be used for *.t and other general testing. This is the current mechanism. The "cache mode" introduces watermarks. The watermarks represent levels of data residing on the hot tier. "cache mode" policy: The % the hot tier is full is called P. Do not promote or demote more than D MB or F files. A random number [0-100] is called R. Rules for migration: if (P < watermark_low) don't demote, always promote. if (P >= watermark_low) && (P < watermark_hi) demote if R < P; promote if R > P. if (P > watermark_hi) always demote, don't promote. gluster volume set {vol} cluster.watermark-hi % gluster volume set {vol} cluster.watermark-low % gluster volume set {vol} cluster.tier-max-mb {D} gluster volume set {vol} cluster.tier-max-files {F} gluster volume set {vol} cluster.tier-mode {test|cache} > Change-Id: I157f19667ec95aa1d53406041c1e3b073be127c2 > BUG: 1257911 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/12039 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Conflicts: xlators/cluster/dht/src/dht-rebalance.c xlators/cluster/dht/src/tier.c Change-Id: Ibfe6b89563ceab98708325cf5d5ab0997c64816c BUG: 1270527 Reviewed-on: http://review.gluster.org/12330 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/ctr: Solution for db locks for tier migrator and ctr using sqlite ↵Joseph Fernandes2015-10-095-91/+405
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | version less than 3.7 i.e rhel 6.7 Problem: On RHEL 6.7, we have sqlite version 3.6.2 which doesnt support WAL journaling mode, as this journaling mode is only available in sqlite 3.7 and above. As a result we cannot have to progreses concurrently accessing sqlite, without running into db locks! Well WAL is also need for performace on CTR side. Solution: This solution is to use CTR db connection for doing queries when WAL mode is absent. i,e tier migrator will send sync_op ipc calls to CTR, which in turn will do the query and create/update the query file suggested by tier migrator. Pending: Well this solution will stop the db locks but the performance is still an issue for CTR. We are developing an in-Memory Transaction Log (iMeTaL) which will help boost the CTR performance by doing in memory udpates on the IO path and later flush the updates to the db in a batch/segment flush. Master patch: http://review.gluster.org/#/c/12191 >> Change-Id: Ie3149643ded159234b5cc6aa6cf93b9022c2f124 >> BUG: 1240577 >> Signed-off-by: Joseph Fernandes <josferna@redhat.com> >> Signed-off-by: Dan Lambright <dlambrig@redhat.com> >> Signed-off-by: Joseph Fernandes <josferna@redhat.com> >> Reviewed-on: http://review.gluster.org/12191 >> Tested-by: Gluster Build System <jenkins@build.gluster.com> >> Reviewed-by: Luis Pabon <lpabon@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ie8c7a7e9566244c104531b579126bb57fbc6e32b BUG: 1270123 Reviewed-on: http://review.gluster.org/12325 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/ec : Mark new entry changelog in entry self-healv3.7.5Ashish Pandey2015-10-061-0/+6
| | | | | | | | | | | | | | | | | | | | Problem : When a new entry is created dirty mark xattrs are not created this will need full heal to be performed, even when there are partial failures. Solution : Marks new entry changelog in self-heal. PS: Also fixed erasing of dirty markers when no data heal is required. BUG: 1258313 Signed-off-by: Ashish Pandey <aspandey@redhat.com> Change-Id: I156e3d3201afa77efe118e1aaace1d91c90a9613 Reviewed-on: http://review.gluster.org/12306 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* fd: Do fd_bind on successful openPranith Kumar K2015-10-052-10/+19
| | | | | | | | | | | | | | | | | | | | | | | - fd_unref should decrement fd->inode->fd_count only if it is present in the inode's fd list. - successful open/opendir should perform fd_bind. >Change-Id: I81dd04f330e2fee86369a6dc7147af44f3d49169 >BUG: 1207735 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/11044 >Reviewed-by: Anoop C S <anoopcs@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1259697 Change-Id: I73b79dd3519aa085fb84dde74b321511cbccce1a Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/12100 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd, dht: volume set for use-readdirp in dhtPranith Kumar K2015-10-041-1/+3
| | | | | | | | | | | | | | | | | | | | | >Change-Id: Icab246b1d02808864d878d949fa56f9f889b538a >BUG: 1265677 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/12221 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> >Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> >Reviewed-by: Kaushal M <kaushal@redhat.com> >(cherry picked from commit 059db0254f5670a34f1a928155c0c7d1cd03b53a) Change-Id: Ifc46ed08fc10b32f5e814aa09c155e11e8c93138 BUG: 1267822 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/12269 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* features/shard: Port log messages to new frameworkKrutika Dhananjay2015-09-291-0/+7
| | | | | | | | | | | | Backport of: http://review.gluster.org/12217 Change-Id: I129ff0d6d1cab15078fa474132c290950f5e1137 BUG: 1266822 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12236 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* posix: xattrop 'GF_XATTROP_ADD_ARRAY_WITH_DEFAULT' implementationvmallika2015-09-281-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/11702 Implementation of xattrop type: GF_XATTROP_ADD_ARRAY_WITH_DEFAULT GF_XATTROP_ADD_ARRAY64_WITH_DEFAULT These operations are similar to 'GF_XATTROP_ADD_ARRAY', except that it adds a default value if xattr is missing or its value is zero on disk. One use-case of this operation is in inode-quota. When a new directory is created, its default dir_count should be set to 1. So when a xattrop performed setting inode-xattrs, it should account initial dir_count 1 if the xattrs are not present Here is the usage of this operation value required in xdata for each key struct array { int32_t newvalue_1; int32_t newvalue_2; ... int32_t newvalue_n; int32_t default_1; int32_t default_2; ... int32_t default_n; }; or struct array { int32_t value_1; int32_t value_2; ... int32_t value_n; } data[2]; fill data[0] with new value to add fill data[1] with default value xattrop GF_XATTROP_ADD_ARRAY_WITH_DEFAULT for i from 1 to n { if (xattr (dest_i) is zero or not set in the disk) dest_i = newvalue_i + default_i else dest_i = dest_i + newvalue_i } value in xdata after xattrop is successful struct array { int32_t dest_1; int32_t dest_2; ... int32_t dest_n; }; > Change-Id: Ic6a08473e99fd98299a839d4d8416081a7534efd > BUG: 1243946 > Signed-off-by: vmallika <vmallika@redhat.com> > Reviewed-on: http://review.gluster.org/11702 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: Ie0c8285d9d582afbc808b0fd878f6c02957ff928 BUG: 1266882 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12241 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* fuse: add "resolve-gids" mount option to overcome 32-groups limitNiels de Vos2015-09-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a --resolve-gids commandline option to the glusterfs binary. This option gets set when executing "mount -t glusterfs -o resolve-gids ...". This option is most useful in combination with the "acl" mount option. POSIX ACL permission checking is done on the FUSE-client side to improve performance (in addition to the checking on the bricks). The fuse-bridge reads /proc/$PID/status by default, and this file contains maximum 32 groups. Any local (client-side) permission checking that requires more than the first 32 groups will fail. By enabling the "resolve-gids" option, the fuse-bridge will call getgrouplist() to retrieve all the groups from the user accessing the mountpoint. This is comparable to how "nfs.server-aux-gids" works. Note that when a user belongs to more than ~93 groups, the volume option server.manage-gids needs to be enabled too. Without this option, the RPC-layer will need to reduce the number of groups to make them fit in the RPC-header. Cherry picked from commit 64a5bf3749c67fcc00773a2716d0c7b61b0b4417: > Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2 > BUG: 1246275 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/11732 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Ravishankar N <ravishankar@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Change-Id: I7ede90d0e41bcf55755cced5747fa0fb1699edb2 BUG: 1246397 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/11875 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* tier/ctr: Solving DB Lock issue due to write contention from db connectionsJoseph Fernandes2015-09-117-6/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of 12031. > Problem: The DB on the brick is been accessed by CTR, for write and > tier migrator, for read and write. The write from tier migrator is reseting > the heat counters after a cycle. Since we are using sqlite, two connections > trying to write would cause a db lock contention. As a result CTR used to fail > to update the db. > Solution: Using the same db connection of CTR for reseting the heat counters. > 1) Introducted a new IPC FOP for CTR > 2) After the query do a ipc syncop to the underlying client xlator associated > to the brick. > 3) CTR in brick will catch the IPC FOP and cleat the heat counters. > Change-Id: I53306bfc08dcdba479deb4ccc154896521336150 > BUG: 1260730 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12031 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Conflicts: xlators/cluster/dht/src/tier.c Change-Id: I88aa289cdf21e216b42c3d8ccfb4e7e828b43772 BUG: 1262341 Reviewed-on: http://review.gluster.org/12161 Reviewed-by: Joseph Fernandes Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: add gluster v tier <vol>Dan Lambright2015-09-101-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a backport of 11984. > Currently the tier feature piggy backs off the rebalance command > syntax to obtain status and this is clumsy. Introduce a new > tier command that can do tier specific operations, starting > with volume status to display counters. > Old commands: > gluster volume attach-tier <vol> [replica count] {bricklist..} > gluster volume detach-tier <vol> {start|stop|commit} > New commands: > gluster volume tier <vol> attach [replica count] {bricklist} | > detach {start|stop|commit} | > status > Change-Id: Ic07b3c6260588162de7d34380f8cbd3d8a7f35d3 > BUG: 1255693 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/11984 > Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Id45bd0fa6b8606dd47863de83a694908da393229 BUG: 1261664 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12143 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
* libglusterfs:Porting log messages to new frameworkMohamed Ashiq2015-09-084-11/+34
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11896 Cherry picked from 038dfe57cf0c5944b0392332dbf5a00bb1208150 >Change-Id: I8625b7dc8941720cc7a864b8fddbcc7b4c485fcd >BUG: 1252836 >Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com> >Reviewed-on: http://review.gluster.org/11896 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> >Reviewed-by: Niels de Vos <ndevos@redhat.com> Change-Id: I8625b7dc8941720cc7a864b8fddbcc7b4c485fcd BUG: 1258769 Signed-off-by: Mohamed Ashiq <mliyazud@redhat.com> Reviewed-on: http://review.gluster.org/12076 Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* posix: xattrop 'GF_XATTROP_GET_AND_SET' implementationvmallika2015-08-271-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/11995 GF_XATTROP_GET_AND_SET stores the existing xattr value in xdata and sets the new value xattrop was reusing input xattr dict to set the results instead of creating new dict. This can be problem for server side xlators as the inout dict will have the value changed. > Change-Id: I43369082e1d0090d211381181e9f3b9075b8e771 > BUG: 1251454 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: I7e0c27fd415131e9983a10d27067f63ed3a7701e BUG: 1257441 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/12022 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* libgfapi: Gracefully exit when glfd is invalidPrashanth Pai2015-08-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | This is a backport of: http://review.gluster.org/10759 When glfs_* methods operating on glfd are invoked after calling glfs_close(), the program segfaults inside __GLFS_ENTRY_VALIDATE_FD trying to deference glfd->fd->inode which is invalid. Also, returning EBADF seemed more specific than EINVAL. BUG: 1240920 Change-Id: I13a92dca52da9a300252b69e026581b3a9e931fd Signed-off-by: Prashanth Pai <ppai@redhat.com> Reviewed-on-master: http://review.gluster.org/10759 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-on: http://review.gluster.org/11571 Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: bipin kunal <bkunal@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* libglusterfs: write error handling when filesystem have no space leftGaurav Kumar Garg2015-08-241-6/+6
| | | | | | | | | | | | | | | | | | | | | When no space left on filesystem and user want to perform any operation which result to change in /var/lib/glusterd/* files then glusterd is failing to write on temporary file. Fix is to handle error when write failed on temporary files due to no space left on file system. Change-Id: I79d595776b42580da35e1282b4a3cd7fe3d14afe BUG: 1253148 Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> Reviewed-on: http://review.gluster.org/11029 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Cedric Buissart (cherry picked from commit e7f737e62a9ea9eac730b6ba3ef8739e497e1540) Reviewed-on: http://review.gluster.org/11902 Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* protocol/server: forget the inodes which got ENOENT in lookupRaghavendra Bhat2015-08-212-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11489 If a looked up object is removed from the backend, then upon getting a revalidated lookup on that object ENOENT error is received. protocol/server xlator handles it by removing dentry upon which ENOENT is received. But the inode associated with it still remains in the inode table, and whoever does nameless lookup on the gfid of that object will be able to do it successfully despite the object being not present. For handling this issue, upon getting ENOENT on a looked up entry in revalidate lookups, protocol/server should forget the inode as well. Though removing files directly from the backend is not allowed, in case of objects corrupted due to bitrot and marked as bad by scrubber, objects are removed directly from the backend in case of replicate volumes, so that the object is healed from the good copy. For handling this, the inode of the bad object removed from the backend should be forgotten. Otherwise, the inode which knows the object it represents is bad, does not allow read/write operations happening as part of self-heal. Change-Id: I268eeaf37969458687425187be6622347a6cc1f1 BUG: 1255604 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.org/11973 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* logging: Stop using global xlator for log_buf allocationsSoumya Koduri2015-08-181-13/+0
| | | | | | | | | | | | | | | | | | | | | | | This reverts commit 765849ee00f6661c9059122ff2346b03b224745f. With http://review.gluster.org/#/c/10417/, struct mem_accnt is no longer embedded in a xlator_t object, but instead is allocated separately. Hence this workaround provided to avoid crashes in logging infrastructure is no longer needed This is a backport of the below fix - http://review.gluster.org/11811 BUG: 1254436 Change-Id: I4bbf7649bdc29424ff1bb878f79e99328efa0b3e Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/11811 Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/11946 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org>
* rpc: add owner xlator argument to rpc_clnt_newKrishnan Parthasarathi2015-08-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The @owner argument tells RPC layer the xlator that owns the connection and to which xlator THIS needs be set during network notifications like CONNECT and DISCONNECT. Code paths that originate from the head of a (volume) graph and use STACK_WIND ensure that the RPC local endpoint has the right xlator saved in the frame of the call (callback pair). This guarantees that the callback is executed in the right xlator context. The client handshake process which includes fetching of brick ports from glusterd, setting lk-version on the brick for the session, don't have the correct xlator set in their frames. The problem lies with RPC notifications. It doesn't have the provision to set THIS with the xlator that is registered with the corresponding RPC programs. e.g, RPC_CLNT_CONNECT event received by protocol/client doesn't have THIS set to its xlator. This implies, call(-callbacks) originating from this thread don't have the right xlator set too. The fix would be to save the xlator registered with the RPC connection during rpc_clnt_new. e.g, protocol/client's xlator would be saved with the RPC connection that it 'owns'. RPC notifications such as CONNECT, DISCONNECT, etc inherit THIS from the RPC connection's xlator. Change-Id: I9dea2c35378c511d800ef58f7fa2ea5552f2c409 BUG: 1253212 Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-on: http://review.gluster.org/11436 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit f7668938cd7745d024f3d2884e04cd744d0a69ab) Reviewed-on: http://review.gluster.org/11908 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* cluster/tier: fix 64 bit issue with sql query using timesDan Lambright2015-08-132-6/+6
| | | | | | | | | | | | | | | | | | This is a backport of 11885. > We overflowed when converting seconds to usecs in preperation for > sql queries. The fix uses uint64_t throughout including subexpressions. > Change-Id: I59bdb742197400dede97f54735b52030920b0d19 > BUG: 1231268 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Iaef7083dac7a162080f4bc23e29dedfe700fa342 BUG: 1252903 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/11913 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Joseph Fernandes
* libgfdb/ctr: Fixing rename or update of linkJoseph Fernandes2015-08-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11584 to 3.7.2 Fixing rename or updating of link in 1) libgfdb : handling return from updatelink 2) ctr xlator: correcting freeing of oldbase name in hardlink list in the update link >Change-Id: I1789c492b81876123cd456051a9ae495e876ad46 >BUG: 1241054 >Signed-off-by: Joseph Fernandes <josferna@redhat.com> >Reviewed-on: http://review.gluster.org/11584 >Tested-by: Joseph Fernandes >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I6da86c06e390f62fe8a44ade7b1efda5771e9ca8 BUG: 1242536 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/11649 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/ec: Minimize usage of EIO errorXavier Hernandez2015-08-081-1/+1
| | | | | | | | | | | | | | | | | | >Change-Id: I82e245615419c2006a2d1b5e94ff0908d2f5e891 >BUG: 1245276 >Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> >Reviewed-on: http://review.gluster.org/11741 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> Change-Id: Ifd3d63f88a686a2963c5ba2e62110249f84f338d BUG: 1250864 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: http://review.gluster.org/11852 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* SSL improvements: ECDH, DH, CRL, and accessible optionsEmmanuel Dreyfus2015-08-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Introduce ssl.dh-param option to specify a file containinf DH parameters. If it is provided, EDH ciphers are available. - Introduce ssl.ec-curve option to specify an elliptic curve name. If unspecified, ECDH ciphers are available using the prime256v1 curve. - Introduce ssl.crl-path option to specify the directory where the CRL hash file can be found. Setting to NULL disable CRL checking, just like the default. - Make all ssl.* options accessible through gluster volume set. - In default cipher list, exclude weak ciphers instead of listing the strong ones. - Enforce server cipher preference. - introduce RPC_SET_OPT macro to factor repetitive code in glusterd-volgen.c - Add ssl-ciphers.t test to check all the features touched by this change. Backport of I7bfd433df6bbf176f4a58e770e06bcdbe22a101a Change-Id: I2947eabe76ae0487ecad52a60befb7de473fc90c BUG: 1247153 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>@ Reviewed-on: http://review.gluster.org/11763 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* libglusterfs removing strerror in loggingMohamed Ashiq Liyazudeen2015-07-284-34/+17
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11464 Cherry picked from 9384a824041aeef0e58644cc9fe71dc7fa13eb35 >Change-Id: I8a0f40834da1151ddaef6139af3782bc076df57e >BUG: 1194640 >Signed-off-by: Mohamed Ashiq Liyazudeen <mliyazud@redhat.com> >Reviewed-on: http://review.gluster.org/11464 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Change-Id: I8a0f40834da1151ddaef6139af3782bc076df57e BUG: 1217722 Signed-off-by: Mohamed Ashiq Liyazudeen <mliyazud@redhat.com> Reviewed-on: http://review.gluster.org/11727 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* logging: Fixed incorrect buffer sizeNithya Balachandran2015-07-271-1/+1
| | | | | | | | | | | | | | An incorrect buffer size passed in to_gf_msg_backtrace prevented the callers from being logged. Change-Id: If1ddc5cd9317b55d9884f26cfddd0b0f86e7a7b2 BUG: 1241841 Signed-off-by: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/11615 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Kaushal M <kaushal@redhat.com>
* rpc: fix binding brick issue while bind-insecure is enabledPrasanna Kumar Kalever2015-07-262-11/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch is backport of http://review.gluster.org/#/c/11512/ > problem: > When bind-insecure is turned on (which is the default now), it may happen > that brick is not able to bind to port assigned by Glusterd for example > 49192-49195... > > It seems to occur because the rpc_clnt connections are binding to ports in > the same range. so brick fails to bind to a port which is already used by > someone else > > solution: > > fix for now is to make rpc_clnt to get port numbers from 65535 in a > descending > order, as a result port clash is minimized > > other fixes: > > previously rdma binds to port >= 1024 if it cannot find a free port < 1024, > even when bind insecure was turned off(ref to commit '0e3fd04e'), this patch > add's a check for bind-insecure in gf_rdma_client_bind function > > This patch also re-enable bind-insecure and allow insecure by default > which was reverted (ref: commit cef1720) previously > Change-Id: Ia1cfa93c5454e2ae0ff57813689b75de282ebd07 > BUG: 1238661 > Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Change-Id: Iea55f9b2a57b5e24d3df2c5fafae12fe99e9dee0 BUG: 1246481 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/11758 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/ec: Make background healing optional behaviorPranith Kumar K2015-07-211-3/+7
| | | | | | | | | | | | | | | | | | Provide options to control number of active background heal count and qlen. >Change-Id: Idc2419219d881f47e7d2e9bbc1dcdd999b372033 >BUG: 1237381 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> >Reviewed-on: http://review.gluster.org/11473 >Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> >Tested-by: Gluster Build System <jenkins@build.gluster.com> BUG: 1238476 Change-Id: I22ba902d9911195656db9e458c01b54cf0afcd7a Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/11680 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
* libgfdb/sql: Fixing broken query of find_unchangedJoseph Fernandes2015-07-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | The find_unchanged query should be "write_heat <= defined_heat" AND "read_heat <= defined_heat" and not "write_heat <= defined_heat" OR "read_heat <= defined_heat" >>Backport of http://review.gluster.org/#/c/11577/ >>Change-Id: Ie82e02aafbb7ea14563007307de3350ea022049a >>BUG: 1240970 >>Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ie5d06fa2e3a3552741d2e6bd76807a69808c3d65 BUG: 1241776 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/11608 Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Pamela Ousley <pousley@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* syncop: Include iatt to 'syncop_link' argsSoumya Koduri2015-07-152-3/+9
| | | | | | | | | | | | | | | | | Include iatt to 'syncop_link' args to fetch proper attributes of the newly linked inode. This is backport of the below fix - http://review.gluster.org/11611 Change-Id: If6b92961bd7a89add3791ed3a9b494087348b492 BUG: 1243408 Reviewed-on: http://review.gluster.org/11611 Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/11677 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* features/bit-rot-stub: deny access to bad objectsRaghavendra Bhat2015-07-141-0/+3
| | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/11126 * Access to bad objects (especially operations such as open, readv, writev) should be denied to prevent applications from getting wrong data. * Do not allow anyone apart from scrubber to set bad object xattr. * Do not allow bad object xattr to be removed. Change-Id: I6903184ab64a9d1ea595330b603935979c33bc26 BUG: 1241529 Reviewed-on: http://review.gluster.org/11603 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* quota/marker: use smaller stacksize in synctask for marker updationvmallika2015-07-142-10/+28
| | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/11499/ Default stacksize that synctask uses is 2M. For marker we set it to 16k Also move market xlator close to io-threads to have smaller stack > Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651 > BUG: 1207735 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: Id1cb6288a38d370956cc47aed5253ff95f04c966 BUG: 1229282 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/11595 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
generated by cgit (git 2.34.1) at 2024-05-01 12:11:27 +0000