summaryrefslogtreecommitdiffstats
path: root/xlators/debug
Commit message (Collapse)AuthorAgeFilesLines
* io-stats: re-port changes since 3.6 on top of FB versionJeff Darcy2017-09-153-55/+90
| | | | | | | | | | | | | | | | | | | Includes io-stats parts of the following patches: 1e421a5 logging: Avoid re-initing log level in io-stats 0facb11 io-stats: Fix overwriting of client profile by the bricks 91004b0 debug/io-stats: Disable fop stats dump by default 62f9659 all: fix various cppcheck warnings e62c0fe build: export minimum symbols from xlators for correct resolution 1d0a0d1 core: use syscall wrappers instead of direct syscalls - tail 0773ca6 all: reduce "inline" usage 8a9328e build: do not #include "config.h" in each file 320455b io-stats: Fixing dereference after null check. 28397ca Avoid conflict between contrib/uuid and system uuid 49d6894 io-stats : null dereference coverity fix. Change-Id: If1bdad6244e5749c6d8c456e6c64b5c5b483e273 Signed-off-by: Jeff Darcy <jdarcy@fb.com>
* Replace namespace/io-stats/io-threads with 3.6-fb versionsJeff Darcy2017-09-154-993/+1883
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This rolls up multiple patches related to namespace identificaton and throttling/QoS. This primarily includes the following, all by Michael Goulet <mgoulet@fb.com>. io-threads: Add weighted round robin queueing by namespace https://phabricator.facebook.com/D5615269 io-threads: Add per-namespaces queue sizes to IO_THREADS_QUEUE_SIZE_KEY https://phabricator.facebook.com/D5683162 io-threads: Implement better slot allocation algorithm https://phabricator.facebook.com/D5683186 io-threads: Only enable weighted queueing on bricks https://phabricator.facebook.com/D5700062 io-threads: Update queue sizes on drain https://phabricator.facebook.com/D5704832 Fix parsing (-1) as default NS weight https://phabricator.facebook.com/D5723383 Parts of the following patches have also been applied to satisfy dependencies. io-throttling: Calculate moving averages and throttle offending hosts https://phabricator.fb.com/D2516161 Shreyas Siravara <sshreyas@fb.com> Hook up ODS logging for FUSE clients. https://phabricator.facebook.com/D3963376 Kevin Vigor <kvigor@fb.com> Add the flag --skip-nfsd-start to skip the NFS daemon stating, even if it is enabled https://phabricator.facebook.com/D4575368 Alex Lorca <alexlorca@fb.com> There are also some "standard" changes: dealing with code that moved, reindenting to comply with Gluster coding standards, gf_uuid_xxx, etc. This patch *does* revert some changes which have occurred upstream since 3.6; these will be re-applied as apppropriate on top of this new base. Change-Id: I69024115da7a60811e5b86beae781d602bdb558d Signed-off-by: Jeff Darcy <jdarcy@fb.com>
* Get glusterfs to output p50, p90, and p95 latenciesSheena Artrip2017-09-131-67/+100
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: [done] separate p99 dumping into general funcs [done] add p95, p90, and p50 stats - add p95, p90, p50 within p99, and generalize - rename config to dump-percentile-lantencies Test Plan: make install glusterfs on dev machine. gluster volume create $name ... mount volume on /mnt/$name <brick1, brick2, ...> dd if=/dev/zero of=/mnt/$name/test check each brick for pn printing /var/lib/glusterd/stats/glusterfsd__$brick.dump Reviewers: sshreyas, kvigor, jdarcy Reviewed By: jdarcy Differential Revision: https://phabricator.intern.facebook.com/D5645951 Change-Id: Ic8ada48d9772bf2d5b3a2ba3c845d91d4e03c9d3 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/18279 Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* Get glusterfs to output p50, p90, and p95 latenciesSheena Artrip2017-09-131-50/+112
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: [done] separate p99 dumping into general funcs [done] add p95, p90, and p50 stats - add p95, p90, p50 within p99, and generalize - rename config to dump-percentile-lantencies Test Plan: make install glusterfs on dev machine. gluster volume create $name ... mount volume on /mnt/$name <brick1, brick2, ...> dd if=/dev/zero of=/mnt/$name/test check each brick for pn printing /var/lib/glusterd/stats/glusterfsd__$brick.dump Reviewers: sshreyas, kvigor, jdarcy Reviewed By: jdarcy Differential Revision: https://phabricator.intern.facebook.com/D5645951 Change-Id: I7bcd7201fc3753316db0ece809491a1cbdbefd32 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/18278 Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* added p99 support to the samples loggingAugustus Wynn2017-09-131-0/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | Summary: added global and by-fop-type calculation of p99 latency to the sampled fop data Test Plan: build local glusterfs mount and looked at the stats while dd if=/dev/zero of=/mnt/fuse/groot/share1/test1 bs=5 Reviewers: sshreyas, mgoulet, jdarcy Reviewed By: jdarcy Subscribers: jdarcy Differential Revision: https://phabricator.intern.facebook.com/D5597662 Change-Id: I3f5cd9c0ea59ae4357827fcbd19bbf009e661c05 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/18277 Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* inodelk-count: Add stats to count the number of lock objectskrad2017-09-131-0/+53
| | | | | | | | | | | | | | | | | | | | | | | | Summary: We want to track the number of locks held by the locks xlator. One of the ways to do it would be to track the total number of pl_lock objects in the system. This patch tracks the total number of pl_lock object and exposes the stat via io-stats JSON dump. Test Plan: WIP, haven't got a pass. Putting the diff to get a sense of this approach would yield what you guys are looking for? Reviewers: kvigor, sshreyas, jdarcy Reviewed By: jdarcy Differential Revision: https://phabricator.intern.facebook.com/D5303071 Change-Id: I946debcbff61699ec28b4d6f243042440107a224 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/18273 Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* debug/io-stats: Track outstanding requestsJunsong Li2017-09-081-2/+16
| | | | | | | | | | | | | | | Summary: Add outstanding-req field to track requests that have been sent down the stack and haven't come back. This is a port of D4908836 to 3.8 Reviewers: sshreyas Change-Id: I5870f63008d553416109c1808a434f526f5a633d Reviewed-on: https://review.gluster.org/18236 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* io-stats: Error count and rate collectionShreyas Siravara2017-09-071-0/+24
| | | | | | | | | | | | | | | | | | | | | | Summary: - This diff adds error counts and rates to the regular io-stats dump. - It outputs keys that look like this: "storage.gluster.nfsd.groot.aggr.errors.<error_name>.count": "6", "storage.gluster.nfsd.groot.inter.errors.<error_name>.per_sec": "0.00" - <error_name> is the lowercase representation of errno values (e.g., ENOENT -> enoent, etc.) - This is a port of D4691581 to 3.8 Reviewers: dph, kvigor Reviewed By: kvigor Change-Id: I96857d4283c47f9d330ae1978f113013e7c78a87 Reviewed-on: https://review.gluster.org/18230 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shreyas Siravara <sshreyas@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* debug/io-stats: Restrict io-thread queue depth stats to NFSShreyas Siravara2017-09-031-20/+21
| | | | | | | | | | | | | | | | Summary: - Fixes the unecessary log spew in other daemons - This is a port of D3646627 to 3.8 Reviewers: rwareing, kvigor Reviewed By: kvigor Change-Id: Id54ab41cdfdd2006d3af2d8774c38025c566c523 Reviewed-on: https://review.gluster.org/18199 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* auditing: Sample creation & removal of filesystem entries as well as errorsShreyas Siravara2017-09-031-2/+235
| | | | | | | | | | | | | | | Summary: - Adds the ability for gluster to log every single CREATE and UNLINK that happens on the bricks (right before invoking sys_unlink() or open(...| O_CREAT) - Makes it so that CREATEs and UNLINKs are not downsampled in io-stats - This is a port of D3268156, D3778968, D3903894 & D3301527 to 3.8 Reviewed By: kvigor Change-Id: I1bce28068c02b7d202f094094237646b4d39794b Reviewed-on: https://review.gluster.org/18198 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* debug/io-stats: Fix multi-volume support for nfsdShreyas Siravara2017-09-031-10/+17
| | | | | | | | | | | | | | | | | | Summary: - Adds iamshd (iamnfsd already there due to fop throttling) options to io-stats xlator. - Leverages these options to correctly write multi-volume NFSd stats - This is a port of D2714648 to 3.8 Test Plan: - Tested on local dev server, verified multiple files are generated for multiple vols Change-Id: Id2014a135fe52045da462eaaa91f336f45cdf167 Reviewed-on: https://review.gluster.org/18195 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* debug/io-stats: Adding stat for weighted & unweighted average latencyRichard Wareing2017-09-031-0/+30
| | | | | | | | | | | | | | | | | | | | | Summary: - Our current approach to measuring "average fop latency" is badly flawed in that it doesn't weight the FOPs correctly according to how many occurred in the time interval. This makes Statisticians very sad. This patch adds an internally computed weighted average latency which will be far more efficient to display via ODS, as well as having the benefit of not being complete nonsense. - This is a port of D3148415 & D3405772 to 3.8 Reviewers: kvigor, dph, sshreyas Reviewed By: sshreyas Change-Id: Ie3618f279b545610b7ed1a8482243fcc8dc53217 Reviewed-on: https://review.gluster.org/18192 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfproxy: Make io-stats aware of the gfproxy daemonShreyas Siravara2017-09-031-0/+17
| | | | | | | | | | | | | | | | | | Summary: - This diff enables gfproxyd to output a stats file that looks like 'glusterfs_gfproxyd_{volname}.dump' - This is a port of D3753684 to 3.8 Test Plan: Tested on devserver, verified output. Reviewers: kvigor Reviewed By: kvigor Change-Id: I8559974e9d24976fd1c8b6145fbc81be40fd4134 Reviewed-on: https://review.gluster.org/18189 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* Log AFR quorum stats in io-stats translator.Kevin Vigor2017-09-031-0/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Add AFR quorum state to io-stats translator. Sample output: { "storage.gluster.nfsd.test-replicate-0.has-quorum": "1", "storage.gluster.nfsd.test-replicate-0.quorum-threshold": "1", "storage.gluster.nfsd.test-replicate-1.has-quorum": "1", "storage.gluster.nfsd.test-replicate-1.quorum-threshold": "1" } The quorum-threshold field shows the number of bricks that can be lost while still maintaining quorum. Negative numbers indicate that quorum has been lost and show the number of bricks that must be brought online to restore quorum. Additionally, I found that the code contained both afr_have_quorum() and afr_has_quorum(), which were mostly cut-n-pasted copies of each other, but with subtle differences. Mercifully, afr_have_quorum() was totally unused, so I nuked it in passing. This is a port of D4089969 to 3.8. Test Plan: Run, observe stats output. Kill brick, observe proper change. fb-smoke. Reviewers: #posix_storage, sshreyas Reviewed By: sshreyas Subscribers: sshreyas Differential Revision: https://phabricator.intern.facebook.com/D4089969 Change-Id: Ifddb351aebfe63998846bb52be8942415ce4c1a9 Reviewed-on: https://review.gluster.org/18188 Reviewed-by: Shreyas Siravara <sshreyas@fb.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* io-stats: Expose io-thread queue depthsShreyas Siravara2017-08-301-0/+25
| | | | | | | | | | | | | | | | | | | | Summary: - This diff exposes the io-thread queue depths by sending a specialized getxattr() call down to the io-threads translator. - Port of D3086477, D3094145, D3095505 to 3.8 Test Plan: Tested on devserver, will run prove tests. Valgrind + ASAN pass as well. Reviewers: rwareing, kvigor Subscribers: dld, moox, dph Differential Revision: https://phabricator.fb.com/D3086477 Change-Id: Ia452a4fcdb9173a751c4cb48d739b25c235f6855 Reviewed-on: https://review.gluster.org/18143 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shreyas Siravara <sshreyas@fb.com>
* Add negative caching to nfs auth cacheShreyas Siravara2017-08-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This diff adds the ability to the nfs daemon to cache hosts it has deauthorized for mounts, not just the hosts it has authorized. This allows a host that has been denied to be deauthorized for ttl # of seconds, or until the nfs daemon has restarted. Test Plan: Use the prove tests to maintain the integrity of the auth code. Test manually to see if the correct code path is being hit. Reviewers: dph, rwareing Reviewed By: rwareing Differential Revision: https://phabricator.fb.com/D1947728 Change-Id: I9728e15913e0900ab34311b13b30eba0b91ce33f Reviewed-on: https://review.gluster.org/18134 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* Merge remote-tracking branch 'origin/release-3.8' into merge-3.8Kevin Vigor2017-02-161-2/+16
|\
| * debug/trace: Print entries' iatts in readdirp cbkKrutika Dhananjay2017-02-061-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | Backport of: http://review.gluster.org/16432 Change-Id: Ie96b04e7961d353f2a193f368ac5ae94a3f64d77 BUG: 1417802 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16482 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* | Merge remote-tracking branch 'origin/release-3.8' into merge-3.8-againKevin Vigor2017-01-051-1/+2
|\| | | | | | | | | Change-Id: I844adf2aef161a44d446f8cd9b7ebcb224ee618a Signed-off-by: Kevin Vigor <kvigor@fb.com>
| * logging: Avoid re-initing log level in io-statsVijay Bellur2017-01-021-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If log level is already set via api or command line, initialization of io-stats xlator overwrites the log level to GF_LOG_INFO. This patch prevents re-initialization of log level if already set. Backport of: > Change-Id: I1f74d94ef8068b95ec696638c0a8b17d8d71aabe > BUG: 1368882 > Signed-off-by: Vijay Bellur <vbellur@redhat.com> > Reported-by: Colin Lord <clord@redhat.com> > Reviewed-on: http://review.gluster.org/15112 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Change-Id: I1f74d94ef8068b95ec696638c0a8b17d8d71aabe BUG: 1378384 Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com> Reviewed-on: http://review.gluster.org/15544 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* | debug/io-stats: Add errors to FOP samplesShreyas Siravara2016-12-181-50/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: - Captures the error of an operation in the FOP sample. - Cherry-pick of D3306106 to io-stats Signed-off-by: Shreyas Siravara <sshreyas@fb.com> Change-Id: Ia32a5b34bbd36981ac693a8829c70fa74b02d38d Reviewed-on: http://review.gluster.org/16175 Tested-by: Shreyas Siravara <sshreyas@fb.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kevin Vigor <kvigor@fb.com>
* | debug/io-stats: [FB-Only] Update JSON key name to 'storage.gluster'Shreyas Siravara2016-12-171-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: - Our keys are prefixed by storage.gluster, not gluster. - This is a cherry-pick of D1184318. Signed-off-by: Shreyas Siravara <sshreyas@fb.com> Change-Id: Iaf3d3b4ccd285853c7750759db20404607941c0e Reviewed-on: http://review.gluster.org/16151 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kevin Vigor <kvigor@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* | debug/io-stats: Track path of operations in FOP samplesShreyas Siravara2016-12-161-124/+447
|/ | | | | | | | | | | | | | | | | | | | | Summary: - This diff adds the ability to track paths in our FOP samples. - It adds a function called `attach_iosstat_to_inode()` which attaches a special struct containing the filename, to the inode. - This diff attaches a struct, `ios_local` to each frame, and tracks paths, locs, fds, and inodes depending on the fop that it is executing. - Operations done on this inode can then reference this path. Signed-off-by: Shreyas Siravara <sshreyas@fb.com> Change-Id: Ie43b2193f66d8c7f59b5d07293e07d6120e3b20a Reviewed-on: http://review.gluster.org/16149 Tested-by: Shreyas Siravara <sshreyas@fb.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kevin Vigor <kvigor@fb.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* build: correctly format some (s)size_t messagesNiels de Vos2016-09-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On 32-bit builds the are are warnings like these: posix.c:6438: warning: format '%ld' expects type 'long int', but argument 11 has type 'ssize_t' Instead of using "%l" for (signed) size_t variables, "%z" should be used. Cherry picked from commit 3af889f02722f4636d2ea30570de6477e8b5a3a9: > BUG: 1198849 > Change-Id: I6f57b5e8ea174dd9e3056aff5da685e497894ccf > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: http://review.gluster.org/14933 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> This patch is not really recommended for backporting, but we do have a new smoke test that fails when these warnings pop-up. It is cleaner to correct the code in the release-3.7 branch then to modify the smoke test to skip this branch. Change-Id: I6f57b5e8ea174dd9e3056aff5da685e497894ccf BUG: 1373530 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15410 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* io-stats: Fix overwriting of client profile by the bricksPoornima G2016-04-121-24/+43
| | | | | | | | | | | | | | | | | | | | | Issue: When the user executes the following command to generate the client perf profile, if the client is on the same node as bricks, the bricks overwrite the profile info written by clients. Also xattr "trusted.io-stats-dump" gets set on the mount point. setxattr -n trusted.io-stats-dump -v /tmp/iostat.log /mnt/fuse Fix: Unwind from setxattr, when xattr is 'io-stats-dump' Change-Id: Iba0e5df2f25f4ba3b1399ac176a3f8a916ff372e BUG: 1322825 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/13872 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* debug/trace: Print {acm}times as integersKrutika Dhananjay2016-03-291-13/+13
| | | | | | | | | | | | | This helps in debugging issues with mtime, ctime etc. Change-Id: I4a0335f17c6b40e6c627658e91f43989c3efca22 BUG: 1302948 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/13826 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* debug/io-stats: Disable fop stats dump by defaultAnoop C S2016-03-241-1/+1
| | | | | | | | | | | | | | | | | Even though someone is not interested in seeing the fops statistics in client logs under DEBUG level we forcefully dump the same. This is due to default dump-interval configured to as 5 seconds. Since can use diagnostics.stats-dump-interval volume set option to change the default dump-interval its better to disable this huge dumping by default. Change-Id: Id14a38bcd92e47d75003279567a0f80acac1d86e BUG: 1320101 Signed-off-by: Anoop C S <anoopcs@redhat.com> Reviewed-on: http://review.gluster.org/13808 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* all: fixes for clang compile warningsKaleb S KEITHLEY2016-02-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | cli/src/cli-cmd-parser.c (chenk) cli/src/cli-xml-output.c (spandit) cli/src/cli.c (chenk) libglusterfs/src/common-utils.c (vmallika) libglusterfs/src/gfdb/gfdb_sqlite3.c (jfernand +1) rpc/rpc-transport/socket/src/socket.c (?) xlators/cluster/afr/src/afr-transaction.c (?) xlators/cluster/dht/src/dht-common.h (srangana +2) xlators/cluster/dht/src/dht-selfheal.c (srangana +2) xlators/debug/io-stats/src/io-stats.c (R. Wareing) xlators/features/barrier/src/barrier.c (vshastry) xlators/features/bit-rot/src/bitd/bit-rot-scrub.h (vshankar +1) xlators/features/shard/src/shard.c (kdhananj +1) xlators/mgmt/glusterd/src/glusterd-ganesha.c (skoduri) xlators/mgmt/glusterd/src/glusterd-handler.c (atinmu) xlators/mgmt/glusterd/src/glusterd-op-sm.h (atinmu) xlators/mgmt/glusterd/src/glusterd-snapshot.c (spandit) xlators/mgmt/glusterd/src/glusterd-syncop.c (atinmu) xlators/mgmt/glusterd/src/glusterd-volgen.c (atinmu) xlators/protocol/client/src/client-messages.h (mselvaga +1) xlators/storage/bd/src/bd-helper.c (M. Mohan Kumar) xlators/storage/bd/src/bd.c (M. Mohan Kumar) xlators/storage/posix/src/posix.c (nbalacha +1) Change-Id: I85934fbcaf485932136ef3acd206f6ebecde61dd BUG: 1293133 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13031 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* all: fix various cppcheck warningsKaleb S KEITHLEY2016-02-151-5/+8
| | | | | | | | | | | | | | | fixes for various warnings reported by cppcheck N.B. cppcheck output is in the bugzilla Change-Id: I33acec127bc4536935fdd8d52a0c490ec54d50b2 BUG: 1292954 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13006 Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* build: export minimum symbols from xlators for correct resolutionKaleb S KEITHLEY2015-12-223-3/+3
| | | | | | | | | | | | | | | | | | | | | | Revisiting http://review.gluster.org/#/c/11814/, which unintentionally introduced warnings from libtool about the xlator .so names. According to [1], the -module option must appear in the Makefile.am file(s); if -module is defined in a macro, e.g. in configure(.ac), then libtool will not recognize that this is a module and will emit a warning. [1] http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078 BUG: 1248669 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/13003 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* core: use syscall wrappers instead of direct syscalls - tailKaleb S KEITHLEY2015-11-161-2/+3
| | | | | | | | | | | | | | | | | | | | tail, as in dog chasing its tail. These are the unwrapped syscalls that have crept in (or were missed) in the previous patches. various xlators and other components are invoking system calls directly instead of using the libglusterfs/syscall.[ch] wrappers. If not using the system call wrappers there should be a comment in the source explaining why the wrapper isn't used. Change-Id: If183487de92fc7cbc47d4c5aa3f3e80eae50b84f BUG: 1267967 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12589 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* io-stats: fix BSD build failureAtin Mukherjee2015-11-021-1/+1
| | | | | | | | | | | Change-Id: Ieb372cb686d32a09c6df31ec849f1b3c52e0e1cd BUG: 1277024 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-on: http://review.gluster.org/12484 Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* debug/io-stats: Add FOP sampling featureRichard Wareing2015-11-012-28/+468
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: - Using sampling feature you can record details about every Nth FOP. The fields in each sample are: FOP type, hostname, uid, gid, FOP priority, port and time taken (latency) to fufill the request. - Implemented using a ring buffer which is not (m/c) allocated in the IO path, this should make the sampling process pretty cheap. - DNS resolution done @ dump time not @ sample time for performance w/ cache - Metrics can be used for both diagnostics, traffic/IO profiling as well as P95/P99 calculations - To control this feature there are two new volume options: diagnostics.fop-sample-interval - The sampling interval, e.g. 1 means sample every FOP, 100 means sample every 100th FOP diagnostics.fop-sample-buf-size - The size (in bytes) of the ring buffer used to store the samples. In the even more samples are collected in the stats dump interval than can be held in this buffer, the oldest samples shall be discarded. Samples are stored in the log directory under /var/log/glusterfs/samples. - Uses DNS cache written by sshreyas@fb.com (Thank-you!), the DNS cache TTL is controlled by the diagnostics.stats-dnscache-ttl-sec option and defaults to 24hrs. Test Plan: - Valgrind'd to ensure it's leak free - Run prove test(s) - Shadow testing on 100+ brick cluster Change-Id: I9ee14c2fa18486b7efb38e59f70687249d3f96d8 BUG: 1271310 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/12210 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* xlators: add JSON FOP statistics dumps every N secondsRichard Wareing2015-10-083-105/+478
| | | | | | | | | | | | | | | | | | | | | | | Summary: - Adds a thread to the io-stats translator which dumps out statistics every N seconds where N is configurable by an option called "diagnostics.stats-dump-interval" - Thread cleanly starts/stops when translator is unloaded - Updates macros to use "Atomic Builtins" (e.g. intel CPU extentions) to use memory barries to update counters vs using locks. This should reduce overhead and prevent any deadlock bugs due to lock contention. Test Plan: - Test on development machine - Run prove -v tests/basic/stats-dump.t Change-Id: If071239d8fdc185e4e8fd527363cc042447a245d BUG: 1266476 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/12209 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Avra Sengupta <asengupt@redhat.com>
* build: export minimum symbols from xlators for correct resolutionKaleb S. KEITHLEY2015-09-243-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've been lucky that we haven't had any symbol collisions until now. Now we have a collision between the snapview-client's svc_lookup() and libntirpc's svc_lookup() with nfs-ganesha's FSAL_GLUSTER and libgfapi. As a short term solution all the snapview-client's FOP methods were changed to static scope. See http://review.gluster.org/11805. This works in snapview-client because all the FOP methods are defined in a single source file. This solution doesn't work for other xlators with FOP methods defined in multiple source files. To address this we link with libtool's '-export-symbols $symbol-file' (a wrapper around `ld --version-script ...` --- on linux anyway) and only export the minimum required symbols from the xlator sharedlib. N.B. the libtool man page says that the symbol file should be named foo.sym, thus the rename of *.exports to *.sym. While foo.exports worked, we will follow the documentation. Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> BUG: 1248669 Change-Id: I1de68b3e3be58ae690d8bfb2168bfc019983627c Reviewed-on: http://review.gluster.org/11814 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* all: reduce "inline" usageJeff Darcy2015-09-012-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | There are three kinds of inline functions: plain inline, extern inline, and static inline. All three have been removed from .c files, except those in "contrib" which aren't our problem. Inlines in .h files, which are overwhelmingly "static inline" already, have generally been left alone. Over time we should be able to "lower" these into .c files, but that has to be done in a case-by-case fashion requiring more manual effort. This part was easy to do automatically without (as far as I can tell) any ill effect. In the process, several pieces of dead code were flagged by the compiler, and were removed. Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155 BUG: 1245331 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/11769 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-by: Venky Shankar <vshankar@redhat.com>
* debug/trace: Print total write size in trace_writevKrutika Dhananjay2015-08-311-2/+7
| | | | | | | | | | | | | | This is some information I find useful while debugging issues in shard translator. Change-Id: Ic35c34c2f52d27b3617fc722dabe558de2056b67 BUG: 1258311 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/12053 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* build: do not #include "config.h" in each fileNiels de Vos2015-05-294-19/+0
| | | | | | | | | | | | | | | | | | Instead of including config.h in each file, and have the additional config.h included from the compiler commandline (-include option). When a .c file tests for a certain #define, and config.h was not included, incorrect assumtions were made. With this change, it can not happen again. BUG: 1222319 Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/10808 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* io-stats: Fixing dereference after null check.arao2015-04-101-1/+2
| | | | | | | | | | | | | CID: 1124492 If 'this' pointer fails to exist then the component is hard-coded in the gf_log if not it is dereferenced. Change-Id: I988137d4f5ac4c9aedef7cef0c75b167a8a5c59f BUG: 789278 Signed-off-by: arao <arao@redhat.com> Reviewed-on: http://review.gluster.org/9639 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* Avoid conflict between contrib/uuid and system uuidEmmanuel Dreyfus2015-04-041-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | glusterfs relies on Linux uuid implementation, which API is incompatible with most other systems's uuid. As a result, libglusterfs has to embed contrib/uuid, which is the Linux implementation, on non Linux systems. This implementation is incompatible with systtem's built in, but the symbols have the same names. Usually this is not a problem because when we link with -lglusterfs, libc's symbols are trumped. However there is a problem when a program not linked with -lglusterfs will dlopen() glusterfs component. In such a case, libc's uuid implementation is already loaded in the calling program, and it will be used instead of libglusterfs's implementation, causing crashes. A possible workaround is to use pre-load libglusterfs in the calling program (using LD_PRELOAD on NetBSD for instance), but such a mechanism is not portable, nor is it flexible. A much better approach is to rename libglusterfs's uuid_* functions to gf_uuid_* to avoid any possible conflict. This is what this change attempts. BUG: 1206587 Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/10017 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* io-stats : null dereference coverity fix.Manikandan Selvaganesh2015-03-301-5/+7
| | | | | | | | | | | | | CID :1124502 Change-Id: I2911be340e8e48a52e951d0f04f6a96f3c219fab BUG: 789278 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/9659 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* core: use gf_time_fmt() instead of localtime()+strftime()Kaleb S. KEITHLEY2014-11-201-37/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | gf_time_fmt() has existed since 3.3; it provides consistent timestamps (i.e. UTC times) throughout the implementation. (BTW, the other name for UTC is GMT.) N.B. many (all?) commercial storage solutions use UTC time for logging. This makes for easier debugging across geographically distributed systems. Also adding a "%s" fmt for portably printing time as simple numeric value on systems regardless of whether 32-bit or 64-bit time_t. Plus a minor tweak to return a ptr to the dest-string to allow gf_time_fmt() to be passed as a param in a *printf(). Someday we should pick the "one true" timestamp format and revise all calls to gf_time_fmt() to use it instead of the five or six different formats. Change-Id: I78202ae14b7246fa424efeea56bf2463e14abfb0 BUG: 1109917 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/8085 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* xlator/io-stat: Check and copy loc->pathShyam2014-11-051-3/+9
| | | | | | | | | | | | | | | | | | Cases where loc->path is NULL, the current code in create/open/mkdir would copy the same blindly and as a result coredump. This is a preventive fix for the coredump. The reason for loc->path to be NULL in certain cases is yet to be determined. One such case is when resolve_loc_touchup fails to get inode_path due to loops in the inode table. Change-Id: Ic2ddf2cc9f2acaf9b939afc11afd193b4402ee7c BUG: 1159221 Signed-off-by: Shyam <srangana@redhat.com> Reviewed-on: http://review.gluster.org/9029 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* io-stats: Adding private data dumping methodVipul Nayyar2014-09-031-3/+74
| | | | | | | | | | | | | | | | | For the glusterfsiostat tool to be able to gather stats about mounted volumes from meta xlator, private information in the io-stats xlator needs to be dumped in the .meta folder. Added functionality for total data being read/written to be dumped along with latency related information for all fop functions present in io-stats. Change-Id: I75486f0ca361844a643861789f6c1406f439674c BUG: 1130023 Signed-off-by: Vipul Nayyar <nayyar_vipul@yahoo.com> Reviewed-on: http://review.gluster.org/8244 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* build: make GLUSTERD_WORKDIR rely on localstatedirHarshavardhana2014-08-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | - Break-way from '/var/lib/glusterd' hard-coded previously, instead rely on 'configure' value from 'localstatedir' - Provide 's/lib/db' as default working directory for gluster management daemon for BSD and Darwin based installations - loff_t is really off_t on Darwin - fix-off the warnings generated by clang on FreeBSD/Darwin - Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all platforms. - Define proper environment for running tests, define correct PATH and LD_LIBRARY_PATH when running tests, so that the desired version of glusterfs is used, regardless where it is installed. (Thanks to manu@netbsd.org for this additional work) Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7 BUG: 1111774 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/8246 Tested-by: Gluster Build System <jenkins@build.gluster.com>
* debug/io-stats: Free conf on init failuresPranith Kumar K2014-06-301-15/+23
| | | | | | | | | | Change-Id: I1d227f7b2b8f8ad8d44df8711654ee885e79cf38 BUG: 1111557 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/8148 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* libglusterfs: Don't allow '-0' as input value for numbersPranith Kumar K2014-06-221-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: io-stats tries to init log-buf-size as uint32. All gf_string2u*** functions which get the unsigned values from string don't want the string to contain '-'. So the GF_OPTION_INIT with '-0' as value fails init in io-stats, but by that time 'ret' is already reset to 0. Io-stats ends up returning 0 even when init failed. Because of this caller of init thinks initialization is successful when it is not. iostat_xlator->private is still NULL. Because of this when a fop tries to access members of io-stat-private structure, it crashes. Fix: I initially thought may be we should fix all gf_string2u*** functions to accept '-0'. But all these functions are used only for setting volume options. If we accept '-0', gluster volume info shows output as follows: diagnostics.brick-log-buf-size: -0 This seemed ugly, so I felt it is better to disallow '-0' as valid input for numbers. Also fixed return value in cases of failures in io-stats. Change-Id: I67ac92853b6d2be70516ad1d07505ffd9f058aa4 BUG: 1111557 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/8129 Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Harshavardhana <harsha@harshavardhana.net> Tested-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* logging: Introduce suppression of repetitive log messagesKrutika Dhananjay2014-04-301-0/+65
| | | | | | | | | Change-Id: I8efa08cc9832ad509fba65a88bb0cddbaf056404 BUG: 1075611 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: http://review.gluster.org/7475 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* debug/trace: Fix length issue found in compilationPranith Kumar K2014-04-291-44/+45
| | | | | | | | | Change-Id: I1cd7fd6464d0912294009c2293ed70f3f6744930 BUG: 1092196 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/7586 Reviewed-by: Anand Avati <avati@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* build: MacOSX Porting fixesHarshavardhana2014-04-243-38/+39
| | | | | | | | | | | | | | | | | | | | | git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs Working functionality on MacOSX - GlusterD (management daemon) - GlusterCLI (management cli) - GlusterFS FUSE (using OSXFUSE) - GlusterNFS (without NLM - issues with rpc.statd) Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac BUG: 1089172 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Signed-off-by: Dennis Schafroth <dennis@schafroth.com> Tested-by: Harshavardhana <harsha@harshavardhana.net> Tested-by: Dennis Schafroth <dennis@schafroth.com> Reviewed-on: http://review.gluster.org/7503 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
generated by cgit (git 2.34.1) at 2026-02-02 03:06:18 +0000