glusterfs.git/xlators/debug/io-stats, branch round-robin2

io-stats: Fix io-stat dump to dump at all levels

2016-06-15T17:38:00+00:00

Previous commit to fix the bug, where io-stat-dump was overwriting
the dump file when the client and a brick was on the same host,
failed to consider the existing behaviour where io-stats can
help generate closely correlated set of stats across clients
and bricks, by triggering the dump using the same command.

This was introduced in commit:
0facb11220aea20a6573b656785922219c9650cf

Further, by limiting the first io-stat to unwind the dump request,
there is no way to trigger other io-stat xlators in the stack to
dump their stat information.

This bug hence is being fixed by this commit keeping the
following in mind,
- We need to trigger io-stat-dump for all instances in the
graph when this attr is set
- We need to write the output to different files, so that
they do not overwrite each others data
- We need to prevent this xattr from being set on the path
that is used to trigger the io-stat-dump information

Change-Id: I31ec380f0d85e10313a9d7b977da0e1ec74638a6
BUG: 1322825
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/14552
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

io-stats: Fix overwriting of client profile by the bricks

2016-04-12T12:10:40+00:00

Issue: When the user executes the following command to generate
the client perf profile, if the client is on the same node as
bricks, the bricks overwrite the profile info written by clients.
Also xattr "trusted.io-stats-dump" gets set on the mount point.

setxattr -n trusted.io-stats-dump -v /tmp/iostat.log /mnt/fuse

Fix: Unwind from setxattr, when xattr is 'io-stats-dump'

Change-Id: Iba0e5df2f25f4ba3b1399ac176a3f8a916ff372e
BUG: 1322825
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/13872
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Ravishankar N 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy

debug/io-stats: Disable fop stats dump by default

2016-03-24T15:05:16+00:00

Even though someone is not interested in seeing the fops statistics in
client logs under DEBUG level we forcefully dump the same. This is due
to default dump-interval configured to as 5 seconds. Since can use
diagnostics.stats-dump-interval volume set option to change the default
dump-interval its better to disable this huge dumping by default.

Change-Id: Id14a38bcd92e47d75003279567a0f80acac1d86e
BUG: 1320101
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/13808
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

all: fixes for clang compile warnings

2016-02-15T09:32:26+00:00

cli/src/cli-cmd-parser.c (chenk)
cli/src/cli-xml-output.c (spandit)
cli/src/cli.c (chenk)
libglusterfs/src/common-utils.c (vmallika)
libglusterfs/src/gfdb/gfdb_sqlite3.c (jfernand +1)
rpc/rpc-transport/socket/src/socket.c (?)
xlators/cluster/afr/src/afr-transaction.c (?)
xlators/cluster/dht/src/dht-common.h (srangana +2)
xlators/cluster/dht/src/dht-selfheal.c (srangana +2)
xlators/debug/io-stats/src/io-stats.c (R. Wareing)
xlators/features/barrier/src/barrier.c (vshastry)
xlators/features/bit-rot/src/bitd/bit-rot-scrub.h (vshankar +1)
xlators/features/shard/src/shard.c (kdhananj +1)
xlators/mgmt/glusterd/src/glusterd-ganesha.c (skoduri)
xlators/mgmt/glusterd/src/glusterd-handler.c (atinmu)
xlators/mgmt/glusterd/src/glusterd-op-sm.h (atinmu)
xlators/mgmt/glusterd/src/glusterd-snapshot.c (spandit)
xlators/mgmt/glusterd/src/glusterd-syncop.c (atinmu)
xlators/mgmt/glusterd/src/glusterd-volgen.c (atinmu)
xlators/protocol/client/src/client-messages.h (mselvaga +1)
xlators/storage/bd/src/bd-helper.c (M. Mohan Kumar)
xlators/storage/bd/src/bd.c (M. Mohan Kumar)
xlators/storage/posix/src/posix.c (nbalacha +1)

Change-Id: I85934fbcaf485932136ef3acd206f6ebecde61dd
BUG: 1293133
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13031
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System

all: fix various cppcheck warnings

2016-02-15T09:32:14+00:00

fixes for various warnings reported by cppcheck

N.B. cppcheck output is in the bugzilla

Change-Id: I33acec127bc4536935fdd8d52a0c490ec54d50b2
BUG: 1292954
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13006
Reviewed-by: Jeff Darcy 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

build: export minimum symbols from xlators for correct resolution

2015-12-22T17:15:01+00:00

Revisiting http://review.gluster.org/#/c/11814/, which unintentionally
introduced warnings from libtool about the xlator .so names.

According to [1], the -module option must appear in the Makefile.am
file(s); if -module is defined in a macro, e.g. in configure(.ac),
then libtool will not recognize that this is a module and will emit a
warning.

[1]
http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules

Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078
BUG: 1248669
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13003
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos

core: use syscall wrappers instead of direct syscalls - tail

2015-11-16T23:20:17+00:00

tail, as in dog chasing its tail. These are the unwrapped
syscalls that have crept in (or were missed) in the previous
patches.

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: If183487de92fc7cbc47d4c5aa3f3e80eae50b84f
BUG: 1267967
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/12589
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

io-stats: fix BSD build failure

2015-11-02T18:21:56+00:00

Change-Id: Ieb372cb686d32a09c6df31ec849f1b3c52e0e1cd
BUG: 1277024
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/12484
Reviewed-by: Raghavendra Talur 
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

debug/io-stats: Add FOP sampling feature

2015-11-01T17:14:34+00:00

Summary:
- Using sampling feature you can record details about every Nth FOP.
  The fields in each sample are: FOP type, hostname, uid, gid, FOP priority,
  port and time taken (latency) to fufill the request.
- Implemented using a ring buffer which is not (m/c) allocated in the IO path,
  this should make the sampling process pretty cheap.
- DNS resolution done @ dump time not @ sample time for performance w/
  cache
- Metrics can be used for both diagnostics, traffic/IO profiling as well
  as P95/P99 calculations
- To control this feature there are two new volume options:
  diagnostics.fop-sample-interval - The sampling interval, e.g. 1 means
  sample every FOP, 100 means sample every 100th FOP
  diagnostics.fop-sample-buf-size - The size (in bytes) of the ring
  buffer used to store the samples.  In the even more samples
  are collected in the stats dump interval than can be held in this buffer,
  the oldest samples shall be discarded.  Samples are stored in the log
  directory under /var/log/glusterfs/samples.
- Uses DNS cache written by sshreyas@fb.com (Thank-you!), the DNS cache
  TTL is controlled by the diagnostics.stats-dnscache-ttl-sec option
  and defaults to 24hrs.

Test Plan:
- Valgrind'd to ensure it's leak free
- Run prove test(s)
- Shadow testing on 100+ brick cluster

Change-Id: I9ee14c2fa18486b7efb38e59f70687249d3f96d8
BUG: 1271310
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/12210
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

xlators: add JSON FOP statistics dumps every N seconds

2015-10-08T12:44:27+00:00

Summary:
- Adds a thread to the io-stats translator which dumps out statistics
  every N seconds where N is configurable by an option called
  "diagnostics.stats-dump-interval"
- Thread cleanly starts/stops when translator is unloaded
- Updates macros to use "Atomic Builtins" (e.g. intel CPU extentions) to
  use memory barries to update counters vs using locks.  This should
  reduce overhead and prevent any deadlock bugs due to lock contention.

Test Plan:
- Test on development machine
- Run prove -v tests/basic/stats-dump.t

Change-Id: If071239d8fdc185e4e8fd527363cc042447a245d
BUG: 1266476
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/12209
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Avra Sengupta