Messages in this thread |  | | | Date | Tue, 3 Nov 2009 20:28:39 +0100 | | From | Ingo Molnar <> | | Subject | Re: [PATCH 3/3] perf latency builtin command |
Clark, John,
* Clark Williams <williams@redhat.com> wrote:
> > This is the first cut at a 'perf latency' command to manage the > hwlat_detector kernel module, used to detect hardware induced > latencies (e.g. SMIs). > > Signed-off-by: Clark Williams <williams@redhat.com> > --- > tools/perf/Documentation/perf-latency.txt | 64 +++++ > tools/perf/Documentation/perf.txt | 2 +- > tools/perf/Makefile | 3 + > tools/perf/builtin-latency.c | 383 > +++++++++++++++++++++++++++++ tools/perf/builtin.h > | 2 +- tools/perf/command-list.txt | 1 + > 6 files changed, 453 insertions(+), 2 deletions(-) > create mode 100644 tools/perf/Documentation/perf-latency.txt > create mode 100644 tools/perf/builtin-latency.c > > diff --git a/tools/perf/Documentation/perf-latency.txt > b/tools/perf/Documentation/perf-latency.txt new file mode 100644 > index 0000000..f615d08 > --- /dev/null > +++ b/tools/perf/Documentation/perf-latency.txt > @@ -0,0 +1,64 @@ > +perf-latency(1) > +=============== > + > +NAME > +---- > +perf-latency - check for hardware latencies > + > +SYNOPSIS > +-------- > +[verse] > +'per latency' [OPTIONS] > + > +DESCRIPTION > +----------- > +This command manages the hwlat_detector kernel module, which is used > +to test the system for hardware induced latencies. The command runs > +for a specified amount of time (default: 60 seconds) and samples the > +system Time Stamp Counter (TSC) register, looking for gaps which > +exceed a threshold. If a gap exceeding the threshold is seen, a > +timestamp and the gap (in microseconds) is printed to the standard > +output. > + > +OPTIONS > +------- > +--duration=<n>{s,m,h,d,w}:: > + The length of time the test should run. (default: 60 seconds) > + > +--window=<n>{us,ms,s,m}:: > + The sample period for the test. (default 1 second) > + > +--width==<n>{us,ms,s,m}:: > + The test time within the sample window. (default 500 > + milliseconds) > + > +--threshold==<n>{us,ms,s}:: > + Threshold above which is considered a latency. (default > 50 microseconds) + > +--cleanup:: > + Force unload of hwlat_detector module and umount of debugfs.
I'm wondering whether we could do something perf event based that makes 'perf latency' self-sufficient and eliminates the debugfs interface.
( We could still merge the first two patches in their current form as they are clear improvements in terms of debugfs access within perf - so no work is lost and progress is possible. )
Basically hwlat_detector is using stop_machine_run() plus a tight rdtsc based loop to sample what is happening in the system. Much of hwlat_detector.c deals with getting that information (and parameters) back and forth between user space and kernel space.
Couldnt we move that functionality a bit closer to perf by creating special events in a tight loop that generate a stream of perf events, and let the rest of perf events take over the details, and do the analysis in the user-space builtin-latency.c code?
Also, do we need stop_machine_run() - couldnt we do the measurement on a specific CPU with irqs (and NMIs) disabled [but other CPUs still running]?
This would all still be possible in the .33 timeframe i suspect, as what we need is really just a special event (via TRACE_EVENT() perhaps), and a way to trigger it via a 'run this many times' parameter. (i.e. event injection - we want to have that kind of support in perf events anyway)
This would simplify and standardize hw-latency detection, without losing any utility - and we wouldnt have to go via some special debugfs interface to access the hwlat_detect module.
Thoughts?
Ingo
|  |