Messages in this thread | | | Date | Tue, 20 Dec 2011 10:15:11 +0100 | From | Ingo Molnar <> | Subject | Re: [RFC 4/5] x86, perf: implements lwp-perf-integration (rc1) |
| |
* Avi Kivity <avi@redhat.com> wrote:
> On 12/19/2011 01:40 PM, Ingo Molnar wrote: > > > > 2) the proper solution: creating a 'user-space vmalloc()' that > > is per mm and that gets inherited transparently, across > > fork() and exec(), and which lies outside the regular vma > > spaces. On 64-bit this should be straightforward. > > That probably has uses outside perf too, but I can see mm nacks piling up.
This can be done in arch/x86/ code if it's too x86 specific - the platform controls the VM layout and can (and does) use special per CPU VM areas.
> > These vmas are not actually 'known' to user-space > > normally - the kernel PMU code knows about it and does > > what we do with PEBS: flushes it when necessary and puts > > it into the regular perf event channels. > > > > This solves the inherited perf record workflow > > immediately: the parent task just creates the buffer, > > which gets inherited across exec() and fork(), into every > > portion of the workload. > > The buffer still needs to be managed. [...]
Of course, like we manage the DS buffer for PEBS.
> [...] While you may be able to juggle different threads on > the same cpu using different events, threads on other cpus > need to use separate LWP contexts and buffers.
Yes, like different threads on different CPUs have different DS buffers, *here and today*.
Try this on (most) modern Intel CPUs:
perf top -e cycles:pp
That will activate that exact mechanism.
The LWPCB and the LWP ring-buffer are really just an extension of that concept: per task buffers which are ring 3 visible.
Note that user-space does not actually have to know about any of these LWP addresses (but can access them if it wants to - no strong feelings about that) - in the correctly implemented model it's fully kernel managed.
In fact the PEBS case had one more complication: there's the BTS branch-tracing feature which we support as well, and which overlaps PEBS use of the DS.
All these PMU hardware limitations can be supported, as long as the instrumentation *capability* adds value to the system in one way or another.
> > System-wide profiling is a small additional variant of > > this: creating such a user-vmalloc() area for all tasks > > in the system so that the PMU code has them ready in the > > context-switch code. > > What about security? Do we want to allow any userspace > process to mess up the buffers? It can even reprogram the LWP > block, so you're counting different things, or at higher > frequencies, or into other processes ordinary vmas?
In most usecases it's the application messing up its own profiling - don't do that if it hurts.
I'd argue that future LWP versions should allow kernel-protected LWP pages, as long as the LWPCB is privileged as well as well. That would be useful for another purpose as well: LWP could be allowed to sample kernel-space execution as well, an obviously useful feature that was left out from LWP for barely explicable reasons.
Granted, LWP was mis-designed to quite a degree, those AMD chip engineers should have talked to people who understand how modern PMU abstractions are added to the OS kernel properly. But this mis-design does not keep us from utilizing this piece of hardware intelligently. PEBS/DS/BTS wasnt a beauty either.
> You could rebuild the LWP block on every context switch I > guess, but you need to prevent access to other cpus' LWP > blocks (since they may be running other processes). I think > this calls for per-cpu cr3, even for threads in the same > process.
Why would we want to rebuild the LWPCB? Just keep one per task and do a lightweight switch to it during switch_to() - like we do it with the PEBS hardware-ring-buffer. It can be in the same single block of memory with the ring-buffer itself. (PEBS has similar characteristics)
Thanks,
Ingo
| |