Messages in this thread | | | Date | Thu, 30 Jan 2014 00:06:57 +0400 | From | Cyrill Gorcunov <> | Subject | Re: [PATCH] x86, perf, p4: Counter corruption when using lots of perf groups |
| |
On Wed, Jan 29, 2014 at 02:37:50PM -0500, Don Zickus wrote: > On a P4 box stressing perf with > > ./perf record -o perf.data ./perf stat -v ./perf bench all > > it was noticed that a slew of unknown NMIs would pop out rather quickly. > > Painfully debugging this ancient platform, led me to notice cross cpu counter > corruption. > > The P4 machine is special in that it has 18 counters, half are used for cpu0 > and the other half is for cpu1 (or all 18 if hyperthreading is disabled). But > the splitting of the counters has to be actively managed by the software. > > In this particular bug, one of the cpu0 specific counters was being used by > cpu1 and caused all sorts of random unknown nmis. > > I am not entirely sure on the corruption path, but what happens is: > > o perf schedules a group with p4_pmu_schedule_events() > o inside p4_pmu_schedule_events(), it notices an hwc pointer is being reused > but for a different cpu, so it 'swaps' the config bits and returns the > updated 'assign' array with a _new_ index. > o perf schedules another group with p4_pmu_schedule_events() > o inside p4_pmu_schedule_events(), it notices an hwc pointer is being reused > (the same one as above) but for the _same_ cpu [BUG!!], so it updates the > 'assign' array to use the _old_ (wrong cpu) index because the _new_ index is in > an earlier part of the 'assign' array (and hasn't been committed yet). > o perf commits the transaction using the wrong index and corrupts the other cpu
Thanks for the fix Don! I fear I won't be able to look precisely tonight, so could it wait until tomorrow? (If it's critical sure such fix should do the trick).
| |