Messages in this thread | | | Date | Wed, 13 Apr 2016 22:10:12 -0400 | From | Waiman Long <> | Subject | Re: [PATCH v4] x86/hpet: Reduce HPET counter read contention |
| |
On 04/13/2016 08:25 PM, Peter Zijlstra wrote: > On Wed, Apr 13, 2016 at 11:37:21AM -0400, Waiman Long wrote: >> The TSC clocksource, on the other hand, is per cpu. So there won't be much >> contention in accessing it. Normally TSC will be used the default clock >> source. However, if there is too much variation in the actual clock speeds >> of the individual CPUs, > Does the system actually have a clock rate skew? Not an offset?
No, the system that I was able talking about didn't have this issue. I did see some prototype machines that had the clock skew problem due to firmware issue.
>> it will cause the TSC calibration to fail and revert >> to use hpet as the clock source. During bootup, hpet will usually be >> selected as the default clock source first. After a short time, the TSC will >> take over as the default clock source. Problem can happen during that short >> period of transition time too. In fact, we have 16-socket Broadwell-EX >> systems that has this soft lockup problem once in a few reboot cycles which >> prompted me to find a solution to fix it. > This 16 socket system is a completely broken trainwreck. Trying to use > HPET with _that_ many CPUs is absolutely insane. > > Please tell your hardware engineers to fix the TSC clock domain.
I was talking about the way the clock source was brought up. If you look at the bootup kernel log, you will see something like (from a 4-socket system):
[ 5.777423] clocksource: Switched to clocksource hpet [ 5.823689] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 7.870387] tsc: Refined TSC clocksource calibration: 2493.990 MHz [ 7.872299] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x23f30c707d3, max_idle_ns: 440795252535 ns [ 8.871787] clocksource: Switched to clocksource tsc
The TSC calibration itself takes some time and it needs a prior clock source (hpet) as a reference. It is during that transition period between hpet and TSC as default clocksource that the 16-socket system may hit a soft lockup occasionally. I don't think it is a hardware issue. That system was using TSC as the clock source when it booted up correctly.
Cheers, Longman
| |