Messages in this thread | | | Date | Thu, 11 Apr 2019 08:42:39 -0700 | From | "Paul E. McKenney" <> | Subject | Re: [PATCH 0/4] Allow CPU0 to be nohz full |
| |
On Tue, Apr 09, 2019 at 07:21:54PM +1000, Nicholas Piggin wrote: > Thomas Gleixner's on April 6, 2019 3:54 am: > > On Fri, 5 Apr 2019, Nicholas Piggin wrote: > >> Thomas Gleixner's on April 5, 2019 12:36 am: > >> > On Thu, 4 Apr 2019, Nicholas Piggin wrote: > >> > > >> >> I've been looking at ways to fix suspend breakage with CPU0 as a > >> >> nohz CPU. I started looking at various things like allowing CPU0 > >> >> to take over do_timer again temporarily or allowing nohz full > >> >> to be stopped at runtime (that is quite a significant change for > >> >> little real benefit). The problem then was having the housekeeping > >> >> CPU go offline. > >> >> > >> >> So I decided to try just allowing the freeze to occur on non-zero > >> >> CPU. This seems to be a lot simpler to get working, but I guess > >> >> some archs won't be able to deal with this? Would it be okay to > >> >> make it opt-in per arch? > >> > > >> > It needs to be opt in. x86 will fall on its nose with that. > >> > >> Okay I can add that. > >> > >> > Now the real interesting question is WHY do we need that at all? > >> > >> Why full nohz for CPU0? Basically this is how their job system was > >> written and used, testing nohz full was a change that came much later > >> as an optimisation. > >> > >> I don't think there is a fundamental reason an equivalent system > >> could not be made that uses a different CPU for housekeeping, but I > >> was assured the change would be quite difficult for them. > >> > >> If we can support it, it seems nice if you can take a particular > >> configuration and just apply nohz_full to your application processors > >> without any other changes. > > > > This wants an explanation in the patches. > > Okay. > > > And patch 4 has in the changelog: > > > > nohz_full has been successful at significantly reducing jitter for a > > large supercomputer customer, but their job control system requires CPU0 > > to be for housekeeping. > > > > which just makes me dazed and confused :) > > > > Other than some coherent explanation and making it opt in, I don't think > > there is a fundamental issue with that. > > I will try to make the changelogs less jibberish then :)
Maybe this is all taken care of now, but do the various clocks stay synchronized with wall-clock time if all CPUs are in nohz_full mode? At one time, at least one CPU needed to keep its scheduler-clock interrupt going in order to keep things in sync.
The ppc timebase register might make it possible to do this without any scheduler-clock interrupts, but figured I should check. ;-)
Thanx, Paul
| |