Messages in this thread | | | Date | Mon, 12 Jun 2023 23:20:10 +0200 | From | Frederic Weisbecker <> | Subject | Re: [PATCH v7 19/21] timer: Implement the hierarchical pull model |
| |
Le Mon, Jun 12, 2023 at 05:57:00PM +0200, Anna-Maria Behnsen a écrit : > Ok. This problem is around only for nohz_full and not for nohz idle?
Only nohz_full right.
> > > The crude trick used by nohz_full in order to re-evaluate the dynticks when > > a timer is queued from the timer softirq is to launch a self IPI (from > > trigger_dyntick_cpu()). Here you would need something like that but > > that's not something we wish either. > > > > In fact we don't want any nohz_full CPU to perform an idle migrator job. > > And the problem here is that whenever a timer interrupt occurs while > > tmc->idle == 1 (and that _might_ happen in nohz_full), it will go up the > > hierarchy as long as there is no active migrator on a given level and > > check for remote expiry. And if something expired it will not only perform > > the remote callbacks handling but also reprogram the next tick to fire in > > the next expiry. That's a potential disturbance on a nohz_full CPU. > > > > There is always an active CPU in nohz_full so there is always an active > > migrator at least at the top level. Therefore you can spare concurrent > > idle migrators if they are nohz_full. > > > > As long as the top level group is not/never idle, the wakeup value will be > KTIME_MAX and so it is no problem for nohz_full cpus - or? The check for > handling remote expiry is still a problem but please read my proposal for > this below.
Good point, I overlooked the fact that data->nextevt is only set if the top level has no migrator.
So the only issue is that a nohz_full CPU may accidentally check/do the remote expiry as a loose idle migrator. Which can add noise, etc...
> > > My suggestion is to make tmigr_requires_handle_remote() return 0 if > > tick_nohz_full_cpu(smp_processor_id()), so that we don't even bother > > raising the softirq. But if the timer softirq happens nevertheless, due > > to some local timer to process, also make tmigr_handle_remote() to > > return early. > > Regarding this problem and also the two things you mentioned in the two > earlier review remarks (timer IRQ which fires too early, IPI when CPU goes > offline), I would propose to use the tmc->wakeup value slightly different > as it is used right now: > > - Whenever a wakeup value is required, because top level group is > completely idle, the value is set in per CPU tmc struct. It could be > then reevaluated in idle code in irq exit path.
So you want to force reevaluation of tmc->wakeup unconditionally on deactivate time through tmigr_new_timer(), right? That would indeed work in any case. At the cost of some more overhead in the idle interrupt path, but perhaps hardly measurable...
The alternative would be to reset tmc->wakeup only when that deadline is reached in tmigr_requires_handle_remote(). And then have a special case in the offlining case. That's less pretty of course.
> > - For checking whether remote expiry is required, the wakeup value could > also be used.
Right.
> > - For nohz_full CPUs wakeup will be always KTIME_MAX - under the > assumption that there is alwasy an active CPU in top level group.
Sounds good!
Thanks.
| |