Messages in this thread | | | Date | Thu, 30 Sep 2021 11:07:18 +0200 | From | Peter Zijlstra <> | Subject | Re: [PATCH 4/5] irq_work: Handle some irq_work in SOFTIRQ on PREEMPT_RT |
| |
On Mon, Sep 27, 2021 at 11:19:18PM +0200, Sebastian Andrzej Siewior wrote: > The irq_work callback is invoked in hard IRQ context. By default all > callbacks are scheduled for invocation right away (given supported by > the architecture) except for the ones marked IRQ_WORK_LAZY which are > delayed until the next timer-tick. > > While looking over the callbacks, some of them may acquire locks > (spinlock_t, rwlock_t) which are transformed into sleeping locks on > PREEMPT_RT and must not be acquired in hard IRQ context. > Changing the locks into locks which could be acquired in this context > will lead to other problems such as increased latencies if everything > in the chain has IRQ-off locks. This will not solve all the issues as > one callback has been noticed which invoked kref_put() and its callback > invokes kfree() and this can not be invoked in hardirq context. > > Some callbacks are required to be invoked in hardirq context even on > PREEMPT_RT to work properly. This includes for instance the NO_HZ > callback which needs to be able to observe the idle context. > > The callbacks which require to be run in hardirq have already been > marked. Use this information to split the callbacks onto the two lists > on PREEMPT_RT: > - lazy_list > Work items which are not marked with IRQ_WORK_HARD_IRQ will be added > to this list. Callbacks on this list will be invoked from timer > softirq handler. The handler here may acquire sleeping locks such as > spinlock_t and invoke kfree(). > > - raised_list > Work items which are marked with IRQ_WORK_HARD_IRQ will be added to > this list. They will be invoked in hardirq context and must not > acquire any sleeping locks. > > [bigeasy: melt tglx's irq_work_tick_soft() which splits irq_work_tick() into a > hard and soft variant. Collected fixes over time from Steven > Rostedt and Mike Galbraith. ] > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
IIRC we have existing problems in -RT due to this irq_work softirq muck.
I think the problem was something Jolsa found a while ago, where perf defers to an irq_work (from NMI context) and that irq_work wants to deliver signals, which it can't on -RT, so the whole thing gets punted to softirq. With the end-result that if you self-profile RT tasks, things come apart or something.
There might have been others as well, I don't know. But generally I think we want *less* softirq, not more.
| |