Messages in this thread | | | From | Wander Lairson Costa <> | Date | Thu, 19 Jan 2023 16:53:44 -0300 | Subject | Re: [PATCH] sched/deadline: fix inactive_task_timer splat with CONFIG_PREEMPT_RT |
| |
On Thu, Jan 19, 2023 at 3:03 PM Valentin Schneider <vschneid@redhat.com> wrote: > > On 04/01/23 15:17, Wander Lairson Costa wrote: > > inactive_task_timer() executes in interrupt (atomic) context. It calls > > put_task_struct(), which indirectly acquires sleeping locks under > > PREEMPT_RT. > > > > Below is an example of a splat that happened in a test environment: > > > > CPU: 1 PID: 2848 Comm: life Kdump: loaded Tainted: G W --------- > > Hardware name: HP ProLiant DL388p Gen8, BIOS P70 07/15/2012 > > Call Trace: > > dump_stack_lvl+0x57/0x7d > > mark_lock_irq.cold+0x33/0xba > > ? stack_trace_save+0x4b/0x70 > > ? save_trace+0x55/0x150 > > mark_lock+0x1e7/0x400 > > mark_usage+0x11d/0x140 > > __lock_acquire+0x30d/0x930 > > lock_acquire.part.0+0x9c/0x210 > > ? refill_obj_stock+0x3d/0x3a0 > > ? rcu_read_lock_sched_held+0x3f/0x70 > > ? trace_lock_acquire+0x38/0x140 > > ? lock_acquire+0x30/0x80 > > ? refill_obj_stock+0x3d/0x3a0 > > rt_spin_lock+0x27/0xe0 > > ? refill_obj_stock+0x3d/0x3a0 > > refill_obj_stock+0x3d/0x3a0 > > ? inactive_task_timer+0x1ad/0x340 > > kmem_cache_free+0x357/0x560 > > inactive_task_timer+0x1ad/0x340 > > ? switched_from_dl+0x2d0/0x2d0 > > __run_hrtimer+0x8a/0x1a0 > > __hrtimer_run_queues+0x91/0x130 > > hrtimer_interrupt+0x10f/0x220 > > __sysvec_apic_timer_interrupt+0x7b/0xd0 > > sysvec_apic_timer_interrupt+0x4f/0xd0 > > ? asm_sysvec_apic_timer_interrupt+0xa/0x20 > > asm_sysvec_apic_timer_interrupt+0x12/0x20 > > RIP: 0033:0x7fff196bf6f5 > > > > Instead of calling put_task_struct() directly, we defer it using > > call_rcu(). A more natural approach would use a workqueue, but since > > in PREEMPT_RT, we can't allocate dynamic memory from atomic context, > > the code would become more complex because we would need to put the > > work_struct instance in the task_struct and initialize it when we > > allocate a new task_struct. > > > > Sorry to come back on this; Juri reminded me offline that put_task_struct() > is invoked in other non-sleepable contexts, not just inactive_task_timer(). >
I guess there is no splat because the usage count doesn't reach zero in those code paths.
> e.g. > > rto_push_irq_work_func() // hard irq work so hardirq context > `\ > push_rt_task() > `\ > put_task_struct() >
This is paired with a get_task_struct() a few lines above inside in the same function.
> Or > > cpu_stopper_thread() // stopper callbacks must not sleep > `\ > push_cpu_stop() > `\ > put_task_struct() >
This is paired with a get_task_struct() from get_push_task()
> ... But then again I'm not aware of any splats happening in these paths. Is > there something special about inactive_task_timer(), or could it be the > issue is there for those other paths but we just haven't had them reported > yet? >
Given that those calls have corresponding get_task_struct() calls that are close in time, there is a low probability of the usage count reaching zero and triggering the splat. In any case, I will work in a v2 that also addresses those call sites.
| |