lkml.org 
[lkml]   [2018]   [Mar]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH RT] Defer migrate_enable migration while task state != TASK_RUNNING
Hi Julia,
Thanks for the quick response!

On Fri, Mar 23, 2018 at 11:59:21AM -0500, Julia Cartwright wrote:
> Hey Joe-
>
> Thanks for the writeup.
>
> On Fri, Mar 23, 2018 at 11:09:59AM -0400, joe.korty@concurrent-rt.com wrote:
> > I see the below kernel splat in 4.9-rt when I run a test program that
> > continually changes the affinity of some set of running pids:
> >
> > do not call blocking ops when !TASK_RUNNING; state=2 set at ...
> > ...
> > stop_one_cpu+0x60/0x80
> > migrate_enable+0x21f/0x3e0
> > rt_spin_unlock+0x2f/0x40
> > prepare_to_wait+0x5c/0x80
> > ...
>
> This is clearly a problem.
>
> > The reason is that spin_unlock, write_unlock, and read_unlock call
> > migrate_enable, and since 4.4-rt, migrate_enable will sleep if it discovers
> > that a migration is in order. But sleeping in the unlock services is not
> > expected by most kernel developers,
>
> I don't buy this, see below:
>
> > and where that counts most is in code sequences like the following:
> >
> > set_current_state(TASK_UNINTERRUPIBLE);
> > spin_unlock(&s);
> > schedule();
>
> The analog in mainline is CONFIG_PREEMPT and the implicit
> preempt_enable() in spin_unlock(). In this configuration, a kernel
> developer should _absolutely_ expect their task to be suspended (and
> potentially migrated), _regardless of the task state_ if there is a
> preemption event on the CPU on which this task is executing.
>
> Similarly, on RT, there is nothing _conceptually_ wrong on RT with
> migrating on migrate_enable(), regardless of task state, if there is a
> pending migration event.

My understanding is, in standard Linux and in rt, setting
task state to anything other than TASK_RUNNING in of itself
blocks preemption. A preemption is not really needed here
as it is expected that there is a schedule() written in that
will shortly be executed. And if a 'involuntary schedule'
(ie, preemption) were allowed to occur between the task
state set and the schedule(), that would change the task
state back to TASK_RUNNING, which would cause the schedule
to NOP. Thus we risk not having paused long enough here
for the condition we were waiting for to become true.

>
> It's clear, however, that the mechanism used here is broken ...
>
> Julia

Thanks,
Joe

\
 
 \ /
  Last update: 2018-03-23 18:21    [W:0.067 / U:2.964 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site