Messages in this thread | | | Date | Mon, 28 May 2018 08:59:32 -0700 | From | Paul Burton <> | Subject | Re: [PATCH 2/2] sched: Warn if we fail to migrate a task |
| |
Hi Peter,
On Mon, May 28, 2018 at 05:06:56PM +0200, Peter Zijlstra wrote: > On Sat, May 26, 2018 at 08:46:48AM -0700, Paul Burton wrote: > > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > > index 2380bc228dd0..cda3affd45b7 100644 > > --- a/kernel/sched/core.c > > +++ b/kernel/sched/core.c > > @@ -1127,7 +1127,8 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, > > struct migration_arg arg = { p, dest_cpu }; > > /* Need help from migration thread: drop lock and wait. */ > > task_rq_unlock(rq, p, &rf); > > - stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); > > + ret = stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); > > + WARN_ON(ret); > > tlb_migrate_finish(p->mm); > > return 0; > > } else if (task_on_rq_queued(p)) { > > I think we can trigger this at will.. Set affinity to the CPU you're > going to take offline and offline concurrently. > > It is possible for the offline to happen between task_rq_unlock() and > stop_one_cpu(), at which point the WARM will then trigger.
Right, good point. The problem is only if stop_one_cpu() returns an error whilst cpu_of(rq) is being brought online, not whilst it's being offlined.
> The point is; and maybe this should be a comment somewhere; that if this > fails, there is nothing we can do about it, and it should be fixed up by > migrate_tasks()/select_task_rq(). > > There is no point in propagating the error to userspace, since if we'd > have slightly different timing and completed the stop_one_cpu() before > the hot-un-plug, migrate_tasks()/select_task_rq() would've had to fix up > anyway.
I agree userspace shouldn't need to care about this but in my case (using the test program I linked from the previous patch) this triggers whilst the CPU is being brought online, not taken offline. That means migrate_tasks() is not involved, and we actually just return from here back out from a sched_setaffinity syscall & continue running the user task on a CPU that is no longer present in the task's cpus_allowed.
I can't think of a good qualifier to limit the warning to only trigger in that scenario though, so in reality perhaps we're best to just trust that with patch 1 applied the problem will go away.
Thanks, Paul
| |