Messages in this thread | | | Date | Thu, 22 Mar 2012 15:49:20 -0700 | From | "Paul E. McKenney" <> | Subject | Re: CPU Hotplug rework |
| |
On Thu, Mar 22, 2012 at 02:55:04PM +1030, Rusty Russell wrote: > On Wed, 21 Mar 2012 10:01:59 +0100, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote: > > On Wed, 2012-03-21 at 09:30 +1030, Rusty Russell wrote: > > > > > (2) Do something more efficient with userspace threads than migrating > > > > > them one at a time. > > > > > > > > Sadly that can't really be done. We need to pick up every task > > > > (userspace, but also running kernel threads) and update their state. > > > > > > What if we had an "orphan" runqueue which everyone pulled from? Then we > > > could grab the lock, move them all to the fake rq, then let stuff happen > > > normally. > > > > Well, we could simply let them sit where they are and fudge load-balance > > to consider it a source but not a destination until its empty, but it > > might be somewhat tricky to make it fast enough to not introduce > > noticable latencies. Also, you really don't want everyone to pull, > > that's a serialization/scalability problem. > > > > Also, since we really only move the currently runnable tasks it > > shouldn't be too many in the first place. Is it really that expensive? > > Good question, requires measurement to answer. > > > > Maybe that's crap, but at least we could move the migration out of the > > > hotplug callback somehow. > > > > Thing is, if its really too much for some people, they can orchestrate > > it such that its not. Just move everybody in a cpuset, clear the to be > > offlined cpu from the cpuset's mask -- this will migrate everybody away. > > Then hotplug will find an empty runqueue and its fast, no? > > I like this solution better.
As long as we have some way to handle kthreads that are algorithmically tied to a given CPU. There are coding conventions to handle this, for example, do everything with preemption disabled and just after each preempt_disable() verify that you are in fact running on the correct CPU, but it is easy to imagine improvements.
Thanx, Paul
| |