Messages in this thread | | | Date | Tue, 12 Oct 2004 09:09:24 +1000 | From | Nick Piggin <> | Subject | Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement |
| |
Matthew Dobson wrote: > On Fri, 2004-10-08 at 17:18, Nick Piggin wrote: > >>Matthew Dobson wrote: >> >>>I think this example is easily achievable with the sched_domains >>>modifications I am proposing. You can still create your 128 CPU >>>exclusive domain, called big_domain (due to my lack of naming >>>creativity), and further divide big_domain into smaller, non-exclusive >>>sched_domains. We do this all the time, albeit statically at boot time, >>>with the current sched_domains code. When we create a 4-node domain on >>>IA64, and underneath it we create 4 1-node domains. We've now >>>partitioned the system into 4 sched_domains, each containing 4 cpus. >>>Balancing between these 4 node-level sched_domains is allowed, but can >>>be disallowed by not setting the SD_LOAD_BALANCE flag. Your example >>>does show that it can be more than just a convenient way to group tasks, >>>but your example can be done with what I'm proposing. >> >>You wouldn't be able to do this just with sched domains, because >>it doesn't know anything about individual tasks. As soon as you >>have some overlap, all your tasks can escape out of your domain. >> >>I don't think there is a really nice way to do overlapping sets. >>Those that want them need to just use cpu affinity for now. > > > Well, the tasks can escape out of the domain iff you have the > SD_LOAD_BALANCE flag set on that domain. If SD_LOAD_BALANCE isn't set, > then when the scheduler tick goes off, and the code looks at the domain, > it will see the lack of the flag and will not attempt to balance the > domain, correct? This is what we currently do with the 'isolated' > domains, right? >
Yeah that's right. Well you have to remove some of the other SD_ flags as well (eg. SD_BALANCE_EXEC, SD_WAKE_BALANCE).
But I don't think there is much point in overlapping sets which don't do any balancing. They might as well not exist at all.
> You're right that you can get some of the more obscure semantics of the > various flavors of cpusets by leveraging sched_domains AND > cpus_allowed. I don't have any desire to remove that ability, just keep > it as the exception. >
I think at this stage, overlapping cpu sets are the exception. It is pretty logical that they're going to require some per-task info, because the balancer can't otherwise differentiate between two tasks on the same runqueue but in different cpu sets.
sched-domains gives you a nice clean way to do exclusive partitioning, and I can't imagine it would be too common to want to do overlapping partitioning. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |