Messages in this thread | | | Date | Thu, 14 Oct 2004 04:23:07 -0700 | From | Paul Jackson <> | Subject | Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement |
| |
Eric wrote: > I have been quite confused by this thread in that I have not seen > any mechanism that looks beyond an individual processes at a time, > which seems so completely wrong.
In the simplest form, we obtain the equivalent of gang scheduling for the several threads of a tightly coupled job by arranging to have only one runnable thread per cpu, each such thread pinned on one cpu, and all threads in a given job simultaneously runnable.
For compute bound jobs, this is often sufficient. Time share (to a coarse granularity of minutes or hours) and overlap of various sized jobs is handled using suspension and migration in order to obtain the above invariants of one runnable thread per cpu at any given time, and of having all threads in a tightly coupled job pinned to distinct cpus and runnable simultaneously.
For jobs that are not compute bound, where other delays such as i/o would allow for running more than one such job at a time (both intermittendly runnable on a finer scale of seconds), then one needs something like gang scheduling in order to keep all the threads in a tightly coupled job running together, while still obtaining maximum utilization of cpu/memory hardware from jobs with cpu duty cycles of less than 50%.
The essential purpose of cpusets is to take the placement of individual threads by the sched_setaffinity and mbind/set_mempolicy calls, and extend that to manage placing groups of tasks on administratively designated and controlled groups of cpus/nodes.
If you see nothing beyond individual processes, then I think you are missing that.
However, it is correct that we haven't (so far as I recall) considered the gang scheduling that you describe. My crystal ball says we might get to that next year.
Gang scheduling isn't needed for the compute bound jobs, because just running a single job at a time on a given subset of a systems cpus and memory obtains the same result.
-- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@sgi.com> 1.650.933.1373 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |