Messages in this thread | | | Date | Mon, 03 May 2004 14:48:44 +0200 | From | Zoltan Menyhart <> | Subject | Re: NUMA API - wish list |
| |
Can you remember back the "old golden days" when there were no open(), read(), lseek(), write(), mmap(), etc., and one had to tell explicitly (job control punched cards) that s/he needed the sectors 123... 145 on the disk on channel 6 unit 7 ? Or somewhat more recently, one had to manage by hand the memory and the overlays. Now we are going to manage (from applications) the topology, CPU or memory binding. Moreover, to have the applications resolve resources management / dependency problems / conflicts among them...
The operating systems should provide for abstractions of the actual HW platform: file system, virtual memory, shared CPUs, etc.
Why should an application care for the actual physical characteristics ? Including counting nanoseconds of some HW resource access time ? We'll end up with some completely un-portable applications.
I think an application should describe what it needs for its optimal run, e.g.: - I need 3 * N (where N = 1, 2, 3,...) CPUs "very close" together and 2.5 Gbytes / N real memory (working set size) for each CPUs "very very close to" their respective CPUs - Should not it fit into a "domain", the CPUs have to be "very very close" to each other 3 by 3 - If no resources for even N == 1, do not start it at all - Use "gang scheduling" for them, otherwise I'll busy wait :-) - In addition, I need M CPUs + X Gbytes of memory "where my previous group is" and I need a disk I/O path of the capacity of 200 Mbytes / sec "more or less close to" my memory - I need "some more" CPUs "somewhere" with some 100 Mbytes of memory "preferably close to" the CPUs and 10 Mbytes / sec TCP/IP bandwidth "close to" my memory
- I need 70 % of the CPU time on my CPUs (the scheduler can select others for the 30 % of the time left)
- O.K. should my request be too much, here is my minimal, "degraded" configuration:...
The OS reserves the resources for the application (exec time assignment) and reports the applications what of its needs have been granted.
When the application allocates some memory, it'll say: you know, this is for the memory pool I've described in the 5th criteria. When it creates threads, it'll say they are in the 2nd group of threads mentioned at the 1st line
The work load manager / load balancer can negotiate other resource assignment at any time with the application. The work load manager / load balancer is free to move a collection of resources from some NUMA domains to others, provided the application's requirements are still met. (No hard binding.)
Billing is done accordingly :-)
As you do not need to know anything about SCSI LUNs, sector IDs, phy- sical memory maps or the other applications when you compile your kernel, why should an application care for HW NUMA details ?
Thanks,
Zoltán Menyhárt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |