I was following this thread some more and thought I would pass on
some first hand experience with SMP and binding.

Where I am currently working we have Quad processor 88k boxes from
motorola running Unix System V . Their backplane is VME. They
each have two network cards. Our server software runns to archiveing
process one for analog data and one for digital data. After some 
research we discovered that the default algorithm for allocating CPUS
to processes was wholly inadequate.

Each of these process have a tendecy to spike for small time durrations but typically
run a low cpu utilization. So we modified the code to bind each process
to a different CPU. In this case cpu 2 and 4. Since 1 and 2 are
on one board and 3 and 4 are on the second board this move the 
irq handling apart. and since CPU 1 handles most of the kernel processes.

How this relates to Linux.

I think for Linux we need the ability to:
- Bind a process to a CPU
  - Bind children to a CPU not always being the same cpu.
- Bind IRQS to a device.
- Be able to determine the relationship between devices, IRQS, and busses.
  (Dean Gaudet pointed out that BX chipsets support two PCI busses.)
  So a process might want to bind all IRQS from one one bus to one CPU
  or CPU cluster.
  ( There is a new chipset for the Pentium III that allows 8 CPUS I think
     these are really two clusters of 4 cpus.)
- Be able to bind a process to a memory location. This allows for 
  optimization of the caches on memory sticks in high performance enviroments.
  (cache on the DImm per say)


Now I imagine the kernel could decide that.
   - It should bind IRQS of multiples of the same device to different cpus.
      so if you had to network cards each one goes to a different CPU.
   - Decide which PCI bus should be matched to which CPU and try to prevent 
     crossover between the two,

   A future enhancement could be wheden a process request a 
   device bound to the other CPU that process is shifted to the new
    CPU. Of corse you would have to careful to prevent ossilation between two sets
   of CPUS, (set size being one or more)

O well I just though I would offer this up as some interesting ideas.

Leslie Donaldson
/----------------------------\ Current Contractor: Onsite Engineering  
|    Leslie F. Donaldson     | Current Customer  : Entergy - Nuclear GGNS
|    Computer Contractor     | Skills: Unix/OS9/VMS/Linux/SUN-OS/C/C++/assembly
| Have Computer will travel. | Curr   : http://www.cs.rose-hulman.edu/~donaldlf
\----------------------------/ Current Email  : donaldlf@cs.rose-hulman.edu
Goth Code V1.1: GoCS$$ TYg(T6,T9) B11Bk!^1 C6b-- P0(1,7) M+ a24 n--- b++:+
                H6'11" g m---- w+ r+++ D--~!% h+ s10 k+++ R-- Ssw LusCA++


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/