Messages in this thread | | | Date | Wed, 19 Apr 2006 07:30:37 -0700 | From | "Martin J. Bligh" <> | Subject | Re: irqbalance mandatory on SMP kernels? |
| |
Erik Mouw wrote: > On Tue, Apr 18, 2006 at 08:19:17PM +0200, Arjan van de Ven wrote: > >>>but spreading IRQ's across all of the CPU's doesn't seem like it's >>>ever the right answer. >> >>well it is in some cases, imagine having 2 cpus and 2 gige nics that are >>very busy doing webserving. That's an obvious case where 1-nic-per-cpu >>ends up doing the right thing... the way it ends up is that each nic has >>a full cpu for itself and it's own apaches... almost fully independent >>of the other one. Now if you moved both irqs to the same cpu, the >>apaches would follow, because if they didn't then you'd be bouncing >>their data *all the time*. And at that point the other cpu will become >>bored ;) > > > So what happens with a dual amd64 system where each CPU has its "own" > NIC? Something like this: > > > MEM0 <--> CPU0 <--- HT ---> CPU1 <--> MEM1 > ^ ^ > | | > HT HT > | | > v v > PCI bridge0 PCI1 bridge > ^ ^ > | | > PCI PCI > | | > v v > GigE NIC0 GigE NIC1 > > The "best" approach would be to run an Apache on each CPU using local > memory and NIC and having the IRQs handled by the local CPU. Does the > irq balancer allow such a configuraion, or would it be hamperd by the > process scheduler deciding to run both Apaches on a single CPU?
The trouble is that we're not smart enough to redirect receiving traffic back to the correct CPU, I think. You'd need to configure different IP addrs on the same subnet to each NIC, and have intelligent bonding for outbound traffic.
Then you'd need NUMA locality of IRQs, by bonding them to their closest CPUs, which is something that's easily done inside the kernel, but is harder in userspace. but I'm not getting into that silly pissing contest again. Either mechanism *could* do it. It's just about creating sensbile APIs and defaults, both of which we're not good at doing as a community.
M. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |