Messages in this thread | | | Date | Thu, 24 Apr 2008 21:19:19 -0700 (PDT) | Subject | Re: MSI, fun for the whole family | From | David Miller <> |
| |
From: Roland Dreier <rdreier@cisco.com> Date: Thu, 24 Apr 2008 20:57:48 -0700
> Now, it is true that the kernel could do something crazy and collapse > all these interrupt vectors into a single "IRQ" and then tell the > interrupt handler which vector it was by passing some "metadata" in, but > why not just give each MSI message it's own IRQ?
Actually, it doesn't make any sense to have more MSI, or "MSI queue" interrupts than you have cpus.
Non-x86 PCI-E controller impelemntations that I am familiar with collect MSI and MSI-X interrupts into "queues", these queues being non-empty is what actually triggers an interrupt to the CPU. And, there are enough MSI queue instances such that you can direct each one to a unique cpu.
The MSI queue interrupt simply scans the ring buffer of pending MSI interrupts and dispatches them to the device.
You can handle PCI-E frabric error messages the same way, and in fact that's what the controllers I am familiar with do.
A Linux implementation of support for this kind of setup can be seen in arch/sparc64/kernel/pci_msi.c:sparc64_msiq_interrupt(). It's very generic and doesn't care whether it's talking to real PCI controller hardware or a hypervisor based interface.
Besides the obvious extra indirection overhead, our IRQ layer is very much capable of supporting multi-level dispatch like this correctly.
| |