Messages in this thread | | | Subject | Re: [PATCH 02/13] irq: Introduce IRQD_AFFINITY_MANAGED flag | From | Bart Van Assche <> | Date | Wed, 15 Jun 2016 21:36:54 +0200 |
| |
On 06/15/2016 06:03 PM, Keith Busch wrote: > On Wed, Jun 15, 2016 at 05:28:54PM +0200, Bart Van Assche wrote: >> On 06/15/2016 05:14 PM, Keith Busch wrote: >>> I think the idea is have the irq_affinity mask match the CPU mapping on >>> the submission side context associated with that particular vector. If >>> two identical adapters generate the same submission CPU mapping, I don't >>> think we can do better than matching irq_affinity masks. >> >> Has this been verified by measurements? Sorry but I'm not convinced that >> using the same mapping for multiple identical adapters instead of spreading >> interrupts will result in better performance. > > The interrupts automatically spread based on which CPU submitted the > work. If you want to spread interrupts across more CPUs, then you can > spread submissions to the CPUs you want to service the interrupts. > > Completing work on the same CPU that submitted it is quickest with > its cache hot access. I have equipment available to demo this. What > affinty_mask policy would you like to see compared with the proposal?
Hello Keith,
Sorry that I had not yet this made this clear but my concern is about a system equipped with two or more adapters and with more CPU cores than the number of MSI-X interrupts per adapter. Consider e.g. a system with two adapters (A and B), 8 interrupts per adapter (A0..A7 and B0..B7), 32 CPU cores and two NUMA nodes. Assuming that hyperthreading is disabled, will the patches from this patch series generate the following interrupt assignment?
0: A0 B0 1: A1 B1 2: A2 B2 3: A3 B3 4: A4 B4 5: A5 B5 6: A6 B6 7: A7 B7 8: (none) ... 31: (none)
The mapping I would like to see is as follows (assuming CPU cores 0..15 correspond to NUMA node 0 and CPU cores 16..31 correspond to NUMA node 1):
0: A0 1: B0 2: (none) 3: (none) 4: A1 5: B1 6: (none) 7: (none) 8: A2 9: B2 10: (none) 11: (none) 12: A3 13: B3 14: (none) 15: (none) ... 31: (none)
Do you agree that - ignoring other interrupt assignments - that the latter interrupt assignment scheme would result in higher throughput and lower interrupt processing latency?
Thanks,
Bart.
| |