Messages in this thread | | | Date | Thu, 22 Dec 2022 14:56:16 -0700 | From | Keith Busch <> | Subject | Re: [PATCH] PCI: vmd: Do not disable MSI-X remapping in VMD 28C0 controller |
| |
On Thu, Dec 22, 2022 at 02:15:20AM -0700, Jonathan Derrick wrote: > On 12/22/22 12:26 AM, korantwork@gmail.com wrote: > > > > However, the bypass mode could increase the interrupts costs in CPU. > > We test 12 disks in the 6 CPU, > > Well the bypass mode was made to improve performance where you have >4 > drives so this is pretty surprising. With bypass mode disabled, VMD will > intercept and forward interrupts, increasing costs. > > I think Nirmal would want to to understand if there's some other factor > going on here.
With 12 drives and only 6 CPUs, the bypass mode is going to get more irq context switching. Sounds like the non-bypass mode is aggregating and spreading interrupts across the cores better, but there's probably some cpu:drive count tipping point where performance favors the other way.
The fio jobs could also probably set their cpus_allowed differently to get better performance in the bypass mode.
| |