lkml.org 
[lkml]   [2012]   [Jul]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [PATCH v3 0/2] kvm: level irqfd and new eoifd
From
Date
On Sun, 2012-07-15 at 13:09 +0300, Avi Kivity wrote:
> On 07/12/2012 08:38 PM, Alex Williamson wrote:
> > On Thu, 2012-07-12 at 10:19 -0600, Alex Williamson wrote:
> >> On Thu, 2012-07-12 at 12:35 +0300, Avi Kivity wrote:
> >> > On 07/11/2012 10:57 PM, Alex Williamson wrote:
> >> > >>
> >> > >> > We still have classic KVM device assignment to provide fast-path INTx.
> >> > >> > But if we want to replace it midterm, I think it's necessary for VFIO to
> >> > >> > be able to provide such a path as well.
> >> > >>
> >> > >> I would like VFIO to have no regressions vs. kvm device assignment,
> >> > >> except perhaps in uncommon corner cases. So I agree.
> >> > >
> >> > > I ran a few TCP_RR netperf tests forcing a 1Gb tg3 nic to use INTx.
> >> > > Without irqchip support vfio gets a bit more than 60% of KVM device
> >> > > assignment. That's a little bit of an unfair comparison since it's more
> >> > > than just the I/O path. With the proposed interfaces here, enabling
> >> > > irqchip, vfio is within 10% of KVM device assignment for INTx. For MSI,
> >> > > I can actually make vfio come out more than 30% better than KVM device
> >> > > assignment if I send the eventfd from the hard irq handler. Using a
> >> > > threaded handler as the code currently does, vfio is still behind KVM.
> >> > > It's hard to beat a direct call chain.
> >> >
> >> > We can have a direct call chain with vfio too, using a custom eventfd
> >> > poll function, no? Assuming we set up a fast path for unicast msi.
> >>
> >> You'll have to help me out a little, eventfd_signal walks the wait_queue
> >> and calls each function. On the injection path that includes
> >> irqfd_wakeup. For an MSI that seems to already provide direct
> >> injection. For level we'll schedule_work, so that explains the overhead
> >> in that path, but it's not too dissimilar to a a threaded irq. vfio
> >> does something very similar, so there's a schedule_work both on inject
> >> and on eoi. I'll have to check whether anything prevents the unmask
> >> from the wait_queue function in vfio, that could be a significant chunk
> >> of the gap.
> >
> > Yep, the schedule_work in the eoi is the culprit. A direct unmask from
> > the wait queue function gives me better results than kvm for INTx.
> > We'll have to see how the leapfrogging goes once KVM switches to
> > injection from the hard handler. I'm still curious what this custom
> > poll function would give us though. Thanks,
> >
>
> btw, why is the overhead so large? A context switch should be on the
> order of 1 microsecond or less. Given that, every 5000 context switches
> per second cost a 1% cpu load on one core. You would need a very heavy
> interrupt load to see a large degradation. Or is the extra latency the
> problem?

I'm using TCP_RR, so latency is the factor. As I mentioned though, I
have way too much kernel debugging enabled to take these as anything
more than rough estimates. Thanks,

Alex



\
 
 \ /
  Last update: 2012-07-16 16:41    [W:0.130 / U:1.128 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site