lkml.org 
[lkml]   [2011]   [Oct]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH RFC 2/2] kvm: set affinity hint for assigned device msi
On Tue, Oct 11, 2011 at 08:38:28PM +0200, Michael S. Tsirkin wrote:
> To forward an interrupt to a vcpu that runs on
> a host cpu different from the current one,
> we need an ipi which likely will cost us as much
> as delivering the interrupt directly to that cpu would.
>
> Set irq affinity hint to point there, irq balancer
> can then take this into accound and balance
> interrupts accordingly.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> virt/kvm/assigned-dev.c | 8 +++++---
> virt/kvm/irq_comm.c | 17 ++++++++++++++++-
> 2 files changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/virt/kvm/assigned-dev.c b/virt/kvm/assigned-dev.c
> index f89f138..b579777 100644
> --- a/virt/kvm/assigned-dev.c
> +++ b/virt/kvm/assigned-dev.c
> @@ -142,9 +142,11 @@ static void deassign_host_irq(struct kvm *kvm,
> for (i = 0; i < assigned_dev->entries_nr; i++)
> disable_irq(assigned_dev->host_msix_entries[i].vector);
>
> - for (i = 0; i < assigned_dev->entries_nr; i++)
> - free_irq(assigned_dev->host_msix_entries[i].vector,
> - (void *)assigned_dev);
> + for (i = 0; i < assigned_dev->entries_nr; i++) {
> + u32 vector = assigned_dev->host_msix_entries[i].vector;
> + irq_set_affinity_hint(vector, NULL);
> + free_irq(vector, (void *)assigned_dev);
> + }
>
> assigned_dev->entries_nr = 0;
> kfree(assigned_dev->host_msix_entries);
> diff --git a/virt/kvm/irq_comm.c b/virt/kvm/irq_comm.c
> index ac8b629..68b1f7c 100644
> --- a/virt/kvm/irq_comm.c
> +++ b/virt/kvm/irq_comm.c
> @@ -22,6 +22,7 @@
>
> #include <linux/kvm_host.h>
> #include <linux/slab.h>
> +#include <linux/interrupt.h>
> #include <trace/events/kvm.h>
>
> #include <asm/msidef.h>
> @@ -80,6 +81,17 @@ inline static bool kvm_is_dm_lowest_prio(struct kvm_lapic_irq *irq)
> #endif
> }
>
> +static void kvm_vcpu_host_irq_hint(struct kvm_vcpu *vcpu, int host_irq)
> +{
> + const struct cpumask *mask;
> + /* raw_smp_processor_id() is ok here: if we get preempted we can get a
> + * wrong value but we don't mind much. */
> + if (host_irq >= 0 && unlikely(vcpu->cpu != raw_smp_processor_id())) {
> + mask = get_cpu_mask(vcpu->cpu);
> + irq_set_affinity_hint(host_irq, mask);
> + }
> +}

Unsure about the internals of irq_set_affinity_hint, but AFAICS its
exported so that irqbalance in userspace can make a decision.

If that is the case, then irqbalance update rate should be high enough
to catch up with a vcpu migrating betweens cpus (which initially does
not appear a sensible arrangement).

The decision to have the host interrupt follow the vcpu seems a good
one, given that it saves an IPI and is potentially more cache friendly
overall.

And AFAICS its more intelligent for the device assignment case than
anything irqbalance can come up with (note it depends on how the APIC is
configured, your patch ignores that).

> +
> int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
> struct kvm_lapic_irq *irq, int host_irq)
> {
> @@ -102,6 +114,7 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
> if (r < 0)
> r = 0;
> r += kvm_apic_set_irq(vcpu, irq);
> + kvm_vcpu_host_irq_hint(vcpu, host_irq);
> } else if (kvm_lapic_enabled(vcpu)) {
> if (!lowest)
> lowest = vcpu;
> @@ -110,8 +123,10 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
> }
> }
>
> - if (lowest)
> + if (lowest) {
> r = kvm_apic_set_irq(lowest, irq);
> + kvm_vcpu_host_irq_hint(vcpu, host_irq);
> + }
>
> return r;
> }
> --
> 1.7.5.53.gc233e


\
 
 \ /
  Last update: 2011-10-13 16:59    [W:1.014 / U:0.720 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site