lkml.org 
[lkml]   [2016]   [May]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Subject[PART1 V5 00/13] KVM: x86: Introduce SVM AVIC support
    Date
    CHANGES FROM RFCv4:
    ==================
    (https://lkml.org/lkml/2016/4/7/87)

    * Removing the RFC since I think this is getting ready.
    * Rebase to latest tip.git.
    * Rename vm_deinit to vm_destroy.
    * Replace svm_vcpu_avic_enabled() with kvm_vcpu_apicv_active().
    * Fix the cluster logical APIC ID calculation logic.
    * Misc clean up based on previous review comments.
    * (NEW) Rename kvm_lapic_get_reg to kvm_lapic_get_reg.
    * (NEW) Introduce kvm_x86_ops.apicv_post_state_restore hook.
    * (NEW) Re-factor the VMEXIT handling code and reuse
    it in the apicv_post_state_restore to implement support for
    vmsave/restore, which has been tested migrating:
    - from AVIC mode to non-AVIC mode
    - from non-AVIC mode to AVIC mode
    * (NEW) Add support for the AVIC VMCB clean bit.

    GITHUB
    ======
    Latest git tree can be found at:
    http://github.com/ssuthiku/linux.git avic_part1_v5

    OVERVIEW
    ========
    This patch set is the first of the two-part patch series to introduce
    the new AMD Advance Virtual Interrupt Controller (AVIC) support.

    Basically, SVM AVIC hardware virtualizes local APIC registers of each
    vCPU via the virtual APIC (vAPIC) backing page. This allows guest access
    to certain APIC registers without the need to emulate the hardware behavior
    in the hypervisor. More information about AVIC can be found in the
    AMD64 Architecture Programmer’s Manual Volume 2 - System Programming.

    http://support.amd.com/TechDocs/24593.pdf

    For SVM AVIC, we extend the existing kvm_amd driver to:
    * Check CPUID to detect AVIC support in the processor
    * Program new fields in VMCB to enable AVIC
    * Introduce new AVIC data structures and add code to manage them
    * Handle two new AVIC #VMEXITs
    * Add new interrupt intjection code using vAPIC backing page
    instead of the existing V_IRQ, V_INTR_PRIO, V_INTR_VECTOR,
    and V_IGN_TPR fields

    Currently, this patch series does not enable AVIC by default.
    Users can enable SVM AVIC by specifying avic=1 during insmod kvm-amd.

    Later, in part 2, we will introduce the IOMMU AVIC support, which
    provides speed up for PCI device pass-through use case by allowing
    the IOMMU hardware to inject interrupt directly into the guest via
    the vAPIC backing page.

    PERFORMANCE RESULTS
    ===================
    Currently, AVIC is supported in the AMD family 15h models 6Xh
    (Carrizo) processors. Therefore, it is used to collect the
    perforamance data shown below.

    Generaly, SVM AVIC alone (w/o IOMMU AVIC) should provide speedup for
    IPI interrupt since hypervisor does not require VMEXIT to inject
    these interrupts. Also, it should speed up the case when hypervisor
    wants to inject an interrupt into a running guest by setting the
    corresponded IRR bit in the vAPIC backing page and trigger
    AVIC_DOORBELL MSR.

    IPI PERFORMANCE
    ===============

    * BENCHMARK 1: HACKBENCH
    For IPI, I have collected some performance number on 2 and 4 CPU running
    hackbech with the following detail:

    hackbench -p -l 100000
    Running in process mode with 10 groups using 40 file descriptors each (== 400 tasks)
    Each sender will pass 100000 messages of 100 bytes

    | 2 vcpus | 4 vcpus
    ------------------------------------------------
    Vanila | 273.76 | 190.21
    AVIC disabled | 260.51 (~5%) | 184.40 (~5%)
    AVIC | 248.53 (~10%) | 155.01 (~20%)

    OVERALL PERFORMANCE
    ===================
    Enabling AVIC should helps speeding up workloads, which generate
    large amount of interrupts. However, it requires additional logics to
    maintain AVIC-specific data structures during vCPU load/unload
    due to vcpu scheduling.

    The goal is to minimize the overhead of AVIC in most cases, so that
    we can achieve equivalent or improvement in overall performance when
    enabling AVIC.

    * BENCHMARK 1: TAR DECOMPRESSION
    This test measures the average running time (of 10 runs) of the following
    tar decompression command with 1, 2, and 4 vcpus.

    tar xf linux-4.3.3.tar.xz

    | 4 vcpus
    ---------------------------------
    Vanila | 10.26
    AVIC disabled | 10.10 (~1.5%)
    AVIC | 10.07 (~1.8%)

    Note: The unit of result below is in seconds (lower is better).

    * BENCHMARK 2: NETPERF w/ virtual network
    This test creates a virtual network by setting up bridge and tap device
    on the host and pass it into the VM as virtio-net-pci device w/ vhost.
    Then it sets up netserver in the host machine, and run netperf
    in the VM with following option:

    netperf -H <netserver ip> -l 60 -t TCP_RR -D 2

    | 1 vcpu
    ------------------------------------
    Vanila | 21623.887
    AVIC disabled | 21538.09 (~-.4%)
    AVIC | 21712.68 (~0.4%)

    Note: The unit of result below is trans/sec (higher is better).

    Preliminary result of both benchmarks show AVIC performance are slightly
    better than the other two cases.

    CURRENT UNTESTED USE-CASES
    ===========================
    - Nested VM

    Any feedback and comments are very much appreciated.

    Thank you,
    Suravee

    Radim Krčmář (1):
    KVM: split kvm_vcpu_wake_up from kvm_vcpu_kick

    Suravee Suthikulpanit (12):
    KVM: x86: Misc LAPIC changes to expose helper functions
    KVM: x86: Rename kvm_apic_get_reg to kvm_lapic_get_reg
    KVM: x86: Introducing kvm_x86_ops VM init/destroy hooks
    KVM: x86: Introducing kvm_x86_ops VCPU blocking/unblocking hooks
    svm: Introduce new AVIC VMCB registers
    KVM: x86: Detect and Initialize AVIC support
    svm: Add interrupt injection via AVIC
    svm: Add VMEXIT handlers for AVIC
    KVM: x86: Introducing kvm_x86_ops.apicv_post_state_restore
    svm: Do not expose x2APIC when enable AVIC
    svm: Do not intercept CR8 when enable AVIC
    svm: Manage vcpu load/unload when enable AVIC

    arch/x86/include/asm/kvm_host.h | 26 +-
    arch/x86/include/asm/svm.h | 12 +-
    arch/x86/include/uapi/asm/svm.h | 9 +-
    arch/x86/kvm/ioapic.c | 2 +-
    arch/x86/kvm/lapic.c | 187 +++++-------
    arch/x86/kvm/lapic.h | 38 ++-
    arch/x86/kvm/svm.c | 660 +++++++++++++++++++++++++++++++++++++++-
    arch/x86/kvm/trace.h | 57 ++++
    arch/x86/kvm/x86.c | 7 +
    include/linux/kvm_host.h | 1 +
    virt/kvm/kvm_main.c | 19 +-
    11 files changed, 891 insertions(+), 127 deletions(-)

    --
    1.9.1

    \
     
     \ /
      Last update: 2016-05-04 21:41    [W:2.690 / U:0.552 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site