lkml.org 
[lkml]   [2020]   [Aug]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[tip: x86/urgent] x86/entry/64: Do not use RDPID in paranoid entry to accomodate KVM
    The following commit has been merged into the x86/urgent branch of tip:

    Commit-ID: 6a3ea3e68b8a8a26c4aaac03432ed92269c9a14e
    Gitweb: https://git.kernel.org/tip/6a3ea3e68b8a8a26c4aaac03432ed92269c9a14e
    Author: Sean Christopherson <sean.j.christopherson@intel.com>
    AuthorDate: Fri, 21 Aug 2020 06:52:29 -04:00
    Committer: Thomas Gleixner <tglx@linutronix.de>
    CommitterDate: Fri, 21 Aug 2020 16:15:27 +02:00

    x86/entry/64: Do not use RDPID in paranoid entry to accomodate KVM

    KVM has an optmization to avoid expensive MRS read/writes on
    VMENTER/EXIT. It caches the MSR values and restores them either when
    leaving the run loop, on preemption or when going out to user space.

    The affected MSRs are not required for kernel context operations. This
    changed with the recently introduced mechanism to handle FSGSBASE in the
    paranoid entry code which has to retrieve the kernel GSBASE value by
    accessing per CPU memory. The mechanism needs to retrieve the CPU number
    and uses either LSL or RDPID if the processor supports it.

    Unfortunately RDPID uses MSR_TSC_AUX which is in the list of cached and
    lazily restored MSRs, which means between the point where the guest value
    is written and the point of restore, MSR_TSC_AUX contains a random number.

    If an NMI or any other exception which uses the paranoid entry path happens
    in such a context, then RDPID returns the random guest MSR_TSC_AUX value.

    As a consequence this reads from the wrong memory location to retrieve the
    kernel GSBASE value. Kernel GS is used to for all regular this_cpu_*()
    operations. If the GSBASE in the exception handler points to the per CPU
    memory of a different CPU then this has the obvious consequences of data
    corruption and crashes.

    As the paranoid entry path is the only place which accesses MSR_TSX_AUX
    (via RDPID) and the fallback via LSL is not significantly slower, remove
    the RDPID alternative from the entry path and always use LSL.

    The alternative would be to write MSR_TSC_AUX on every VMENTER and VMEXIT
    which would be inflicting massive overhead on that code path.

    [ tglx: Rewrote changelog ]

    Fixes: eaad981291ee3 ("x86/entry/64: Introduce the FIND_PERCPU_BASE macro")
    Reported-by: Tom Lendacky <thomas.lendacky@amd.com>
    Debugged-by: Tom Lendacky <thomas.lendacky@amd.com>
    Suggested-by: Andy Lutomirski <luto@kernel.org>
    Suggested-by: Peter Zijlstra <peterz@infradead.org>
    Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
    Link: https://lore.kernel.org/r/20200821105229.18938-1-pbonzini@redhat.com
    ---
    arch/x86/entry/calling.h | 10 ++++++----
    1 file changed, 6 insertions(+), 4 deletions(-)

    diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
    index 98e4d88..ae9b0d4 100644
    --- a/arch/x86/entry/calling.h
    +++ b/arch/x86/entry/calling.h
    @@ -374,12 +374,14 @@ For 32-bit we have the following conventions - kernel is built with
    * Fetch the per-CPU GSBASE value for this processor and put it in @reg.
    * We normally use %gs for accessing per-CPU data, but we are setting up
    * %gs here and obviously can not use %gs itself to access per-CPU data.
    + *
    + * Do not use RDPID, because KVM loads guest's TSC_AUX on vm-entry and
    + * may not restore the host's value until the CPU returns to userspace.
    + * Thus the kernel would consume a guest's TSC_AUX if an NMI arrives
    + * while running KVM's run loop.
    */
    .macro GET_PERCPU_BASE reg:req
    - ALTERNATIVE \
    - "LOAD_CPU_AND_NODE_SEG_LIMIT \reg", \
    - "RDPID \reg", \
    - X86_FEATURE_RDPID
    + LOAD_CPU_AND_NODE_SEG_LIMIT \reg
    andq $VDSO_CPUNODE_MASK, \reg
    movq __per_cpu_offset(, \reg, 8), \reg
    .endm
    \
     
     \ /
      Last update: 2020-08-21 16:22    [W:3.386 / U:0.788 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site