lkml.org 
[lkml]   [2017]   [Dec]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[patch V163 42/51] x86/mm: Abstract switching CR3
    From: Dave Hansen <dave.hansen@linux.intel.com>

    In preparation to adding additional PCID flushing, abstract the
    loading of a new ASID into CR3.

    [ PeterZ: Split out from big combo patch ]

    Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
    Signed-off-by: Ingo Molnar <mingo@kernel.org>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
    Cc: Andy Lutomirski <luto@kernel.org>
    Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Brian Gerst <brgerst@gmail.com>
    Cc: David Laight <David.Laight@aculab.com>
    Cc: Denys Vlasenko <dvlasenk@redhat.com>
    Cc: Eduardo Valentin <eduval@amazon.com>
    Cc: Greg KH <gregkh@linuxfoundation.org>
    Cc: H. Peter Anvin <hpa@zytor.com>
    Cc: Josh Poimboeuf <jpoimboe@redhat.com>
    Cc: Juergen Gross <jgross@suse.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Will Deacon <will.deacon@arm.com>
    Cc: aliguori@amazon.com
    Cc: daniel.gruss@iaik.tugraz.at
    Cc: hughd@google.com
    Cc: keescook@google.com
    ---
    arch/x86/mm/tlb.c | 22 ++++++++++++++++++++--
    1 file changed, 20 insertions(+), 2 deletions(-)

    --- a/arch/x86/mm/tlb.c
    +++ b/arch/x86/mm/tlb.c
    @@ -100,6 +100,24 @@ static void choose_new_asid(struct mm_st
    *need_flush = true;
    }

    +static void load_new_mm_cr3(pgd_t *pgdir, u16 new_asid, bool need_flush)
    +{
    + unsigned long new_mm_cr3;
    +
    + if (need_flush) {
    + new_mm_cr3 = build_cr3(pgdir, new_asid);
    + } else {
    + new_mm_cr3 = build_cr3_noflush(pgdir, new_asid);
    + }
    +
    + /*
    + * Caution: many callers of this function expect
    + * that load_cr3() is serializing and orders TLB
    + * fills with respect to the mm_cpumask writes.
    + */
    + write_cr3(new_mm_cr3);
    +}
    +
    void leave_mm(int cpu)
    {
    struct mm_struct *loaded_mm = this_cpu_read(cpu_tlbstate.loaded_mm);
    @@ -230,7 +248,7 @@ void switch_mm_irqs_off(struct mm_struct
    if (need_flush) {
    this_cpu_write(cpu_tlbstate.ctxs[new_asid].ctx_id, next->context.ctx_id);
    this_cpu_write(cpu_tlbstate.ctxs[new_asid].tlb_gen, next_tlb_gen);
    - write_cr3(build_cr3(next->pgd, new_asid));
    + load_new_mm_cr3(next->pgd, new_asid, true);

    /*
    * NB: This gets called via leave_mm() in the idle path
    @@ -243,7 +261,7 @@ void switch_mm_irqs_off(struct mm_struct
    trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, TLB_FLUSH_ALL);
    } else {
    /* The new ASID is already up to date. */
    - write_cr3(build_cr3_noflush(next->pgd, new_asid));
    + load_new_mm_cr3(next->pgd, new_asid, false);

    /* See above wrt _rcuidle. */
    trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, 0);

    \
     
     \ /
      Last update: 2017-12-18 12:58    [W:4.031 / U:0.032 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site