lkml.org 
[lkml]   [2021]   [Jul]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH v28 12/32] x86/mm: Update ptep_set_wrprotect() and pmdp_set_wrprotect() for transition from _PAGE_DIRTY to _PAGE_COW
    Date
    When Shadow Stack is introduced, [R/O + _PAGE_DIRTY] PTE is reserved for
    shadow stack. Copy-on-write PTEs have [R/O + _PAGE_COW].

    When a PTE goes from [R/W + _PAGE_DIRTY] to [R/O + _PAGE_COW], it could
    become a transient shadow stack PTE in two cases:

    The first case is that some processors can start a write but end up seeing
    a read-only PTE by the time they get to the Dirty bit, creating a transient
    shadow stack PTE. However, this will not occur on processors supporting
    Shadow Stack, and a TLB flush is not necessary.

    The second case is that when _PAGE_DIRTY is replaced with _PAGE_COW non-
    atomically, a transient shadow stack PTE can be created as a result.
    Thus, prevent that with cmpxchg.

    Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided many
    insights to the issue. Jann Horn provided the cmpxchg solution.

    Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
    Reviewed-by: Kees Cook <keescook@chromium.org>
    Reviewed-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    ---
    arch/x86/include/asm/pgtable.h | 36 ++++++++++++++++++++++++++++++++++
    1 file changed, 36 insertions(+)

    diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
    index cf7316e968df..df4ce715560a 100644
    --- a/arch/x86/include/asm/pgtable.h
    +++ b/arch/x86/include/asm/pgtable.h
    @@ -1278,6 +1278,24 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
    static inline void ptep_set_wrprotect(struct mm_struct *mm,
    unsigned long addr, pte_t *ptep)
    {
    + /*
    + * If Shadow Stack is enabled, pte_wrprotect() moves _PAGE_DIRTY
    + * to _PAGE_COW (see comments at pte_wrprotect()).
    + * When a thread reads a RW=1, Dirty=0 PTE and before changing it
    + * to RW=0, Dirty=0, another thread could have written to the page
    + * and the PTE is RW=1, Dirty=1 now. Use try_cmpxchg() to detect
    + * PTE changes and update old_pte, then try again.
    + */
    + if (cpu_feature_enabled(X86_FEATURE_SHSTK)) {
    + pte_t old_pte, new_pte;
    +
    + old_pte = READ_ONCE(*ptep);
    + do {
    + new_pte = pte_wrprotect(old_pte);
    + } while (!try_cmpxchg(&ptep->pte, &old_pte.pte, new_pte.pte));
    +
    + return;
    + }
    clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte);
    }

    @@ -1322,6 +1340,24 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm,
    static inline void pmdp_set_wrprotect(struct mm_struct *mm,
    unsigned long addr, pmd_t *pmdp)
    {
    + /*
    + * If Shadow Stack is enabled, pmd_wrprotect() moves _PAGE_DIRTY
    + * to _PAGE_COW (see comments at pmd_wrprotect()).
    + * When a thread reads a RW=1, Dirty=0 PMD and before changing it
    + * to RW=0, Dirty=0, another thread could have written to the page
    + * and the PMD is RW=1, Dirty=1 now. Use try_cmpxchg() to detect
    + * PMD changes and update old_pmd, then try again.
    + */
    + if (cpu_feature_enabled(X86_FEATURE_SHSTK)) {
    + pmd_t old_pmd, new_pmd;
    +
    + old_pmd = READ_ONCE(*pmdp);
    + do {
    + new_pmd = pmd_wrprotect(old_pmd);
    + } while (!try_cmpxchg((pmdval_t *)pmdp, (pmdval_t *)&old_pmd, pmd_val(new_pmd)));
    +
    + return;
    + }
    clear_bit(_PAGE_BIT_RW, (unsigned long *)pmdp);
    }

    --
    2.21.0
    \
     
     \ /
      Last update: 2021-07-22 22:54    [W:4.149 / U:0.284 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site