lkml.org 
[lkml]   [2020]   [Mar]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH v6 3/9] mm: Split huge pages on write-notify or COW
    Date
    From: Thomas Hellstrom <thellstrom@vmware.com>

    The functions wp_huge_pmd() and wp_huge_pud() currently relies on the
    huge_fault() callback to split huge page table entries if needed.
    However for module users that requires export of the split_huge_xxx()
    functionality which may be undesired. Instead split pre-existing huge
    page-table entries on VM_FAULT_FALLBACK return.

    We currently only do COW and write-notify on the PTE level, so if the
    huge_fault() handler returns VM_FAULT_FALLBACK on wp faults,
    split the huge pages and page-table entries. Also do this for huge PUDs
    if there is no huge_fault() handler and the vma is not anonymous, similar
    to how it's done for PMDs.

    Note that fs/dax.c still does the splitting in the huge_fault() handler, but as
    huge_fault() A follow-up patch can remove the dax.c split_huge_pmd() if needed.

    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
    Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: Ralph Campbell <rcampbell@nvidia.com>
    Cc: "Jérôme Glisse" <jglisse@redhat.com>
    Cc: "Christian König" <christian.koenig@amd.com>
    Cc: Dan Williams <dan.j.williams@intel.com>
    Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
    Acked-by: Christian König <christian.koenig@amd.com>
    ---
    mm/memory.c | 27 +++++++++++++++++++--------
    1 file changed, 19 insertions(+), 8 deletions(-)

    diff --git a/mm/memory.c b/mm/memory.c
    index 0bccc622e482..1e3fc1988790 100644
    --- a/mm/memory.c
    +++ b/mm/memory.c
    @@ -3932,11 +3932,14 @@ static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf, pmd_t orig_pmd)
    {
    if (vma_is_anonymous(vmf->vma))
    return do_huge_pmd_wp_page(vmf, orig_pmd);
    - if (vmf->vma->vm_ops->huge_fault)
    - return vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD);
    + if (vmf->vma->vm_ops->huge_fault) {
    + vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD);

    - /* COW handled on pte level: split pmd */
    - VM_BUG_ON_VMA(vmf->vma->vm_flags & VM_SHARED, vmf->vma);
    + if (!(ret & VM_FAULT_FALLBACK))
    + return ret;
    + }
    +
    + /* COW or write-notify handled on pte level: split pmd. */
    __split_huge_pmd(vmf->vma, vmf->pmd, vmf->address, false, NULL);

    return VM_FAULT_FALLBACK;
    @@ -3949,12 +3952,20 @@ static inline bool vma_is_accessible(struct vm_area_struct *vma)

    static vm_fault_t create_huge_pud(struct vm_fault *vmf)
    {
    -#ifdef CONFIG_TRANSPARENT_HUGEPAGE
    +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \
    + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)
    /* No support for anonymous transparent PUD pages yet */
    if (vma_is_anonymous(vmf->vma))
    - return VM_FAULT_FALLBACK;
    - if (vmf->vma->vm_ops->huge_fault)
    - return vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PUD);
    + goto split;
    + if (vmf->vma->vm_ops->huge_fault) {
    + vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PUD);
    +
    + if (!(ret & VM_FAULT_FALLBACK))
    + return ret;
    + }
    +split:
    + /* COW or write-notify not handled on PUD level: split pud.*/
    + __split_huge_pud(vmf->vma, vmf->pud, vmf->address);
    #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
    return VM_FAULT_FALLBACK;
    }
    --
    2.21.1
    \
     
     \ /
      Last update: 2020-03-04 11:29    [W:5.197 / U:0.016 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site