lkml.org 
[lkml]   [2015]   [Sep]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCHv10 06/36] mm: handle PTE-mapped tail pages in gerneric fast gup implementaiton
    Date
    With new refcounting we are going to see THP tail pages mapped with PTE.
    Generic fast GUP rely on page_cache_get_speculative() to obtain
    reference on page. page_cache_get_speculative() always fails on tail
    pages, because ->_count on tail pages is always zero.

    Let's handle tail pages in gup_pte_range().

    New split_huge_page() will rely on migration entries to freeze page's
    counts. Recheck PTE value after page_cache_get_speculative() on head
    page should be enough to serialize against split.

    Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Tested-by: Sasha Levin <sasha.levin@oracle.com>
    Tested-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
    Acked-by: Jerome Marchand <jmarchan@redhat.com>
    Acked-by: Vlastimil Babka <vbabka@suse.cz>
    ---
    mm/gup.c | 8 +++++---
    1 file changed, 5 insertions(+), 3 deletions(-)

    diff --git a/mm/gup.c b/mm/gup.c
    index 745a50f2d57d..6880085d3790 100644
    --- a/mm/gup.c
    +++ b/mm/gup.c
    @@ -1097,7 +1097,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
    * for an example see gup_get_pte in arch/x86/mm/gup.c
    */
    pte_t pte = READ_ONCE(*ptep);
    - struct page *page;
    + struct page *head, *page;

    /*
    * Similar to the PMD case below, NUMA hinting must take slow
    @@ -1109,15 +1109,17 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,

    VM_BUG_ON(!pfn_valid(pte_pfn(pte)));
    page = pte_page(pte);
    + head = compound_head(page);

    - if (!page_cache_get_speculative(page))
    + if (!page_cache_get_speculative(head))
    goto pte_unmap;

    if (unlikely(pte_val(pte) != pte_val(*ptep))) {
    - put_page(page);
    + put_page(head);
    goto pte_unmap;
    }

    + VM_BUG_ON_PAGE(compound_head(page) != head, page);
    pages[*nr] = page;
    (*nr)++;

    --
    2.5.0


    \
     
     \ /
      Last update: 2015-09-03 17:41    [W:4.586 / U:0.012 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site