lkml.org 
[lkml]   [2021]   [Feb]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH v4 3/4] mm/gup: add a range variant of unpin_user_pages_dirty_lock()
    Date
    Add a unpin_user_page_range_dirty_lock() API which takes a starting page
    and how many consecutive pages we want to unpin and optionally dirty.

    To that end, define another iterator for_each_compound_range()
    that operates in page ranges as opposed to page array.

    For users (like RDMA mr_dereg) where each sg represents a
    contiguous set of pages, we're able to more efficiently unpin
    pages without having to supply an array of pages much of what
    happens today with unpin_user_pages().

    Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
    Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
    Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
    Reviewed-by: John Hubbard <jhubbard@nvidia.com>
    ---
    include/linux/mm.h | 2 ++
    mm/gup.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++
    2 files changed, 64 insertions(+)

    diff --git a/include/linux/mm.h b/include/linux/mm.h
    index 89fca443e6f1..43f59e4eb760 100644
    --- a/include/linux/mm.h
    +++ b/include/linux/mm.h
    @@ -1255,6 +1255,8 @@ static inline void put_page(struct page *page)
    void unpin_user_page(struct page *page);
    void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages,
    bool make_dirty);
    +void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages,
    + bool make_dirty);
    void unpin_user_pages(struct page **pages, unsigned long npages);

    /**
    diff --git a/mm/gup.c b/mm/gup.c
    index 384571be2c66..6b6d861727cf 100644
    --- a/mm/gup.c
    +++ b/mm/gup.c
    @@ -213,6 +213,32 @@ void unpin_user_page(struct page *page)
    }
    EXPORT_SYMBOL(unpin_user_page);

    +static inline void compound_range_next(unsigned long i, unsigned long npages,
    + struct page **list, struct page **head,
    + unsigned int *ntails)
    +{
    + struct page *next, *page;
    + unsigned int nr = 1;
    +
    + if (i >= npages)
    + return;
    +
    + next = *list + i;
    + page = compound_head(next);
    + if (PageCompound(page) && compound_order(page) >= 1)
    + nr = min_t(unsigned int,
    + page + compound_nr(page) - next, npages - i);
    +
    + *head = page;
    + *ntails = nr;
    +}
    +
    +#define for_each_compound_range(__i, __list, __npages, __head, __ntails) \
    + for (__i = 0, \
    + compound_range_next(__i, __npages, __list, &(__head), &(__ntails)); \
    + __i < __npages; __i += __ntails, \
    + compound_range_next(__i, __npages, __list, &(__head), &(__ntails)))
    +
    static inline void compound_next(unsigned long i, unsigned long npages,
    struct page **list, struct page **head,
    unsigned int *ntails)
    @@ -301,6 +327,42 @@ void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages,
    }
    EXPORT_SYMBOL(unpin_user_pages_dirty_lock);

    +/**
    + * unpin_user_page_range_dirty_lock() - release and optionally dirty
    + * gup-pinned page range
    + *
    + * @page: the starting page of a range maybe marked dirty, and definitely released.
    + * @npages: number of consecutive pages to release.
    + * @make_dirty: whether to mark the pages dirty
    + *
    + * "gup-pinned page range" refers to a range of pages that has had one of the
    + * pin_user_pages() variants called on that page.
    + *
    + * For the page ranges defined by [page .. page+npages], make that range (or
    + * its head pages, if a compound page) dirty, if @make_dirty is true, and if the
    + * page range was previously listed as clean.
    + *
    + * set_page_dirty_lock() is used internally. If instead, set_page_dirty() is
    + * required, then the caller should a) verify that this is really correct,
    + * because _lock() is usually required, and b) hand code it:
    + * set_page_dirty_lock(), unpin_user_page().
    + *
    + */
    +void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages,
    + bool make_dirty)
    +{
    + unsigned long index;
    + struct page *head;
    + unsigned int ntails;
    +
    + for_each_compound_range(index, &page, npages, head, ntails) {
    + if (make_dirty && !PageDirty(head))
    + set_page_dirty_lock(head);
    + put_compound_head(head, ntails, FOLL_PIN);
    + }
    +}
    +EXPORT_SYMBOL(unpin_user_page_range_dirty_lock);
    +
    /**
    * unpin_user_pages() - release an array of gup-pinned pages.
    * @pages: array of pages to be marked dirty and released.
    --
    2.17.1
    \
     
     \ /
      Last update: 2021-02-12 14:16    [W:5.037 / U:0.532 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site