Messages in this thread Patch in this message | | | From | Toshi Kani <> | Subject | [PATCH] mm: Fix MAP_POPULATE and mlock() for DAX | Date | Tue, 16 Jun 2015 16:28:30 -0600 |
| |
DAX has the following issues in a shared or read-only private mmap'd file. - mmap(MAP_POPULATE) does not pre-fault - mlock() fails with -ENOMEM
DAX uses VM_MIXEDMAP for mmap'd files, which do not have struct page associated with the ranges. Both MAP_POPULATE and mlock() call __mm_populate(), which in turn calls __get_user_pages(). Because __get_user_pages() requires a valid page returned from follow_page_mask(), MAP_POPULATE and mlock(), i.e. FOLL_POPULATE, fail in the first page.
Change __get_user_pages() to proceed FOLL_POPULATE when the translation is set but its page does not exist (-EFAULT), and @pages is not requested. With that, MAP_POPULATE and mlock() set translations to the requested range and complete successfully.
MAP_POPULATE still provides a major performance improvement to DAX as it will avoid page faults during initial access to the pages.
mlock() continues to set VM_LOCKED to vma and populate the range. Since there is no struct page, the range is pinned without marking pages mlocked.
Note, MAP_POPULATE and mlock() already work for a write-able private mmap'd file on DAX since populate_vma_page_range() breaks COW, which allocates page caches.
Signed-off-by: Toshi Kani <toshi.kani@hp.com> --- mm/gup.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/mm/gup.c b/mm/gup.c index 6297f6b..16d536f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -490,8 +490,20 @@ retry: } BUG(); } - if (IS_ERR(page)) + if (IS_ERR(page)) { + /* + * No page may be associated with VM_MIXEDMAP. Proceed + * FOLL_POPULATE when the translation is set but its + * page does not exist (-EFAULT), and @pages is not + * requested by the caller. + */ + if ((PTR_ERR(page) == -EFAULT) && (!pages) && + (gup_flags & FOLL_POPULATE) && + (vma->vm_flags & VM_MIXEDMAP)) + goto next_page; + return i ? i : PTR_ERR(page); + } if (pages) { pages[i] = page; flush_anon_page(vma, page, start);
| |