lkml.org 
[lkml]   [2009]   [Sep]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: regression in page writeback
    On Mon, Sep 28, 2009 at 03:15:07PM +0800, Wu Fengguang wrote:
    > On Mon, Sep 28, 2009 at 09:07:00AM +0800, Dave Chinner wrote:
    > >
    > > pageout is so horribly inefficient from an IO perspective it is not
    > > funny. It is one of the reasons Linux sucks so much when under
    > > memory pressure. It basically causes the system to do random 4k
    > > writeback of dirty pages (and lumpy reclaim can make it
    > > synchronous!).
    > >
    > > pageout needs an enema, and preferably it should defer to background
    > > writeback to clean pages. background writeback will clean pages
    > > much, much faster than the random crap that pageout spews at the
    > > disk right now.
    > >
    > > Given that I can basically lock up my 2.6.30-based laptop for 10-15
    > > minutes at a time with the disk running flat out in low memory
    > > situations simply by starting to copy a large file(*), I think that
    > > the way we currently handle dirty page writeback needs a bit of a
    > > rethink.
    > >
    > > (*) I had this happen 4-5 times last week moving VM images around on
    > > my laptop, and it involved the Linux VM switching between pageout
    > > and swapping to make more memory available while the copy was was
    > > hammering the same drive with dirty pages from foreground writeback.
    > > It made for extremely fragmented files when the machine finally
    > > recovered because of the non-sequential writeback patterns on the
    > > single file being copied. You can't tell me that this is sane,
    > > desirable behaviour, and this is the sort of problem that I want
    > > sorted out. I don't beleive it can be fixed by maintaining the
    > > number of uncoordinated, competing writeback mechanisms we currently
    > > have.
    >
    > I imagined some lumpy pageout policy would help, but didn't realize
    > it's such a severe problem that can happen in daily desktop workload..
    >
    > Below is a quick patch. Any comments?

    Wow, it's much easier to reuse write_cache_pages for lumpy pageout :)

    ---
    mm/page-writeback.c | 36 ++++++++++++++++++++++++------------
    mm/shmem.c | 1 +
    mm/vmscan.c | 6 ++++++
    3 files changed, 31 insertions(+), 12 deletions(-)

    --- linux.orig/mm/vmscan.c 2009-09-29 07:21:51.000000000 +0800
    +++ linux/mm/vmscan.c 2009-09-29 07:46:59.000000000 +0800
    @@ -344,6 +344,8 @@ typedef enum {
    PAGE_CLEAN,
    } pageout_t;

    +#define LUMPY_PAGEOUT_PAGES (512 * 1024 / PAGE_CACHE_SIZE)
    +
    /*
    * pageout is called by shrink_page_list() for each dirty page.
    * Calls ->writepage().
    @@ -408,6 +410,10 @@ static pageout_t pageout(struct page *pa
    return PAGE_ACTIVATE;
    }

    + wbc.range_start = (page->index + 1) << PAGE_CACHE_SHIFT;
    + wbc.nr_to_write = LUMPY_PAGEOUT_PAGES - 1;
    + generic_writepages(mapping, &wbc);
    +
    /*
    * Wait on writeback if requested to. This happens when
    * direct reclaiming a large contiguous area and the
    --- linux.orig/mm/page-writeback.c 2009-09-29 07:33:13.000000000 +0800
    +++ linux/mm/page-writeback.c 2009-09-29 08:10:39.000000000 +0800
    @@ -799,6 +799,12 @@ retry:
    if (nr_pages == 0)
    break;

    + if (wbc->for_reclaim && done_index + nr_pages - 1 !=
    + pvec.pages[nr_pages - 1]->index) {
    + pagevec_release(&pvec);
    + break;
    + }
    +
    for (i = 0; i < nr_pages; i++) {
    struct page *page = pvec.pages[i];

    @@ -852,24 +858,30 @@ continue_unlock:
    if (!clear_page_dirty_for_io(page))
    goto continue_unlock;

    + /*
    + * active and unevictable pages will be checked at
    + * rotate time
    + */
    + if (wbc->for_reclaim)
    + SetPageReclaim(page);
    +
    ret = (*writepage)(page, wbc, data);
    if (unlikely(ret)) {
    if (ret == AOP_WRITEPAGE_ACTIVATE) {
    unlock_page(page);
    ret = 0;
    - } else {
    - /*
    - * done_index is set past this page,
    - * so media errors will not choke
    - * background writeout for the entire
    - * file. This has consequences for
    - * range_cyclic semantics (ie. it may
    - * not be suitable for data integrity
    - * writeout).
    - */
    - done = 1;
    - break;
    }
    + /*
    + * done_index is set past this page,
    + * so media errors will not choke
    + * background writeout for the entire
    + * file. This has consequences for
    + * range_cyclic semantics (ie. it may
    + * not be suitable for data integrity
    + * writeout).
    + */
    + done = 1;
    + break;
    }

    if (nr_to_write > 0) {
    --- linux.orig/mm/shmem.c 2009-09-29 08:07:22.000000000 +0800
    +++ linux/mm/shmem.c 2009-09-29 08:08:02.000000000 +0800
    @@ -1103,6 +1103,7 @@ unlock:
    */
    swapcache_free(swap, NULL);
    redirty:
    + wbc->pages_skipped++;
    set_page_dirty(page);
    if (wbc->for_reclaim)
    return AOP_WRITEPAGE_ACTIVATE; /* Return with page locked */

    \
     
     \ /
      Last update: 2009-09-29 02:17    [W:4.667 / U:0.068 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site