lkml.org 
[lkml]   [2009]   [Mar]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: ftruncate-mmap: pages are lost after writing to mmaped file.
    From
    On Wed, Mar 18, 2009 at 3:40 PM, Linus Torvalds
    <torvalds@linux-foundation.org> wrote:
    >
    >
    > On Wed, 18 Mar 2009, Andrew Morton wrote:
    >
    >> On Wed, 18 Mar 2009 12:44:08 -0700 Ying Han <yinghan@google.com> wrote:
    >> >
    >> > The "bad pages" count differs each time from one digit to 4,5 digit
    >> > for 128M ftruncated file. and what i also found that the bad page
    >> > number are contiguous for each segment which total bad pages container
    >> > several segments. ext "1-4, 9-20, 48-50" (  batch flushing ? )
    >
    > Yeah, probably the batched write-out.
    >
    > Can you say what filesystem, and what mount-flags you use? Iirc, last time
    > we had MAP_SHARED lost writes it was at least partly triggered by the
    > filesystem doing its own flushing independently of the VM (ie ext3 with
    > "data=journal", I think), so that kind of thing does tend to matter.

    /etc/fstab
    "/dev/hda1 / ext2 defaults 1 0"

    >
    > See for example commit ecdfc9787fe527491baefc22dce8b2dbd5b2908d.
    >
    >> > (The failure is reproduced based on 2.6.29-rc8, also happened on
    >> > 2.6.18 kernel. . Here is the simple test case to reproduce it with
    >> > memory pressure. )
    >>
    >> Thanks.  This will be a regression - the testing I did back in the days
    >> when I actually wrote stuff would have picked this up.
    >>
    >> Perhaps it is a 2.6.17 thing.  Which, IIRC, is when we made the changes to
    >> redirty pages on each write fault.  Or maybe it was something else.
    >
    > Hmm. I _think_ that changes went in _after_ 2.6.18, if you're talking
    > about Peter's exact dirty page tracking. If I recall correctly, that
    > became then 2.6.19, and then had the horrible mm dirty bit loss that
    > triggered in librtorrent downloads, which got fixed sometime after 2.6.20
    > (and back-ported).
    >
    > So if 2.6.18 shows the same problem, then it's a _really_ old bug, and not
    > related to the exact dirty tracking.
    >
    > The exact dirty accounting patch I'm talking about is d08b3851da41 ("mm:
    > tracking shared dirty pages"), but maybe you had something else in mind?
    >
    >> Given the amount of time for which this bug has existed, I guess it isn't a
    >> 2.6.29 blocker, but once we've found out the cause we should have a little
    >> post-mortem to work out how a bug of this nature has gone undetected for so
    >> long.
    >
    > I'm somewhat surprised, because this test-program looks like a very simple
    > version of the exact one that I used to track down the 2.6.20 mmap
    > corruption problems. And that one got pretty heavily tested back then,
    > when people were looking at it (December 2006) and then when trying out my
    > fix for it.
    >
    > Ying Han - since you're all set up for testing this and have reproduced it
    > on multiple kernels, can you try it on a few more kernel versions? It
    > would be interesting to both go further back in time (say 2.6.15-ish),
    > _and_ check something like 2.6.21 which had the exact dirty accounting
    > fix. Maybe it's not really an old bug - maybe we re-introduced a bug that
    > was fixed for a while.

    I will give a try.
    >
    >                                Linus
    >
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2009-03-19 00:21    [W:2.898 / U:0.308 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site