[lkml]   [2007]   [Jul]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]
    On 2007.07.27 20:16:32 +0200, Rene Herman wrote:
    > On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
    >> Updatedb or another process that uses the FS heavily runs on a users
    >> 256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
    >> pressure that causes other applications to be swapped to disk. In the
    >> morning the user has to wait for the system to swap those applications
    >> back in.
    >> Questions about it:
    >> Q) Does swap-prefetch help with this? A) [From all reports I've seen (*)]
    >> Yes, it does.
    > No it does not. If updatedb filled memory to the point of causing swapping
    > (which noone is reproducing anyway) it HAS FILLED MEMORY and swap-prefetch
    > hasn't any memory to prefetch into -- updatedb itself doesn't use any
    > significant memory.
    > Here's swap-prefetch's author saying the same:
    > | It can't help the updatedb scenario. Updatedb leaves the ram full and
    > | swap prefetch wants to cost as little as possible so it will never
    > | move anything out of ram in preference for the pages it wants to swap
    > | back in.
    > Now please finally either understand this, or tell us how we're wrong.

    Con might have been wrong there for boxes with really little memory.

    My desktop box has not even 300k inodes in use (IIRC someone posted a df
    -i output showing 1 million inodes in use). Still, the memory footprint
    of the "sort" process grows up to about 50MB. Assuming that the average
    filename length stays, that would mean 150MB for the 1 million inode
    case, just for the "sort" process.

    Now, sort cannot produce any output before its got all its input, so
    that RSS usage exists at least as long as the VFS cache is growing due
    to the ongoing search for files.

    And then, all that memory that "sort" uses is required, because sort
    needs to output its results. So if there's memory pressure, the VFS
    cache is likely to be dropped, because "sort" needs its data, for
    sorting and producing output. And then sort terminates and leaves that
    whole lot of memory _unused_. The other actions of updatedb only touch
    the locate db, which is just a few megs (4.5MB here) big so the cache
    won't grow that much again.

    OK, so we got about, say, at least 128MB of totally unused memory, maybe
    even more. If you look at the vmstat output I sent, you see that I had
    between 90MB and 128MB free, depending on the swappiness setting, with
    increased inode usage, that could very well scale up.

    Conclusion: updatedb does _not_ leave the RAM full. And for a box with
    little memory (say 256MB) it might even be 50% or more memory that is
    free after updatedb ran. Might that make swap prefetch kick in?

    Any faults in that reasoning?

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2007-07-28 01:17    [W:0.023 / U:5.264 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site