[lkml]   [1998]   [Oct]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: 'disposable' dirty pages [was: Out Of Memory in v. 2.1]
    On Thu, Oct 08, 1998 at 10:50:14AM -0400, Kenneth Albanowski wrote:
    > Elegant. I proposed something related to the libc list a while ago
    > (calling mmap(/dev/zero) on large-enough released chunks of memory), but
    > this has the benefit of not making the VMA list grow for each release.

    Note, mmap(/dev/zero) ought not to make the VMA list grow either -- the
    memory is already in a zero-mapped VMA. I don't know if the VMA merging
    code is clever enough to notice the file offset is irrelevant in this
    case. It should be.

    > However, it does mean that the memory is not automatically available to
    > the system, at least not without a reclamation cycle. Consider a program
    > which allocates 20meg on start, and then lives with 64k of RAM forever
    > afterwards. That memory will never be released until the system actively
    > goes looking, due to low-memory conditions.

    The system already has this problem for many kinds of free memory, when
    it's deciding what to swap out etc. Don't worry about it, the
    reclamation seems to be reasonably efficient.

    It could perhaps make better use of noticing pmds (not ptes) which are
    old to narrow the search? I digress.

    > I don't think it would make
    > sense to count the DISCARDABLE pages as actual free memory, as it isn't
    > available as such. Maybe another column in the memory report?

    It might be reasonable count them in the "cached" column.

    > If this were implemented (and, assuming a flag bit for DISCARDABLE bit is
    > available, it seems to be worthwhile without any significant costs), then
    > I'd also suggest a "harvester daemon" that goes searching for discardable
    > pages, and reclaims then for the OS. Here (and with the low-memory
    > reclamation, come to think of it) some concern needs to be taken for the
    > VMA lists. Reclaiming a contiguous chunk of pages is more efficient then
    > individual.

    It's called kswapd, which already does that looking for pages to swap
    out. Reclaiming DISCARDABLE pages is very similar. Or am I mistaken?

    > > The process doesn't care whether any of those pages are discarded or
    > > not.
    > Which isn't quite true, unfortunately. To make this true, the act of
    > making the page DISCARDABLE should also zero it.

    That's silly. You don't need DISCARDABLE to zero pages -- just zero-map
    them. If that makes a new VMA in malloc()ed memory, fix the kernel so
    it doesn't because it shouldn't.

    > Otherwise, you can tell whether a page was discarded by whether it
    > contains data, which in turn means programs with bugs behaving
    > differently depending on whether their pages were discarded.

    Well they already have that problem, because free() does not,
    traditionally, zero freed regions. Some debugging allocators do, and
    there's nothing stopping you using one to debug the appliation.

    > A minor problem, but these pages would not be transparent. (Free could
    > of course zero the pages itself, and everyone using mmap directly
    > merely needs to be aware of the issue.)

    free() doesn't traditionally zero pages because that's slow.
    Applications generally cope, though you're right to point out that bugs
    get in this way. To make matters worse, user-space zeroing would dirty
    all the pages in a block, which may will not have been touched
    otherwise. For large regions, zero-mapping is better for this.

    Efence is probably even better for debuggering.

    -- Jamie

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:44    [W:0.022 / U:6.116 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site