lkml.org 
[lkml]   [2000]   [Mar]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Overcomittable memory
    On Tue, 21 Mar 2000, James Sutherland wrote:

    > On Tue, 21 Mar 2000 13:57:07 +0100 (CET), you wrote:
    > >On Tue, 21 Mar 2000, James Sutherland wrote:
    > >
    > >[...[
    > >> So? If you touch the memory, overcommit doesn't come into play.
    > >
    > >All processes should do that for it to work. Example (you have 128MB of swap):
    > >
    > >you malloc() 1MB of space, and touch every page on it. Does swap gets

    ^^^^^^^^^^^^^^^^^^^^^^^^^

    > >allocated at this time? Assume it is (I'm not sure: the pages are dirty,
    > >but get written on swap when needed, at later time).
    > >
    > >Another process malloc 128MB of space. It does not touch the pages,
    > >overcommiting allows malloc() to succeed.
    > >
    > >RAM gets used up by some one else. Your pages get swapped out.
    >
    > No - you haven't allocated (touched) the pages yet. They are purely
    > imaginary pages until then.

    Please read again. I've already touched all the pages, as you suggested.
    Maybe i'm not been clear:

    1) process A malloc()s 1MB. It DOES touch them, right after malloc().

    2) process B malloc()s 128MB. It does NOT touch all of them. Only some.

    3) process C malloc()s 128MB. It does NOT touch all of them.

    4) process C starts accessing its pages sequentially. RAM gets allocated.

    5) RAM is short (suppose 64MB). the system starts paging-out A, B, C pages.
    Here are you sure that A swap space is already allocated? I'm not. Anyway,
    let's assume it is. As long as the systems pages out B and C pages,
    it allocates swap space. Sooner or later we're short of swap, because
    both B and C have address spaces larger than available swap (128MB - 1MB
    taken by A).

    6) A (who's been sleeping all the time) wakes up, accesses one of its pages.
    No page-frame is available, so the system tries to free one used by C
    or B. But no swap space is available. Page fault for process A fails
    to provide a free page-frame to A. Boom.

    > >Later, you access one page of yours. It's not in RAM. The system has to
    > >fetch it from swap. First, it has to locate a free page-frame. There is none.
    > >So, it has to swap-out some other page. But, wait, swap space is full.
    > >What happens? It's your process causing the page fault.
    > >
    > >Even if your process has all swap space allocated, it's not enough.
    > >Overcommiting for other processes still hits you.
    >
    > You seem to misunderstand. Basically, malloc() allocates ADDRESS
    > SPACE, not memory. USING that address space then allocates VM (RAM or
    > swap). Problems only arise if you haven't really allocated your memory
    > yet, you only think you have (because you misunderstand the malloc()
    > implementation.)

    That's what *I*'ve been saying in previous messages. I'm not misunderstanding
    malloc(). You are misunderstanding that your 'safe' process may still
    page-fault. Even if its own pages are on swap, you may need to free
    other processes pages, and you can't be sure *they* can be swapped out.

    > If YOU touch memory when malloc()ed (i.e. you REALLY allocate memory,
    > not just address space to put memory in) you're fine. Only the other
    > processes can be hit.

    No. Only if ALL processes touch ALL pages after malloc() you are
    (partially) safe. They also need NOT to grow their stacks, for example.

    >
    > James.
    >

    .TM.
    --
    ____/ ____/ /
    / / / Marco Colombo
    ___/ ___ / / Technical Manager
    / / / ESI s.r.l.
    _____/ _____/ _/ Colombo@ESI.it


    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 13:57    [W:4.147 / U:0.556 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site