Messages in this thread | | | Date | Tue, 21 Mar 2000 16:18:13 +0100 (CET) | From | Marco Colombo <> | Subject | Re: Overcomittable memory |
| |
On Tue, 21 Mar 2000, James Sutherland wrote:
> On Tue, 21 Mar 2000 13:57:07 +0100 (CET), you wrote: > >On Tue, 21 Mar 2000, James Sutherland wrote: > > > >[...[ > >> So? If you touch the memory, overcommit doesn't come into play. > > > >All processes should do that for it to work. Example (you have 128MB of swap): > > > >you malloc() 1MB of space, and touch every page on it. Does swap gets
^^^^^^^^^^^^^^^^^^^^^^^^^
> >allocated at this time? Assume it is (I'm not sure: the pages are dirty, > >but get written on swap when needed, at later time). > > > >Another process malloc 128MB of space. It does not touch the pages, > >overcommiting allows malloc() to succeed. > > > >RAM gets used up by some one else. Your pages get swapped out. > > No - you haven't allocated (touched) the pages yet. They are purely > imaginary pages until then.
Please read again. I've already touched all the pages, as you suggested. Maybe i'm not been clear:
1) process A malloc()s 1MB. It DOES touch them, right after malloc().
2) process B malloc()s 128MB. It does NOT touch all of them. Only some.
3) process C malloc()s 128MB. It does NOT touch all of them.
4) process C starts accessing its pages sequentially. RAM gets allocated.
5) RAM is short (suppose 64MB). the system starts paging-out A, B, C pages. Here are you sure that A swap space is already allocated? I'm not. Anyway, let's assume it is. As long as the systems pages out B and C pages, it allocates swap space. Sooner or later we're short of swap, because both B and C have address spaces larger than available swap (128MB - 1MB taken by A). 6) A (who's been sleeping all the time) wakes up, accesses one of its pages. No page-frame is available, so the system tries to free one used by C or B. But no swap space is available. Page fault for process A fails to provide a free page-frame to A. Boom. > >Later, you access one page of yours. It's not in RAM. The system has to > >fetch it from swap. First, it has to locate a free page-frame. There is none. > >So, it has to swap-out some other page. But, wait, swap space is full. > >What happens? It's your process causing the page fault. > > > >Even if your process has all swap space allocated, it's not enough. > >Overcommiting for other processes still hits you. > > You seem to misunderstand. Basically, malloc() allocates ADDRESS > SPACE, not memory. USING that address space then allocates VM (RAM or > swap). Problems only arise if you haven't really allocated your memory > yet, you only think you have (because you misunderstand the malloc() > implementation.)
That's what *I*'ve been saying in previous messages. I'm not misunderstanding malloc(). You are misunderstanding that your 'safe' process may still page-fault. Even if its own pages are on swap, you may need to free other processes pages, and you can't be sure *they* can be swapped out.
> If YOU touch memory when malloc()ed (i.e. you REALLY allocate memory, > not just address space to put memory in) you're fine. Only the other > processes can be hit.
No. Only if ALL processes touch ALL pages after malloc() you are (partially) safe. They also need NOT to grow their stacks, for example.
> > James. >
.TM. -- ____/ ____/ / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _____/ _____/ _/ Colombo@ESI.it
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |