Messages in this thread |  | | | From | "Khimenko Victor" <> | | Date | Thu, 23 Mar 2000 14:03:37 +0300 (MSK) | | Subject | Re: Overcommitable memory?? |
| |
In <15AD5D7936F8D21198240000F831776E0390394D-100000@dboexc1.dbo.cpqcorp.net> Paul Jakma (paul.jakma@compaq.com) wrote: > On Thu, 16 Mar 2000, James Sutherland wrote: >> On Wed, 15 Mar 2000, Paul Jakma wrote:
>> That doesn't happen. malloc() ALLOCATES the memory to the process. It is >> *NOT* overcommitted. It may be backed by swapspace rather than physical >> memory, but that block of memory *IS* available to the process. >>
> so malloc() isn't overcommited? malloc()'ed memory is guaranteed to be > available - ie the memory is reserved and accounted for at malloc() > time?
He just UTTERY clueless. malloc() WILL happily overcommit HUGE amount of memory. You'll need more then one process though (see sample below).
> (i always thought malloc()'s were noted by the kernel ie in the page > table for the process, but that the kernel didn't actually allocate a > real page until the process tried to access that page - ie malloc() is > overcommited)
Exactly. Even more: if you'll READ from said page kernel will NOT allocate it for you ! It'll count this page as RSS though.
> If so, how do we get into trouble?
>> malloc() does this. fork() doesn't, because there is no memory to >> allocate.
> i must be misunderstanding you, cause I'm pretty sure fork() is supposed > to copy the complete VM of the parent. :) And on older Unix's like > ultrix you really had to have lot's of swap if you wanted to allow big > processes to do a fork().
Exactly.
>> The whole point of fork() is that you are *NOT* simply >> duplicating the in-VM image of the process!
> no, the whole point of overcommiting is so that we don't have to copy > the VM of a process at fork() time, eg COW. But fork() does specify that > the child process will have a complete copy of the parents VM.
Once again. fork() supposed to do COMPLETE copy of process. And COW is just to speed up things, not to change semantic.
> (whether we copy it at fork() time like really ancient os's, COW with > reserved backing store like ultrix or COW overcommit like linux is an > implementation issue).
>> > but that only applies to processes that try malloc() at the point of >> > OOM. You still have a bunch of processes with memory they have already >> > malloc()'ed but havn't allocated yet. >> >> Nope. >>
> really really sure about this? cause it goes completely against my > understanding of how linux works. but i'd be very glad to be corrected > if i'm wrong. I've honestly thought that malloc() was overcommited up > till now.
Oh. Instead of arguing in vain just make simple check. Take small program: -- cut -- #include <stdio.h>
int main(int argc,char *argv[]) { /* Allocate 100MiB of memory */ char *p=(char *)malloc(100*1024*1024); /* Not just read area to make sure compiler will not optimize anything */ /* int x; for (x=0;x<100*1024*1024;x++) { *p+=p[x]; } */ /* wait for keypress */ getchar(); } -- cut -- Start 10 copies with following command -- cut -- for i in 0 1 2 3 4 5 6 7 8 9 ; do { ./test $i & } ; done -- cut -- And the type `ps l'. Here I got the following result: -- cut -- [khim@localhost khim]$ ps l FLAGS UID PID PPID PRI NI SIZE RSS WCHAN STA TTY TIME COMMAND 100 503 460 459 15 0 2152 1268 wait4 S vc/1 0:00 -bash 0 503 485 460 3 0 103404 324 do_signal T vc/1 0:00 ./test 0 0 503 486 460 4 0 103404 324 do_signal T vc/1 0:00 ./test 1 0 503 487 460 5 0 103404 324 do_signal T vc/1 0:00 ./test 2 0 503 488 460 5 0 103404 324 do_signal T vc/1 0:00 ./test 3 0 503 489 460 5 0 103404 324 do_signal T vc/1 0:00 ./test 4 0 503 490 460 8 0 103404 324 do_signal T vc/1 0:00 ./test 5 0 503 491 460 9 0 103404 324 do_signal T vc/1 0:00 ./test 6 0 503 492 460 9 0 103404 324 do_signal T vc/1 0:00 ./test 7 0 503 493 460 10 0 103404 324 do_signal T vc/1 0:00 ./test 8 0 503 494 460 10 0 103404 324 do_signal T vc/1 0:00 ./test 9 100000 503 495 460 16 0 1452 568 R vc/1 0:00 ps -l -- cut -- and free says that: -- cut -- total used free shared buffers cached Mem: 126208 58360 67848 0 3908 38688 -/+ buffers/cache: 15764 110444 Swap: 0 0 0 -- cut -- Do you really think my non-existing swap can keep all 1000MiB of allocated memory ? If you'll add for(...) from comment to program you'll see that ALL TEN processes will have RSS of 50-60MiB at some time (while they still running not stopped that is) ! Total of 500-600MiB of RSS while there are only 128MiB RAM available (swap is completely disabled for test). If it's not overcommiting then what IS overcommiting ???
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
|  |