Messages in this thread | | | From | John Ripley <> | Subject | Re: Recursive malloc crashing Linux. (Well almost) | Date | Thu, 16 Dec 1999 13:31:59 +0000 |
| |
On Thu, 16 Dec 1999, you wrote: > [John Ripley <john@empeg.com>] > > I've encountered this problem too, but with the additional > > restriction of having an environment with 8MB of RAM and no swap. > > I'm not sure I agree with the ability to allocate more memory than > > you have available. > > echo 0 > /proc/sys/vm/overcommit_memory > > grep -A10 overcommit /usr/src/linux/Documentation/proc.txt > > Peter
Let's change that slightly:
# cat /proc/sys/vm/overcommit_memory 0 # grep -A10 overcommit Documentation/proc.txt overcommit_memory This file contains one value. The following algorithm is used to decide if there's enough memory: if the value of overcommit_memory is positive, then there's always enough memory. This is a useful feature, since programs often malloc() huge amounts of memory 'just in case', while they only use a small part of it. Leaving this value at 0 will lead to the failure of such a huge malloc(), when in fact the system has enough memory for the program to run.
On the other hand, enabling this feature can cause you to run out of memory and thrash the system to death, so large and/or important servers will want to set this value to 0.
As you can see, overcommit defaults to off. Under such conditions, malloc is supposed to fail gracefully if enough memory cannot be allocated. However, you can quite happily allocate 1GB, so long as you allocate it in chunks of memory smaller than your physical limit. I've included a short demonstration program for this. On an empeg (8MB ram, no swap) this results in:
# ./killmem Allocating 256 amounts of 4MB == 1024MB Successful Filling block 0 Filling block 1 killmem: memory violation at pc=0x0200047c, lr=0x4005f7d8 (bad address=0x4065d000, code 1) Bus error
Which is kind of not very graceful. On my PC, 128MB ram, ~133MB swap, it gets as far as block 34 (136MB) before causes a complete lockup (not even magic sysrq works) - which is also not very graceful. The empeg runs 2.2.12, the PC runs 2.3.30. A solution I've thought of would be to have vm_enough_memory() (in mm/mmap.c) also take into account anonymous mmap pages that are yet to be paged, which requires another global count such as nr_pending_anon_pages.
Has anyone got any other solutions to this?
-- John Ripley, empeg Ltd. http://www.empeg.com #include <stdio.h> #include <unistd.h> #include <sys/mman.h>
#define NBUFS 256 #define MMAP_SIZE (4*1048576)
main() { int i, j; char *buf[NBUFS], *p;
printf("Allocating %d amounts of %dMB == %dMB\n", NBUFS, MMAP_SIZE >> 20, (MMAP_SIZE >> 20) * NBUFS); for(i=0; i<NBUFS; i++) { buf[i] = (char *) mmap(0, 4096*1024, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); if(buf[i] == MAP_FAILED) { printf("Failed for block %d\n", i); return 1; } }
printf("Successful\n"); for(i=0; i<NBUFS; i++) { printf("Filling block %d\n", i); p = buf[i]; for(j=MMAP_SIZE; j; j--) *p++ = 0; }
printf("Done\n"); }
| |