Messages in this thread Patch in this message | | | Date | Tue, 2 Aug 2005 17:21:31 +0200 (CEST) | From | Simon Derr <> | Subject | bug in __vm_enough_memory() |
| |
Hi all,
We have found what seems to be a small bug in __vm_enough_memory() when sysctl_overcommit_memory is set to OVERCOMMIT_NEVER.
When this bug occurs the systems fails to boot, with /sbin/init whining about fork() returning ENOMEM.
We hunted down the problem to this:
The deferred update mecanism used in vm_acct_memory(), on a SMP system, allows the vm_committed_space counter to have a negative value.
This should not be a problem since this counter is known to be inaccurate.
But in __vm_enough_memory() this counter is compared to the `allowed' variable, which is an unsigned long. This comparison is broken since it will consider the negative values of vm_committed_space to be huge positive values, resulting in a memory allocation failure.
A proposed fix is attached below.
Signed-off-by: Jean-Marc.Saffroy@ext.bull.net Signed-off-by: Simon.Derr@bull.net
Index: linux-2.6.12/mm/mmap.c =================================================================== --- linux-2.6.12.orig/mm/mmap.c 2005-08-02 15:45:30.000000000 +0200 +++ linux-2.6.12/mm/mmap.c 2005-08-02 16:28:48.289575957 +0200 @@ -144,7 +144,10 @@ int __vm_enough_memory(long pages, int c leave 3% of the size of this process for other processes */ allowed -= current->mm->total_vm / 32; - if (atomic_read(&vm_committed_space) < allowed) + /* cast `allowed' as a signed long because vm_committed_space + * sometimes has a negative value + */ + if (atomic_read(&vm_committed_space) < (long)allowed) return 0; vm_unacct_memory(pages); Index: linux-2.6.12/mm/nommu.c =================================================================== --- linux-2.6.12.orig/mm/nommu.c 2005-06-17 21:48:29.000000000 +0200 +++ linux-2.6.12/mm/nommu.c 2005-08-02 16:28:46.384302543 +0200 @@ -1167,7 +1167,10 @@ int __vm_enough_memory(long pages, int c leave 3% of the size of this process for other processes */ allowed -= current->mm->total_vm / 32; - if (atomic_read(&vm_committed_space) < allowed) + /* cast `allowed' as a signed long because vm_committed_space + * sometimes has a negative value + */ + if (atomic_read(&vm_committed_space) < (long)allowed) return 0; vm_unacct_memory(pages); - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |