lkml.org 
[lkml]   [2000]   [Mar]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Virtual vs. physical swap & shared memory forks (clone)
On Sat, 25 Mar 2000, Linda Walsh wrote:

>> The "malloc() not returning NULL" argument is complete nonsense. It's not
>> worth throwing away a useful feature (overcommit) with concrete benefits
>> over this.
>>
>> In fact, malloc() not returning NULL isn't a bug, it's a feature. It lets
>> you do things that you couldn't otherwise accomplish, for no cost.
>---
> The man page says malloc will return a NULL on failure. Define
>failure. Please explain what this failure could be if not 'insufficient
>memory'.

That depends on how you set /proc/sys/vm/overcommit_memory. If nonzero,
malloc never fails. If zero, malloc fails if you try to allocate more
memory than is currently free. This doesn't really disable overcommit
though.

This is perfectly reasonable behavior. The "hard limit" is the system
virtual memory.

[...]

[Rik van Riel's algorithm for determining which process to kill when OOM]

>> Then the process with the highest badness is killed.
>---
> That is a good heuristic if you want to simulate determinism. I
>used to come up with such formulae when I wanted to simulate "human" type
>behavior in a Dungeons and Dragons game I wrote. But I suspect it while there
>is a deterministic formula behind the actions, it would still appear
>non-deterministic to the average user.

That is why I want to add a system call to adjust the badness value for
each task. The sysadmin can decide what tasks are killed first.

[...]

>> When you run out of memory, overcommit or not, tasks are going to die.
>---
> No. Current behavior on IRIX is suspending the tasks asking for
>more memory resulting in possible deadlock if all processes are
>suspended -- unlikely but possible. That's a caveat w/using virtual
>memory. I'd support a system configuration ability to allow choice
>between suspend or kill behaviors.

I'd rather avoid the deadlock, it's not all _that_ unlikely (Murphy's law,
etc.), and the kernel should be robust. The kernel OOM killer is only
invoked in the extreme case where you are completely out of memory. It's
an emergency measure, and SIGKILL is appropriate.

Sending SIGSTOP before that point is reasonable, but why not do this in
user-space? My daemon (which is rapidly nearing useable status) will send
SIGSTOP to tasks if they haven't exceeded the hard limit, SIGTERM and
SIGKILL otherwise.

>> Removing overcommit might make malloc() return null, but that's only one
>> of a host of ways to allocate memory. The other methods don't have a
>> return value. So arguing that "overcommit is bad, because it breaks the
>> malloc() return value" is pointless.
>---
> What other methods? calloc - ENOMEM, open <object>, ENOMEM, fork:
>ENOMEM. Etc. All what you would expect if there was NOMEM.

Implicit methods. Think stack...

...and I don't buy the argument that stack growth can be limited by
careful programming. You have to consider the stack of ALL processes, not
just your own.

>> The problem here is that the system is out of memory. This has nothing to
>> do with overcommitment. The solution is to make the kernel more
>> intelligent about killing processes when OOM. Anything else just hides the
>> problem.
>---
> You are assumming I want a system that kills anything. Any kills
>other than explicit kills by a user of their processes is a denial of
>service. It means you have a system with low integrity/reliability. This
>is just plain unacceptable for mission critical systems.

If you want to avoid SIGKILL, then put more memory in the machine so it
can handle the load. But if you don't SIGKILL when OOM, you're inviting
disaster.

> For your system, since you aren't running mission critical stuff, you
>can choose to allocate 1G of Vmem, and implement 'killing' on the above
>formula. But the *simple* behavior should be the most predictable -- that
>of "no mem" resulting in ENOMEM on calls asking for memory.

Why bother, when there is a definate disadvantage go doing so, and your
process can run OOM without getting ENOMEM anyway?

Dave

David Whysong dwhysong@physics.ucsb.edu
Astrophysics graduate student University of California, Santa Barbara
My public PGP keys are on my web page - http://www.physics.ucsb.edu/~dwhysong
DSS PGP Key 0x903F5BD6 : FE78 91FE 4508 106F 7C88 1706 B792 6995 903F 5BD6
D-H PGP key 0x5DAB0F91 : BC33 0F36 FCCD E72C 441F 663A 72ED 7FB7 5DAB 0F91



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:57    [W:0.104 / U:1.724 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site