lkml.org 
[lkml]   [2000]   [Mar]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Endless overcommit memory thread.
On Fri, 24 Mar 2000, Linda Walsh wrote:

>David Whysong wrote:
>>
>> Partly right. But the real reason for overcommit is practical: you can do
>> more with overcommitted memory, and there are no additional failure modes.
>>
>> If we define "failure" as a situation where a user program must be killed
>> because we are out of memory, then non-overcommit systems should always
>> fail more often than overcommitted ones. So I don't see any downside, AT
>> ALL, to memory overcommit.
>---
> You have non-deterministic behavior.

Non-deterministic with the current kernels that randomly kill things, yes.
I certainly don't like the situation. But "fixing" the problem by adding
new system calls isn't a good solution -- you've redefined the problem
such that all current software is broken, and needs to be rewritten to use
your syscall.

A better solution is to impose sane, deterministic behavior in the
overcommitted case. This can be done with optional memory quotas in
conjunction with Rik van Riel's kernel patch. But removing overcommit
doesn't solve anything.

> First, the kernel should reserve some amount of memory so it will
>never run out of memory.

...and that's hard to do. AFAIK Linux reserves a fraction of memory for
the kernel (256 pages on my machine), but doesn't guarantee anything
beyond that.

>Ideally, there should be two limits. One level would require processes
>have UID==0 (or some CAP - CAP_USE_RESERVE_SPACE) to alloc beyond, a
>second the kernel reserves for itself. If all processes become blocked
>on waiting for memory, the kernel starts killing user-level processes
>with the largest first. Probably another CAP for CAP_DONT_KILL_FOR_MEM
>to protect system processes executing in user space.

Killing from largest to smallest isn't a good idea. That often makes the X
server go first. Have a look at Rik van Riel's OOM killer patch for a
better example. I think that the policy of what process to kill should be
configurable.

> For the fork case, a 2nd system call -- I propose 'sproc' -- only
>because it's a model used to solve the overcommit problem. From the man page:

"Overcommit problem?" What problem?

[sproc = fork() with shared virtual address space but separate stack]

I don't see how this solves anything. We already have vfork(),
unfortunately. And I'm not enough of a kernel hacker to see the difference
between sproc() and vfork(). Heck, my manpages don't even describe the
difference between fork() and vfork()...

> The point is that if you want trusted behavior, you want it to be
>deterministic so the behavior is very predictable. I assert this is also
>important to commercial institutions.

Again, this isn't very meaningful. Any non-deterministic behavior isn't a
result of overcommitment, it's due to the fact that the kernel hasn't been
informed of what to do when OOM. That can be fixed without removing memory
overcommitment. Just implement quotas, or alternately task priorities and
have the kernel kill the lowest priority tasks first. After all, by the
time you start killing tasks on an overcommitted system, you would have
been killing tasks long before without overcommit...

> Telling users: "um, yeah we guarantee auditing (C2 security), but a
>user can possibly kill off auditing or other random system processes,
>covering all their tracks because Linux is inherently unreliable and
>non-deterministic" is REALLY bad.

The problem is not overcommit. The problem is that the system doesn't
handle OOM well. It would be better to solve the problem than cover it up
under some new system call.

Dave

David Whysong dwhysong@physics.ucsb.edu
Astrophysics graduate student University of California, Santa Barbara
My public PGP keys are on my web page - http://www.physics.ucsb.edu/~dwhysong
DSS PGP Key 0x903F5BD6 : FE78 91FE 4508 106F 7C88 1706 B792 6995 903F 5BD6
D-H PGP key 0x5DAB0F91 : BC33 0F36 FCCD E72C 441F 663A 72ED 7FB7 5DAB 0F91




-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:57    [W:0.086 / U:0.192 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site