Messages in this thread | | | Date | Fri, 24 Mar 2000 22:15:36 -0800 (PST) | From | David Whysong <> | Subject | Re: Endless overcommit memory thread. |
| |
On Fri, 24 Mar 2000, Linda Walsh wrote:
>David Whysong wrote: >> >> Partly right. But the real reason for overcommit is practical: you can do >> more with overcommitted memory, and there are no additional failure modes. >> >> If we define "failure" as a situation where a user program must be killed >> because we are out of memory, then non-overcommit systems should always >> fail more often than overcommitted ones. So I don't see any downside, AT >> ALL, to memory overcommit. >--- > You have non-deterministic behavior.
Non-deterministic with the current kernels that randomly kill things, yes. I certainly don't like the situation. But "fixing" the problem by adding new system calls isn't a good solution -- you've redefined the problem such that all current software is broken, and needs to be rewritten to use your syscall.
A better solution is to impose sane, deterministic behavior in the overcommitted case. This can be done with optional memory quotas in conjunction with Rik van Riel's kernel patch. But removing overcommit doesn't solve anything.
> First, the kernel should reserve some amount of memory so it will >never run out of memory.
...and that's hard to do. AFAIK Linux reserves a fraction of memory for the kernel (256 pages on my machine), but doesn't guarantee anything beyond that.
>Ideally, there should be two limits. One level would require processes >have UID==0 (or some CAP - CAP_USE_RESERVE_SPACE) to alloc beyond, a >second the kernel reserves for itself. If all processes become blocked >on waiting for memory, the kernel starts killing user-level processes >with the largest first. Probably another CAP for CAP_DONT_KILL_FOR_MEM >to protect system processes executing in user space.
Killing from largest to smallest isn't a good idea. That often makes the X server go first. Have a look at Rik van Riel's OOM killer patch for a better example. I think that the policy of what process to kill should be configurable.
> For the fork case, a 2nd system call -- I propose 'sproc' -- only >because it's a model used to solve the overcommit problem. From the man page:
"Overcommit problem?" What problem?
[sproc = fork() with shared virtual address space but separate stack]
I don't see how this solves anything. We already have vfork(), unfortunately. And I'm not enough of a kernel hacker to see the difference between sproc() and vfork(). Heck, my manpages don't even describe the difference between fork() and vfork()...
> The point is that if you want trusted behavior, you want it to be >deterministic so the behavior is very predictable. I assert this is also >important to commercial institutions.
Again, this isn't very meaningful. Any non-deterministic behavior isn't a result of overcommitment, it's due to the fact that the kernel hasn't been informed of what to do when OOM. That can be fixed without removing memory overcommitment. Just implement quotas, or alternately task priorities and have the kernel kill the lowest priority tasks first. After all, by the time you start killing tasks on an overcommitted system, you would have been killing tasks long before without overcommit...
> Telling users: "um, yeah we guarantee auditing (C2 security), but a >user can possibly kill off auditing or other random system processes, >covering all their tracks because Linux is inherently unreliable and >non-deterministic" is REALLY bad.
The problem is not overcommit. The problem is that the system doesn't handle OOM well. It would be better to solve the problem than cover it up under some new system call.
Dave
David Whysong dwhysong@physics.ucsb.edu Astrophysics graduate student University of California, Santa Barbara My public PGP keys are on my web page - http://www.physics.ucsb.edu/~dwhysong DSS PGP Key 0x903F5BD6 : FE78 91FE 4508 106F 7C88 1706 B792 6995 903F 5BD6 D-H PGP key 0x5DAB0F91 : BC33 0F36 FCCD E72C 441F 663A 72ED 7FB7 5DAB 0F91
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |