lkml.org 
[lkml]   [2000]   [Mar]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: Overcommitable memory??
Date
On 21 Mar 2000 11:29:16 +0100, Rask Ingemann Lambertsen <rask-linux@kampsax.k-net.dk> wrote:
>Den 14-Mar-00 01:12:45 skrev Khimenko Victor fřlgende om "Re: Overcommitable memory??":
>> In <3050.107T650T8914010rask@kampsax.k-net.dk> Rask Ingemann Lambertsen
>> (rask@kampsax.k-net.dk) wrote:
>
>>> Without overcommit that just can not happen. There will be either a free
>>> page of memory or a free page of swap into which you can swap something
>>> else out.
>
>> No. There WILL NOT be anything available in worst-case scenario.
>
> Yes there will. Once a program has allocated its memory, it is always
>available as long as you don't overcommit memory.

Yes, but it impose too much overhead.
It means no COW (or preallocate on COW)
It means no growable stack (so, what, you just allocate maximum amount ? 8
MB ? 32 MB ? 256 MB ? 1 G? PER PROCESS!)

It means that happy little server that has been running for years and
serving WWW pages and doing FTP with 128MB of VM, will suddently need more
gigs of swap than it than it has complete disk space). [Yes, you will say that
disk space is cheap... It depends where are you living.]


> Perhaps we are simply thinking of two different worst-case scenarios? The
>one I'm thinking of is the case where applications use all the memory they
>have allocated. A kernel which doesn't overcommit memory needs to do nothing
>to handle that situation.

Again, OOM has nothing to do with overcommit. It has to do with allocating
memory. Only thing non-overcommit gives you is the fact that you will run
out of memory faster.

Imagine:
100 apache servers running
inetd
syslog

www connections are coming happily, and then one FTP connection comes. inetd
spawns ftpd, and few more apaches fork(). ftpd tries to log connection to
syslog, and syslog fails to malloc().

System is non-overcommiting. You have OOM. Only question is who do you kill
(and it is the same question as with overcommiting system!!): syslog, which
caused OOM ? ftpd which made it die? inetd ? few of the apaches ? How you
determine the victim ? (in the meantime, TCP connections continue coming in,
and now kernels wants to allocate some more networking buffers, but is OOM)

Overcommiting or not, there is no easy solution here.

Probably it would be best to have syscall to define 'importance' of process
when (not if) OOM condition stikes (something like nice(2), users may change
one range of values, root can change more). So when we hit OOM, we kill
biggest process with lowest 'importance'.

So you could have init(8) and sshd(8) running at -20 (NEVER
KILL), apache parent and inet(8) at -10, apache children at 5, and user
processes at default 0. Or whatever you decide to do. For starters, you
could just add quick hardcoded syscalls to apache, inet, sshd and few others
(no more than one afternoon of work) and later, those could turn to be
proper directives for apache etc.

I do not believe that any system will develop AI good enough to read
administrators mind and set those policies itself any time soon.

Quick hack could just go ahead without using new syscall and use existing
nice(2) value for example :-)

Note that I do think that per user VM quotas are good idea, but IN ADDITION
to idea like above, not instead of it.

For example, imagine machine with 1000 users, and some of them regularry use
up to 100MB of VM - however there are rearly more than 50 at the time (due
to some other factors), and most of them are below 10MB of VM. With only
per-user limits, you would be forced to either allocate 100 GB of VM (for
100MB per user) or separate users in groups depending how much VM they
usually use, and then allocate VM per-group. both are overkill, because such
situations where that would be needed are extremely rare (if not impossible)


--
Opinions above are GNU-copylefted.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:57    [W:0.026 / U:1.256 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site