Messages in this thread | | | From | "Matija Nalis" <> | Subject | Re: Overcommitable memory?? | Date | 22 Mar 2000 11:19:42 GMT |
| |
On 21 Mar 2000 11:29:16 +0100, Rask Ingemann Lambertsen <rask-linux@kampsax.k-net.dk> wrote: >Den 14-Mar-00 01:12:45 skrev Khimenko Victor fřlgende om "Re: Overcommitable memory??": >> In <3050.107T650T8914010rask@kampsax.k-net.dk> Rask Ingemann Lambertsen >> (rask@kampsax.k-net.dk) wrote: > >>> Without overcommit that just can not happen. There will be either a free >>> page of memory or a free page of swap into which you can swap something >>> else out. > >> No. There WILL NOT be anything available in worst-case scenario. > > Yes there will. Once a program has allocated its memory, it is always >available as long as you don't overcommit memory.
Yes, but it impose too much overhead. It means no COW (or preallocate on COW) It means no growable stack (so, what, you just allocate maximum amount ? 8 MB ? 32 MB ? 256 MB ? 1 G? PER PROCESS!)
It means that happy little server that has been running for years and serving WWW pages and doing FTP with 128MB of VM, will suddently need more gigs of swap than it than it has complete disk space). [Yes, you will say that disk space is cheap... It depends where are you living.]
> Perhaps we are simply thinking of two different worst-case scenarios? The >one I'm thinking of is the case where applications use all the memory they >have allocated. A kernel which doesn't overcommit memory needs to do nothing >to handle that situation.
Again, OOM has nothing to do with overcommit. It has to do with allocating memory. Only thing non-overcommit gives you is the fact that you will run out of memory faster.
Imagine: 100 apache servers running inetd syslog
www connections are coming happily, and then one FTP connection comes. inetd spawns ftpd, and few more apaches fork(). ftpd tries to log connection to syslog, and syslog fails to malloc().
System is non-overcommiting. You have OOM. Only question is who do you kill (and it is the same question as with overcommiting system!!): syslog, which caused OOM ? ftpd which made it die? inetd ? few of the apaches ? How you determine the victim ? (in the meantime, TCP connections continue coming in, and now kernels wants to allocate some more networking buffers, but is OOM)
Overcommiting or not, there is no easy solution here.
Probably it would be best to have syscall to define 'importance' of process when (not if) OOM condition stikes (something like nice(2), users may change one range of values, root can change more). So when we hit OOM, we kill biggest process with lowest 'importance'.
So you could have init(8) and sshd(8) running at -20 (NEVER KILL), apache parent and inet(8) at -10, apache children at 5, and user processes at default 0. Or whatever you decide to do. For starters, you could just add quick hardcoded syscalls to apache, inet, sshd and few others (no more than one afternoon of work) and later, those could turn to be proper directives for apache etc.
I do not believe that any system will develop AI good enough to read administrators mind and set those policies itself any time soon.
Quick hack could just go ahead without using new syscall and use existing nice(2) value for example :-)
Note that I do think that per user VM quotas are good idea, but IN ADDITION to idea like above, not instead of it.
For example, imagine machine with 1000 users, and some of them regularry use up to 100MB of VM - however there are rearly more than 50 at the time (due to some other factors), and most of them are below 10MB of VM. With only per-user limits, you would be forced to either allocate 100 GB of VM (for 100MB per user) or separate users in groups depending how much VM they usually use, and then allocate VM per-group. both are overkill, because such situations where that would be needed are extremely rare (if not impossible)
-- Opinions above are GNU-copylefted.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |