Messages in this thread | | | Date | Sat, 8 Jul 2000 03:17:15 -0400 (EDT) | From | "Mike A. Harris" <> | Subject | Re: Kernel 2.2.14 OOM killer strikes. |
| |
On Fri, 7 Jul 2000, Derek Martin wrote:
>> the OOM condition 'logically'. That is the problem I believe, is >> that the kernel needs to try and determine using logic which >> process is the REAL bad guy. Next to impossible... > >Yeah I get it now. Though based on my (admittedly limited) >understanding of the way the kernel currently decides to kill processes, >that doesn't seem like it's any _worse_ a scenario, just not any better. > >> I'm dreaming up a daemon written in C for speed that: >> >> 1) Reads a config file of legal possible swapfile locations and >> min/max sizes it can use, etc.. >> 2) Has configurable high/low watermarks to determine when to >> create new swap space. >> 3) Can monitor memory/swap usage on the fly every X number of >> seconds (configurable), and if swap usage hits a certain high >> water mark, the daemon can check disk space in the legal places >> set forth by the config file, determine which (if any) locations >> have enough space to create a swap file, and then creates a swap >> file or files as quickly as possible (with nice -20) - possibly >> even suspending other processes (configurable), then enables the >> new swap file to handle the situation. > >I can see why this might make sense in some cases, but I don't think it >really solves the problem. If you OOM because of a badly behaved program, >or a malicous user's program, you're still going to have the same problem, >and much less disk space.
Right. It doesn't solve the problem at all - I don't think anything can ever solve the problem - if you define the problem as being stopping bad apps from OOM'ing the system. It needs to be taken on a case by case basis - what is best for a given system under certain conditions.
In my case, a 'dynswap daemon' would give me a larger window of time in which to 'catch' an OOM situation and use human brain logic to kill processes. I could code the daemon to play a wav file when a swap file is added, another when another swap is added - increasing the 'defcon' number or alarmingness each time, to warn me trouble is near. In this case, I'd be not using up disk space at all, because it is free space unneeded by other things at the time - or I wouldn't configure it to use it, and once the problem was solved - either by me stopping the rogue process, and then the daemon downsizing swap accordingly, or by OOM occuring eventually anyway.. I'd be left just having to kill a swapfile off the disk. The time it buys me during interactive use, might be worth it for the times it works. The kernel's OOM killer just kicks in for backup. A couple hundred extra megs of swap, might even allow an app to complete what it was doing and not OOM. For example, if I had a dynswap daemon the other night, and had it set up that it could add up to 400Mb of swap, I wouldn't have OOM'd because midnight commander would have got the 150Mb it needed. I believe the app to be designed poorly in this case, rather than memleaking or buggy. The OOM wouldn't have occured, I could have grumbled, hit F10, and a few seconds to a minute later - dynswap would have grabbed me back my disk space.
The more I think about it, the more I like the idea. It could even make swap space on non ext2 volumes if necessary.
I'm going to explore if anyone else has coded up such a program.
>Based on what others have said in this thread, it sounds to me like the >right answer is for the kernel to do something like keep track of how fast >each process is allocating/using memory, and kill the highest users. And >it also sounds like that's in the works. So I'll shut up now... :)
Various solutions are definitely needed, so that each particular situation can choose the best-fit solution. In my case, I think both a dynswap daemon and a better OOM killer would be best. On a webserver or some other important machine, I'd likely side with the existing setup, or one of the other proposed solutions like bean counters, etc..
One thing is interesting is that a lot of people are working on ideas to solve this. That makes me happy for sure!
Take care, TTYL
-- Mike A. Harris Linux advocate Computer Consultant GNU advocate Capslock Consulting Open Source advocate
I've overclocked my keyboard interface. It's quite messy dipping my hands into the mineral oil, but *MAN* is my keyboard ever fast now! - Anonymous Coward
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |