[lkml]   [2009]   [Oct]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Memory overcommit
On Fri, 30 Oct 2009, Vedran Furac wrote:

> Well, you are kernel hacker, not me. You know how linux mm works much
> more than I do. I just reported a, what I think is a big problem, which
> needs to be solved ASAP (2.6.33).

The oom killer heuristics have not been changed recently, why is this
suddenly a problem that needs to be immediately addressed? The heuristics
you've been referring to have been used for at least three years.

> I'm afraid that we'll just talk much
> and nothing will be done with solution/fix postponed indefinitely. Not
> sure if you are interested, but I tested this on windowsxp also, and
> nothing bad happens there, system continues to function properly.

I'm totally sympathetic to testcases such as your own where the oom killer
seems to react in an undesirable way. I agree that it could do a much
better job at targeting "test" and killing it without negatively impacting
other tasks.

However, I don't think we can simply change the baseline (like the rss
change which has been added to -mm (??)) and consider it a major
improvement when it severely impacts how system administrators are able to
tune the badness heuristic from userspace via /proc/pid/oom_adj. I'm sure
you'd agree that user input is important in this matter and so that we
should maximize that ability rather than make it more difficult. That's
my main criticism of the suggestions thus far (and, sorry, but I have to
look out for production server interests here: you can't take away our
ability to influence oom badness scoring just because other simple
heuristics may be more understandable).

> > Much better is to allow the user to decide at what point, regardless of
> > swap usage, their application is using much more memory than expected or
> > required. They can do that right now pretty well with /proc/pid/oom_adj
> > without this outlandish claim that they should be expected to know the rss
> > of their applications at the time of oom to effectively tune oom_adj.
> Believe me, barely a few developers use oom_adj for their applications,
> and probably almost none of the end users. What should they do, every
> time they start an application, go to console and set the oom_adj. You
> cannot expect them to do that.

oom_adj is an extremely important part of our infrastructure and although
the majority of Linux users may not use it (I know a number of opensource
programs that tune its own, however), we can't let go of our ability to
specify an oom killing priority.

There are no simple solutions to this problem: the model proposed thus
far, which has basically been to acknowledge that oom killer is a bad
thing to encounter (but within that, some rationale was found that we can
react however we want??) and should be extremely easy to understand (just
kill the memory hogger with the most resident RAM) is a non-starter.

What would be better, and what I think we'll end up with, is a root
selectable heuristic so that production servers and desktop machines can
use different heuristics to make oom kill selections. We already have
/proc/sys/vm/oom_kill_allocating_task which I added 1-2 years ago to
address concerns specifically of SGI and their enormously long tasklist
scans. This would be variation on that idea and would include different
simplistic behaviors (such as always killing the most memory hogging task,
killing the most recently started task by the same uid, etc), and leave
the default heuristic much the same as currently.

 \ /
  Last update: 2009-10-30 20:47    [W:0.153 / U:3.160 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site