lkml.org 
[lkml]   [1998]   [Oct]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Out Of Memory in v. 2.1
On Tue, 6 Oct 1998, Andrea Arcangeli wrote:
> On Tue, 6 Oct 1998, Rik van Riel wrote:
>
> >It also introduces a new one: having kswapd stop when
> >try_to_free_page() fails. When try_to_free_page() fails,
>
> Tell me how can kswapd free memory if try_to_free_page() fail! If
> try_to_free_page() fails it means that we really have not enough
> memory and we _must_ stop.

You forget that try_to_free_page() doesn't scan all memory.
It also only ages memory if all memory it scanned was touched
or had a high age. This means that we might need multiple
scans in order to actually free something.

> >Your patch doesn't really fix the bug. It makes sure that
> >kswapd has complete failures more often, and this can be
> >considered a bad thing instead of something nice.
>
> For what I understand from the code there is a simple function
> called do_try_to_free_page(). This function when called must try to
> free a page. When this function fails it means that there isn' t a
> freeable page.

Nope, it just means that this function didn't encounter a
freeable page. There might be more memory in the machine
than what is scanned in one pass of try_to_free_page()...

It could also mean that all memory it saw had a page age of
> 0, so it couldn't be freed _yet_, this memory _can_ be
freed on the next pass --> problem solved, if you allow the
next pass, that is.

> Ok I can agree that we could wait two/three consecutive fail of
> do_try_to_free_page() (for increase the pressure, even if returning
> at the first fail seems to work just fine) before put kswapd to
> sleep but this doesn' t change my point.

We've done that before and we removed it from the kernel
around 2.1.8? because it didn't work. Please re-read the
archives for a brushup of factual knowledge...
(this mainly is for the innocent bystanders who are reading
this -- Andrea probably knows enough to not have to do this)

> >about detecting OOM yet. (the one problem is the legitimate
> >case where ram and swap are full but we have a lot of swap cache)
>
> If there' s a lot of swap cache you can run shrink_mmap() or follow
> the swap cache list and checking if the swap cache page->count is 1,
> in such case you can delete_from_swap_cache(page) (you can hack my
> shrink_swap_cache() to do this).

That's a good one, checking for the return value of shrink_mmap().
If we are in the situation I mentioned, shrink_mmap() should fail.

> >It is not always right to kill the current process. It could
> >be X or you two-week-old simulation that just needs a few
> >extra kilobytes do format it's OUTPUT -- bang, there go two
> >weeks of calculations...
>
> Agreed. But if the current process is not iopl I think we should
> kill the process that has generated the page fault when
> __get_free_pages() returned NULL. You simply need to change
> force_sig() in mm/*.c with a change_which_process_to_kill() to do
> that.

Why do you think we should kill the program that generated
the pagefault? I think that most people would like the
obvious offender to be killed, if only to minimize the number
of programs that have to be killed.

> >And all just because a newly started Netscape allocated all
> >but the last few kilobytes of memory. In that case, it might
> >be _far_ better to kill Netscape or something else that's both
> >new and large.
>
> No. Wrong. If netscape is swapped out and sleeping, killing Netscape
> would be a wrong choice. Eventually it could be tunable but sure
> killing Netscape if it' s sleeping it' s a wrong default I think.

If it is swapped out and sleeping, it will have an RSS of 0,
making it non-interesting for the OOM killer -- have you
actually read the code?

> The _only_ way an OOM killer could be useful is to avoid killing of
> X.

This is obviously _not_ true; a lot of sysadmins and other
folks have different opinions on this -- and it's their
machines and jobs we're talking about...

You have no way of tellling what other people think is
most important to them, especially not in the case of
sysadmins with 15+ years of Unix experience!

> >Killing the current process equals killing a random process;
> >most people don't like this, especially not when it is the X
> >server. This means that quite a lot of people do need an OOM
> >killer.
>
> For X OK.

For other programs OK as well. See above.

> >With your patch the system will run out of memory more often,
> >that is a very bad thing to happen.
>
> Can you reproduce a condition where without my patch 2.1.124 doesn' t
> deadlock and instead 2.1.124 + my patch fail?
>
> And Linux must go OOM when there' s no memory!

It does. With your patch it kills a random process, with
my patch it deadlocks for a few seconds and then it kills
a non-random process -- doing far less harm to the system
in the long run.

> >Killing the machine is easy with your patch: if it happens
> >to be X that needs to swap something in when the memory is
> >gone, X will be killed and the machine will be dead (at least
> >the console will).
>
> If you won' t able to understand yourself ask me and I' ll hack for
> you a svga application to put the card in graphics mode and forking
> many many times running my leak program. I' ll see if your OOM
> killer will be able to put the console in text mode again.

Svga programs have IOPL too, they won't be killed by
my OOM killer. Coding up a bad svga app doesn't really
count, since it has to run as root -- no good root
would install a program like that.

Besides, if it forked over and over, your patch would kill
it too and the system would be screwed as well, besides,
your system has _no_ fork-bomb protection, so how is it
going to help there?

> >My patch never kills the X server or any other IOPL process.
>
> I agree it' s a good thing to have a way to learn to the kernel to
> kill a process only as last resort, but note that iopl is not enough
> and we should have another way to detect the X server since XFreeFB
> should not run in iopl anymore. So I don' t think we should put an
> OOM killer in the stock 2.2 (we should just be in code freezee (so
> think at feature freezee...)).

I don't plan to push it in right now -- even though I
have a lot of sysadmins asking me too :)

For one thing, the patch isn't ready yet (your suggestion
about shrink_mmap() probably would make it ready) and then
there's the code freeze. Putting in something like this
will undoubtedly trigger requests from other people as
well.

I think I'll just provide it as a patch and put pointers
to it all over the place ;)

Rik.
+-------------------------------------------------------------------+
| Linux memory management tour guide. H.H.vanRiel@phys.uu.nl |
| Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ |
+-------------------------------------------------------------------+


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:44    [W:0.085 / U:0.060 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site