lkml.org 
[lkml]   [2000]   [Mar]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: Avoiding OOM on overcommit...?
Date
"A month of sundays ago David Whysong wrote:"
> On Mon, 27 Mar 2000, Peter T. Breuer wrote:
> >"A month of sundays ago David Whysong wrote:"
> >>[ptb wrote]
> >> >And I arrived later too. But while we're on the subject of swap space,
> >> >doesn't "reserve me 8MB of disk-based swap as backing for my stack" cure
> >> >everyones OOM blues? I propose that that's a fair use for swap nowadays.
> >>
> >> I don't know; what happens now if you use more than 8 MB of stack? Do "bad

> >I am suggesting only that programs/systems which need guarantees start up
> >with real swap space backing processes stacks.
>
> Ok, so you aren't doing away with overcommit entirely, just trying to
> guarantee memory for one process. Makes sense... but the same old problem

Or for all processes, if the system is configured appropriately.

> still exists if some other task runs the system out of memory. And since

You will never be out of stack space this way. You can be out of
mallocable memory and that should be handled by the programmer assuming
that malloc can return NULL .. given that we are ssuming the programmer
calls gnu_malloc instead of linux_malloc, or whetever.

What I am proposing is that processes bind to portions of swap space to
provide guaranteed backing _for them_. This is not a shared resource.
Nor is it a resource that will normally be used. It is (1) a place for them to rest
their stack when some other processes demands cause their stack to be swapped
out of ram; (2) a place for them to get more virtual memory for the stack from.
(1) protects "other processes". (2) protects them, although you won't like the
horrendously slow result. I suggest that only 4K of the backing store be paged
in at a time. You still have to reserve 4K of swap for the kernel as a whole,
of course!

> the stack still dynamically grows, you have to make sure that you never
> use more than the preallocated backing store.
>
> Is that right?

Yes. But there is nothing to make sure of. I propose that the process
cannot start unless it has the backing store for its rlimit available -
and that that store will be bound to it by the kernel for its and it
alones future use until death do it part.

I.e. non-shared swap, rather than non-shared ram.

> >Yes it is. The touch will cause a segfault if there is no memory.
> >Since this happens in your malloc wrapper, you can return 0 and handle
> >that in the normal program. (ahem .. you better already had the stack
> >required for the segv handler in the wrapper).
>
> Ah, the light dawns! This makes sense, and seems like a pretty good
> solution to the problem. You can reserve memory for your program, but not
> at the expense of the rest of the system!
>
> FINALLY someone comes up with a solution to the malloc() return value

This is not my new solution. It is the old, traditional one. I believe
it was issued as gnu_malloc originally.

> issue. And it's one that I can live with -- and entirely in user-space,
> even! :-) And you don't force all tasks to use it.
>
> It's not enough to make Linda Walsh happy, but I think it's pretty good.

Linda wants unlimited memory.

> Not that I would use it myself, I'm pretty happy with current behavior
> (with Rik's patch) and "echo 1 > /proc/sys/vm/overcommit_memory". But I
> seem to be in the minority...
>
> >You get a segfault at the time of the malloc call, instead of later.
> >The wrapper will handle it. Ummm .. you can guarantee stack space for
> >the handler by using longjmp just before you call malloc in the
> >wrapper I vaguely see.
>
> I'll take your word for it.

There are several mechanisms available that will guarrantee space for
the handler.

> >> >Fork overhead is dealt with by having a reserve for the kernel.
> >>
> >> Seems like a kludge to me.
> >
> >The latter is, but I believe that's the way things are now.
> >
> >Requiring swap backing for a processes stack is not a kludge.
>
> Yes, the kludge comment was referring to the reserve for fork.

Peter

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:57    [W:0.296 / U:0.108 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site