lkml.org 
[lkml]   [2000]   [Mar]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Avoiding OOM on overcommit...?
---------  Received message begins Here  ---------

> From jas88@cam.ac.uk Fri Mar 24 08:51:35 2000
> Received: from jaguars.cableinet.net (jaguars-int.cableinet.net [193.38.113.9])
> by tomcat.admin.navo.hpc.mil (8.9.3/8.9.3) with SMTP id IAA67287
> for <pollard@tomcat.admin.navo.hpc.mil>; Fri, 24 Mar 2000 08:51:34 -0600 (CST)
> Received: (qmail 26854 invoked by uid 21); 24 Mar 2000 14:03:33 -0000
> Received: from unknown (HELO usr136-dun1.cableinet.co.uk) (213.48.60.113)
> by jaguars with SMTP; 24 Mar 2000 14:03:33 -0000
> From: James Sutherland <jas88@cam.ac.uk>
> To: Jesse Pollard <pollard@tomcat.admin.navo.hpc.mil>
> Cc: Jesse Pollard <pollard@tomcat.admin.navo.hpc.mil>,
> dwhysong@physics.ucsb.edu, Jesse Pollard <pollard@cats-chateau.net>,
> Jesse Pollard <pollard@tomcat.admin.navo.hpc.mil>,
> vonbrand@pincoya.inf.utfsm.cl, david parsons <orc@pell.portland.or.us>,
> linux-kernel@vger.rutgers.edu
> Subject: Re: Avoiding OOM on overcommit...?
> Date: Fri, 24 Mar 2000 14:53:50 +0000
> Message-ID: <440ndsgagdnem9dvqcmc871i57uq7bev2e@4ax.com>
> References: <200003221333.HAA34480@tomcat.admin.navo.hpc.mil>
> In-Reply-To: <200003221333.HAA34480@tomcat.admin.navo.hpc.mil>
> X-Mailer: Forte Agent 1.7/32.534
> MIME-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 8bit
> X-MIME-Autoconverted: from quoted-printable to 8bit by tomcat.admin.navo.hpc.mil id IAA67287
> Status: N
> Content-Length: 1612
> X-Lines: 49
> X-Display-Position: 0
>
> On Wed, 22 Mar 2000 07:33:12 -0600 (CST), you wrote:
> >James Sutherland <jas88@cam.ac.uk>
> (snip)
> >> Yours, if you don't have enough resources available to you to run it.
> >> Otherwise, both run fine.
> >
> >BUT "when I use up the resources given to me" - If the resources weren't
> >available, why did the system give them to me?
>
> It didn't.

It let me start the process. That confirmed that the resources were available.
I cannot cover race conditions between your process and mine. You may use
a sbrk call to get memory. So can I, the return from sbrk resumes my process
which fills memory with data. Your process starts to fill memory with data,
and gets aborted because the page you requested really didn't get to you.

>
> >> >Who gets killed - your process or mine?
> >> Yours, because there aren't enough resources to run it.
> >
> >The system told me there were enough resources.
>
> You asked it if there was 128Mb of VM free; there was. Half an hour
> later, you try to use 128Mb of VM and fail. There is a rather simpler
> explanation than the nasty kernel having lied to you...
>
> >> >Which is the correct one?
> >> Yours, as above.
> >As determined by what?
>
> System policy.

What policy. kernel gives you memory. kernel gives me memory. oops one
process didn't get memory.

>
> >> >How do you know it is the correct one?
> >> Because it would put you above the limit available to you.
> >
> >But the system told me the resources were available. And what limit?
> >The kernel doesn't support resource quotas.
>
> I'm pretending for the moment it does; if we can pretend it doesn't
> have overcommit, we can pretend it does have per-user rlimits, too.
> We have already agreed that it SHOULD have this feature, and
> eventually, it will. If you want it so badly, pay for it.

rlimits are a per process limit available to the user to adjust. quotas are
a per user limit specifing the maximum amount of a resource that the user
may use.

> >> >If it happens again, are the answers the same?
> >> Yes.
> >
> >BUGGGY. The system gave the resource to me. See above. What distinguishes
> >my job from yours?
>
> I loaded a program when there WERE enough resources to support it. You
> loaded the same program when there were not. I got the resources I was
> allowed, you got the resources available to you.

So did I.

You don't seem to recognize the context switching and the race
condition that can occur. If the resources were not available then my process
should not have been started.
ENOMEM.
Out of virtual space.
Could not allocate memory for process.
Could not allocate swap space.

It has been documented in several places that this condition exists and
is a problem. The more CPUs and processes there are the more likely the
failure. The race condition can occur even on single CPU systems, just
not as likely:

Process 1. process 2.
start process
Sbrk
start filling memory
context switch]
start process
sbrk
start filling memory
used up all resources (just enough for
the proces
computation
context switch
continue filling memory
OOM - kill process 1, if process 2 is lucky.
context switch
continue computation
Your ideal is:

Process 1. process 2.
start process
sbrk
start filling memory
used up resources
computation
context switch
start process
sbrk
start filling memory
OOM - kill process 2, if process 1 is lucky
continue computation

Either is possible, and I belive would occur fairly between the
two since the kernel doesn't appear to play favorites.
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:57    [W:0.040 / U:0.976 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site