lkml.org 
[lkml]   [2000]   [Mar]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectVirtual vs. physical swap & shared memory forks (sprocX)
David Whysong wrote:
> Non-deterministic with the current kernels that randomly kill things, yes.
> I certainly don't like the situation. But "fixing" the problem by adding
> new system calls isn't a good solution -- you've redefined the problem
> such that all current software is broken, and needs to be rewritten to use
> your syscall.
---
All? That's a pretty strong statement. Programs that currently
behave will continue to run. Programs that spawn off hundreds of 40 megabyte
processes are being careless. They are relying on the non-determistic
operating system not enforcing memory restrictions.

An additional facility -- at the administrator's option they can
add "vswap" (also from IRIX) "Virtual Swap space":

A file that the system consideres to be a certain size (e.g. 400MB)
but actually occupies no disk space. This is useful because many programs
request much more swap space than they really need in order to run, and tie up
the real swap space unnecessarily. When you add virtual swap space, the system
lets you start applications even when they request more swap space than is
actually available. In most cases this is fine, because there is enough real
swap space for them to run.

This way you can allow overcommitment if you run applications that
need it, but mission-critical applications can be run on a system with
only 'physical' swap.

This would require no reprogramming of bad apps, but an admin
would have to explicitly enable some amount of virtual swap space.

The "-v" option for swap (IRIX) says:

-v vlength
Set the virtual length of the swap area to vlength. Normally, if this field
is not specified, the amount of logical swap is increased by the size of the
physical swap device being added. This option tells the system to increase
the logical swap amount by vlength. Thus, the difference between vlength
and the actual size of the device is amount of virtual swap that is added.
The virtual length must be greater than or equal to the physical length of
the swap area. If not specified, the virtual length is set equal to the actual
length. See swapctl(2) for a discussion on virtual swap spaces. In
general, this option should only be used when there is no other way to get
enough swap resources (even via NFS) and it is understood the potential
problems creating virtual swap can cause. See the discussion of Logical
Swap Space below.
...
Virtual swap discussion:

Programs that have large address spaces and large programs that fork, may
receive EAGAIN along with the "out of logical swap space" message on the
console. This can also happen when debugging a large program with dbx or
other debugger. There are two ways to avoid this error: adding more real swap
space, or adding virtual swap space. Adding real swap space means allocating
an additional disk partition or a regular file (either local or remote via NFS) to
be used as a swap device (using the -a option shown above and the examples
below). This is the required approach for programs that use most of the virtual
addresses they allocate. The advantage of this approach is that it continues to
avoid memory deadlocks, but requires physical disk space to be allocated.

The alternative is to add virtual swap space using the -v option. This increases
the amount of logical swap space without using any physical disk space. This is
suitable when the programs involved do not intend to use the virtual address
space they allocate (i.e., when the address space is sparse or when a large
program that forks intends to exec soon afterwards without modifying many
pages). In these cases, physical swap space is not required and so adding virtual
swap space allows the kernel to complete the logical swap space reservation
and avoid the EAGAIN errors. The advantage of this approach is that it does
not require any disk space, but adds the risk of encountering a memory
deadlock. Memory deadlocks occur when the system has over-committed
logical swap space by allowing the total private virtual space of the processes
on the system to exceed real swap space. When processes attempt to use the
allocated virtual space, the kernel has no place to store the data (since virtual
swap space has no associated disk space), and a memory deadlock results. In
these instances, the kernel kills one or more processes to free up enough logical
swap space to break the deadlock. For this reason, virtual swap space should
not be used in cases where the program will attempt to use the memory. For
example, programs that expect malloc(3C) to return NULL when there is no
more memory will in fact be allocated virtual memory that they could not use
without causing a memory deadlock. The -v option should therefore be used
with care.


> A better solution is to impose sane, deterministic behavior in the
> overcommitted case. This can be done with optional memory quotas in
> conjunction with Rik van Riel's kernel patch. But removing overcommit
> doesn't solve anything.
---
Sure it does. If you run out of memory, then 'malloc' will return
NULL. (Yeah, I'm changing my story on the fly -- default return
failure unless vswap is used....then we can have the above)

>
> > First, the kernel should reserve some amount of memory so it will
> >never run out of memory.
>
> ...and that's hard to do. AFAIK Linux reserves a fraction of memory for
> the kernel (256 pages on my machine), but doesn't guarantee anything
> beyond that.
---
So it dynamically reserves 256 more pages than it is currently using,
so it will be 256 pages away from being out of memory when it realizes there
is a problem? So kernel running out of memory shouldn't ever happen -- as
it should always have a 256-page buffer more than it is currently using.

>
> >Ideally, there should be two limits. One level would require processes
> >have UID==0 (or some CAP - CAP_USE_RESERVE_SPACE) to alloc beyond, a
> >second the kernel reserves for itself. If all processes become blocked
> >on waiting for memory, the kernel starts killing user-level processes
> >with the largest first. Probably another CAP for CAP_DONT_KILL_FOR_MEM
> >to protect system processes executing in user space.
---
Note, I'm serious about the above CAP's. Again -- if you want to
protect your 'X', you can make sure it runs with the "don't kill cap".

>
> Killing from largest to smallest isn't a good idea. That often makes the X
> server go first. Have a look at Rik van Riel's OOM killer patch for a
> better example. I think that the policy of what process to kill should be
> configurable.
---
Can you describe it's behavior? I don't happen to have a copy, but
if it's a good algorithm, it should be deterministic and well documented.


> I don't see how this solves anything. We already have vfork(),
---
Vfork? You mean this one (linux manpage):

BUGS
Under Linux, vfork is merely an alias for fork.

---
That's not very useful.


> unfortunately. And I'm not enough of a kernel hacker to see the difference
> between sproc() and vfork(). Heck, my manpages don't even describe the
> difference between fork() and vfork()...
---
The one's included with SuSE, RedHat state the above. Mandrake's states:
BUGS
It is rather unfortunate that Linux revived this spectre
from the past. The BSD manpage states: "This system call
will be eliminated when proper system sharing mechanisms
are implemented. Users should not depend on the memory
sharing semantics of vfork as it will, in that case, be
made synonymous to fork."

Formally speaking, the standard description given above
does not allow one to use vfork() since a following exec
might fail, and then what happens is undefined.


> Again, this isn't very meaningful. Any non-deterministic behavior isn't a
> result of overcommitment, it's due to the fact that the kernel hasn't been
> informed of what to do when OOM. That can be fixed without removing memory
> overcommitment. Just implement quotas, or alternately task priorities and
> have the kernel kill the lowest priority tasks first. After all, by the
> time you start killing tasks on an overcommitted system, you would have
> been killing tasks long before without overcommit...
---
No -- you'd return failures on malloc (assuming no virtual swap).
Not random or senseless killing. Then each app can choose what to do when
it runs into an out of memory condition instead of expecting that the sysadmin
will know the correct behavior for every app running on the system.


> The problem is not overcommit. The problem is that the system doesn't
> handle OOM well. It would be better to solve the problem than cover it up
> under some new system call.
---
The system would handle it just fine if you returned NULL on mallocs
or ENOMEMs/EAGAINs on forks. So what would you want 1) when I'm in vi, I
attempt to spawn a shell it returns "insufficient memory", or 2) the system
starts deciding by some sysadmin set policy about what to kill first.

The user of a such a system wouldn't know what to expect.

-l

--
Linda A Walsh | Trust Technology, Core Linux, SGI
law@sgi.com | Voice: (650) 933-5338

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:57    [W:0.089 / U:0.092 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site