Messages in this thread | | | From | "David Schwartz" <> | Subject | RE: A good reason to use vfork() | Date | Wed, 3 Nov 1999 15:31:13 -0800 |
| |
> David Schwartz writes: > > Okay, first, assume that 'vfork' on Linux is never going to do > anything but > > copy-on-write. > > *I* wouldn't assume that, ever. Copy-on-write vfork() is fork() with a > different name. _I'm not saying that is a bad thing._ It's just not > vfork(). Assuming vfork() is not just an alias for fork() is not wise. > Assuming vfork() *is* just an alias for fork() is just plain silly.
Well, if someone implements a true 'vfork' for Linux, then this argument is moot. Similarly, if overcommitting is on, this argument is moot. So there's only an issue if you make those assumptions.
> _The main point I've been trying to make_ is that having *both* overcommit > and no-overcommit behaviors available *simultaneously* is a GOOD THING.
I could not disagree more. If a person turns overcommitting off, he does so because he doesn't want processes killed randomly for reasons beyond his control.
> > Given these two assumptions, I believe it is disastrous to > allow 'vfork' to > > succeed if there aren't enough swap pages to back every single > copy-on-write > > page. > > So, with the second assumption "that overcommiting is turned > off", it would > be disastrous if overcommiting were turned on. I never would have thought > of that. :-)
Exactly. If overcommitting is off, the system should not let itself get into a case where dirtying a copy-on-write page requires it to kill a process.
> Since you have a firm grasp of why overcommiting can be bad, I'll give you > a real world example of why it can be good. In microelectronics, database > sizes -- in RAM -- frequently exceed 1GB. Sometimes, they even > exceed 4GB. > (I've personally had to build special versions of Perl for > processing these > monsters.) Because of the expense, hosts processing these databases are > only rarely configured with _more than twice the amount of RAM_ > it takes to > process one of of these large databases at a time. Programs processing > this data *frequently* run for more than 10 hours. Sometimes they run for > several *days*. If this process starts paging because of lack of RAM, it > can increase the run time by factors of 10. At the completion of > processing, these programs sometimes call external programs. Frequently, > external programs are a required part of printing the result. Sometimes, > external programs are a required part of saving the result to disk in a > useful format. Saving the result to disk in a useful format is usually > _the entire purpose_ of running the program in the first place.
This is why you should either:
1) Turn overcomitting on, or
2) Configure the machine with lots of swap.
> popen(), ENOMEM, bummer! (Similar applies to Perl in additional > contexts.)
Okay, that's bad. But if you turn overcommitting off and don't have enough swap, that's your decision.
> Now imagine yourself as a Developer or Sys Admin, explaining to a > room full > of engineers and project managers why a machine that costs more money than > you make in a year needs twice the huge amount of RAM it already has when > they can log in, run "top", and observe that the total amount of RAM used > never exceeds 90%, *hundreds* of MB RAM free, right up to the very end of > processing. You can bet some of the engineers already know the external > filter program never uses more than 12MB RAM, ever. Alternately, imagine > explaining why you need to allocate 8GB of swap on a particularly fast and > expensive disk -- when it will *never* be used -- by design. Imagine > explaining why the project schedule just slipped the better part > of a week.
I'd rather do that than explain why even though overcomitting is off, the system still overcomitted, and had to kill an innocent process.
> While plotting world domination:-), please consider the many engineers who > work in environments like these who would *love* to see Linux > play a larger > role.
Yes, that's one of the main reasons we have the ability to turn off overcommitting.
DS
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |