Messages in this thread | | | Subject | Re: Overcommitable memory?? | From | Olaf Weber <> | Date | 17 Mar 2000 20:07:56 +0100 |
| |
James Sutherland writes: > On 15 Mar 2000, Rask Ingemann Lambertsen wrote:
>> Not at all. COW is a performance optimisation which does not depend on >> overcommitment of memory in any way. Why would you want to turn it off?
> Because it *IS* overcommitment of memory. You can have two processes, each > with their 200Mb of data, in a machine with 256Mb RAM+swap, quite happily > - until they start writing to it, at which point you discover you have > overcommitted your memory, and things go wrong.
You're conflating two things: the COW optimization and whether or not virtual memory is actually reserved. For example, in a system that doesn't overcommit, suppose you have process that forks: at that point, the kernel reserves enough pages of virtual memory to be able to give the new process unique pages if it needs them. COW means that those reserved pages are only pressed into service when they are actually written to.
How many pages is enough? In the case of a fork, you only need to reserve pages for the writable pages of the old process. The read-only pages (the program text segment) can be shared (and have the binary as backing store to boot). On an exec, the kernel will of course reset the count of reserved pages to match the new executable. (And the exec could fail if it tries to start a new program that requires a larger data segment than available memory allows.)
The thing about fork/exec is that the requirement for extra virtual memory when a large process forks a small program (emacs forks ls) is short-lived.
Read-only data is not a problem, so apart from fork/exec, how many cases are there where you have processes sharing large numbers of writable pages? Note that for overcommitment to actually "work" in those cases, those pages should hardly ever be written to: if they are all touched in the long run, then you do really need the extra memory, and reserving it now will prevent nasty surprises later. And if the pages are de-facto read-only, would it not be better if the application marked them as such before forking?
I have some experience with the pros and cons of overcommitment on IRIX workstations, where you can specify how many pages the kernel is allowed to overcommit. When the system is stressed and overcommitment isn't allowed, the first sign is typically that you cannot print from netscape or something similarly irritating. When overcommitment is allowed, the first sign is processes dying at random, with the X server usually among the first to go. I don't overcommit at all.
If during normal work you get processes killed due to overcommitment, or unable to fork, exec, or malloc due to memory shortage, you need to either get more (virtual) memory or lessen the workload.
One thing that irks me about the current discussion is the complete lack of data: I would be interesting to know how much additional VM a sane non-overcommitting regime requires when compared with the overcommitting case? It seems no-one actually knows.
-- Olaf Weber
Do not meddle in the affairs of sysadmins, for they are quick to anger and have no need for subtlety.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |