Messages in this thread | | | Date | Wed, 19 Feb 1997 14:43:18 -0500 | From | Brett Hollon (BHOLLON) <> | Subject | Memory overcommitting (was Re: http://www.redhat.com/redhat/) |
| |
Ted,
I can see a very serious problem with this. Just how can you tell the difference between a hardware failure, buffer overrun, ignored return value from malloc, (any other of a number of ways you can generate a seg fault through programming errors), and bumping into one of these over-committed memory areas?
It would seem prudent to at least track the amount of virtual memory that has been committed and not allow that figure to exceed the amount available (say the sum of the phys ram and swap space). In fact, I thought this is what was being done.
BSD does something similar to this (though not all that well) in that all memory allocations have their swap space allocated at request time. Any request for which swap space cannot be assigned is failed. This is efficient speed-wise, but very inneficient in terms of resources, as it does not allow for a system with less swap space than RAM to use all of it's RAM.
It would seem to me to be fairly simple and inexpensive to simply keep track of the current total commitment for each process, and a sum for the system, and fail any allocation that pushes the system into an overcommited state. This is not foolproof of course, eg if swap space is removed from the system, then you could end up overcommitted, but it seems to me that we would want a system that is running out of virtual memory to fail gracefully, by failing allocation requests, rather than having it fail in some other fashion, say by getting seg faults in processes that are accessing memory that has been allocated to them.
thanks, Brett
>Date: Tue, 18 Feb 1997 23:17:29 -0500 >From: "Theodore Y. Ts'o" <tytso@MIT.EDU>
> Date: Tue, 18 Feb 1997 14:27:44 -0700 (MST) > From: Jody Matsushima <matsushj@unidata.ucar.edu> > > > We are trying to do some software development on our machine > > that runs RedHat Linux (Intel - Pentium). One of the > > developers has asked me if there is any kind of fix for a > > problem with malloc. The problem seems to be that when you > > use the malloc command, you can allocate memory that is not > > available to allocate. Has anybody else reported this? If > > you need more information, I can get it to you.
>This is not a bug; it's a feature. Linux does not do hard allocation of >memory, because its not efficient. A very large amount of memory could >potentially be used is in fact never used. For example, when a process >fork()s, the data pages are not copied right away, but marked >copy-on-write, and only copied when they are modified. If they are >never modified, the extra data page is never needed. If however Linux >were to guarantee that every single memory access was always guaranteed >to succeed, it would have to reserve room for all of those data pages, >even if 99.99999% of the cases they would never be used.
>In general, this is very, very, very rarely a problem. It would perhaps >useful for you to comment on *why* you think this is a problem. What is >it specifically that you are trying to do?
> - Ted
| |