Messages in this thread |  | | | Date | Fri, 24 Mar 2000 13:16:02 +0100 (CET) | | From | Marco Colombo <> | | Subject | Re: Avoiding OOM on overcommit...? |
| |
On 22 Mar 2000, Olaf Weber wrote:
> Marco Colombo writes: > > On 21 Mar 2000, Olaf Weber wrote: > > >> On ther other hand, most programs do not use sbrk(2) to obtain memory, > >> they use malloc(3). The difference is that (following the C standard) > >> there is a guarantee that if malloc succeeds, then the memory can be > >> used. On a system that unconditionally overcommits, this guarantee > >> doesn't hold. > > > Does the standard say something about paging? (I don't know, but i think no) > > Paging is transparent, until you *time* what you're doing. Then you'll > > see what the difference is between RAM and disk access time. > > Etc. I think you're just being silly here, and know it.
Yes, partly i am. But you are used to paging, you may think it is natural that when you need real RAM, you have to ask for it. To non Unix programmers, this is weird. Just like having you application "sleep" with a spin loop, which is a common programming tecnique in non timesharing systems. It is not bad *C language* programming, it's bad *UNIX* programming.
The OS may always signal you that something happened: the user of your controlling terminal just exited (hangup), your program has been requested to terminate, we're OOM, etc. That's OS semantic. It's outside the scope of the C standard.
You may say the Linux kernel should not kill random process, and i may agree. Don't say malloc() behaviour is not "standard". The C language does not say anything about when or why a process should get a signal by the OS. The C language, BTW, has no notion of what a process is.
> > malloc() "allocates" memory only on its own opaque pool. It has no > > OS meaning. > > Correct. But the C standard says that it has certain properties (and > implies some others). Basically, if I write a program and get reports
Pardon me, a standard either "says" something, or not. You may imply something while reading it.
> that it dies with a SIGBUS in some circumstances, I go on a bug-hunt > for the use of misaligned addresses and such. I would be very unhappy
That's OS (UNIX) semantic. What exacly SIGBUS, SIGSEGV, and the like mean is up to the kernel. You should know you may get a signal under OOM. So, before bug-hunting, just check if the system was OOM.
> to discover that I've been looking at a memory shortage masquerading > as a program bug. (I do test the return of malloc.) If the signal
Why do you call receiving a signal a "program bug"? A program may receive SIGHUP, SIGINT, SIGTERM, and many others, under normal (usually external) conditions. OOM is an "external" condition. You may suggest that we need a new signal, SIGABOUTTOOOM, but this is OS matter.
> sent had been (say) SIGNOMEM instead then I'd at least be spared that > particular incovenience.
you need better support from the OS, then. Adherence to C standard is another matter.
> > On systems with no protected memory (yes, you compile C programs on > > them) you can access every address you like. No need to malloc. You > > may write you own heap management routine. libc provides one. On > > system with memory protection, you can still write your own heap > > management routine, which uses whatever system call to do its jobs. > > malloc is a "standard" way to manage a heap. > > If you want to write even moderately portable code, then you haven't > got many options besides trusting malloc. More sophisticated methods > of obtaining memory tend to differ between systems, and even between > different revisions of a system. Writing your own memory management > routines will (in portable code) typically be done _on top of_ malloc > rather than in stead of it. > > Properly coping with memory that may not be there when it is used > requires special programming techniques. I'm not unwilling to use > such techniques, when appropriate, but not in all cases.
No problem. Is is problem that your program gets stoppen when it gets SIGTSTP? If no, you don't have to use any "special programming techniques". If yes, you may think of trapping the signal and inform the user that stopping your application is not possible. Are you concerned of your program being killed at OOM time? Use special techniques to avoid triggering the OOM killer (i.e., do not page-fault). It's harder than just using signal(), i admit it. It may also be impossible to do in a really "safe" way (your stack may grow...). So we need a better kernel API to handle such cases. Again, not a C standard problem.
> [...] > > >> In any case, it seems people are now mostly shooting the same > >> arguments at each others. More constructive (but definitely more > >> work) would be to instrument a kernel to actually keep track of memory > >> in sufficient detail, and (for example) log messages when it is > > > You're missing that an overcommitting MM system is more advanced than > > a nonovercommitting one. The kernel "keeps track of memory" the right way. > > An overcommitting MM system _can be_ more advanced. It can also > overcommit because it cannot figure out how not do so. As far as I > can tell linux currently falls in the latter category: it cannot > tell to what extent memory has been overcommitted, nor does it allow > for selective overcommitment.
IFAIK, older linux versions had no overcommitting. The kernel *evolved* from a non-overcommitting mm to a overcommitting one.
> -- > Olaf Weber > > Do not meddle in the affairs of sysadmins, > for they are quick to anger and have no need for subtlety. >
.TM. -- ____/ ____/ / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _____/ _____/ _/ Colombo@ESI.it
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
|  |