Marco Colombo writes:> On 22 Mar 2000, Olaf Weber wrote:>> Marco Colombo writes:>>> malloc() "allocates" memory only on its own opaque pool. It has no>>> OS meaning.>> Correct.  But the C standard says that it has certain properties (and>> implies some others).  Basically, if I write a program and get reports> Pardon me, a standard either "says" something, or not. You may imply> something while reading it.\nitpick{I don't \emph{imply} things when reading the standard, I\emph{infer} them.}If you visit comp.std.c on a regular basis, you'll find that thereregular arguments about the exact meaning of the C standard, which canhinge of things that the text implies without stating them explicitly.>> that it dies with a SIGBUS in some circumstances, I go on a bug-hunt>> for the use of misaligned addresses and such.  I would be very unhappy> That's OS (UNIX) semantic. What exacly SIGBUS, SIGSEGV, and the like> mean is up to the kernel. You should know you may get a signal under> OOM. So, before bug-hunting, just check if the system was OOM.And how do you know the system may have been OOM, if you get a reportfrom someone else who may not have realised this?  Bear in mind thatan OOM situation may not have been logged in the syslog, or left anysign other than "randomly" crashing processes.>> to discover that I've been looking at a memory shortage masquerading>> as a program bug.  (I do test the return of malloc.)  If the signal> Why do you call receiving a signal a "program bug"?I don't.  But surely you're aware that the typical reason for aprocess being sent SIGBUS or SIGSEGV is a program bug.  This asopposed to SIGHUP, SIGTERM, and most of the other signals.In a similar vein, if a process dies due to SIGXCPU or SIGXFSZ thatyou know it dies because it ran into a cpu or file limit.  Yes, Iwould like to have some OS support from an overcommitting OS, so thatif it decides to kill a process of mine because it ran out memory Iget SIGXMEM instead of SIGBUS.>> Properly coping with memory that may not be there when it is used>> requires special programming techniques.  I'm not unwilling to use>> such techniques, when appropriate, but not in all cases.> No problem. Is is problem that your program gets stoppen when it gets> SIGTSTP? If no, you don't have to use any "special programming techniques".> If yes, you may think of trapping the signal and inform the user that> stopping your application is not possible. Are you concerned of your> program being killed at OOM time? Use special techniques to avoid> triggering the OOM killer (i.e., do not page-fault). It's harder> than just using signal(), i admit it. It may also be impossible to do> in a really "safe" way (your stack may grow...). So we need a better> kernel API to handle such cases. Again, not a C standard problem.One thing that you should bear in mind is what would happen if"everyone" (i.e., most of the processes of your normal workload) usethese techniques.  For example, if they all use mlock, you find that(1) they all need to run as (setuid) root, and (2) you might as wellrun without swap on a non-overcommitting system.  That is not anoptimal outcome.>>> You're missing that an overcommitting MM system is more advanced>>> than a nonovercommitting one. The kernel "keeps track of memory">>> the right way.>> An overcommitting MM system _can be_ more advanced.  It can also>> overcommit because it cannot figure out how not to do so.  As far>> as I can tell linux currently falls in the latter category: it>> cannot tell to what extent memory has been overcommitted, nor does>> it allow for selective overcommitment.> IFAIK, older linux versions had no overcommitting. The kernel *evolved*> from a non-overcommitting mm to a overcommitting one.Which doesn't mean the current implementation cannot be improved upon.There is sufficient room for improvement that it is certainly arguablethat right now the kernel doesn't keep track of memory the right way.-- Olaf WeberDo not meddle in the affairs of sysadmins,        for they are quick to anger and have no need for subtlety.-To unsubscribe from this list: send the line "unsubscribe linux-kernel" inthe body of a message to majordomo@vger.rutgers.eduPlease read the FAQ at http://www.tux.org/lkml/