lkml.org 
[lkml]   [1998]   [Jun]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Q: memory management

On Tue, 2 Jun 1998, Richard Gooch wrote:

> Peter Horton writes:
> >
> > I have a machine with 72MB of RAM and 128MB of swap.
> >
> > If I try to do an allocation of 512MB using sbrk(), it fails. However, I
> > can successfully do 512 sbrk()'s of 1MB. This causes a problem if I store
> > the pointers to the 1MB blocks, and then try and write to the 512MB of
> > memory I have just allocated ... the kernel seems to lock up solid. If I
> > write to each 1MB as it is allocated, the allocation fails around 180MB
> > (amount of free memory).
> >
> > If I understand correctly I can allocate more memory than I have because
> > the physical memory pages are not allocated until you access a block
> > (allowing sparse data structures).
>
> Yep.
>
> > So, my first question is why does the single sbrk(512MB) fail whilst the
> > 512 x sbrk(1MB) succeeds ?
>
> It's a similar dumb-arse bug that you'll find with mips4_IRIX6.2
> machines. On my Linux box I can allocate up to 2046 MBytes, but the
> largest single block is only 128 MBytes. This box has 128 MBytes of
> RAM and 128 MBytes of swap.
>
> On a certain mips4_IRIX6.2 machine (with 1 GByte RAM and 10 GBytes of
> swap) I can malloc 6454 MBytes but the single largest block I can
> allocate is only 2048 MBytes. This is if I allocate ever increasing
> blocks. If the first thing my programme does is allocate 800 MBytes,
> that will fail.
> Allocating 800 MBytes by mmapping /dev/zero works straight away.
>
> Clearly, the Linux and IRIX implementations of sbrk(2) are flawed. I
> note that Solaris 2 doesn't have these problems.
>
> > Secondly, does the kernel lock up, or did I just not wait long enough for
> > it to come back ? ... I'm not keen to try it again, as it resulted in an
> > hour of fsck'ing ... 8-)
>
> You could easily spend hours waiting for your machine to stop
> thrashing. Heavy paging also wears down your discs. The solution is:
> Don't Do That[tm].
>
> If you don't have sparse data, then don't use swap, just buy more
> RAM. This prevents thrashing, since you can't over-commit VM.
>
> If you are stuck with large sparse arrays larger than RAM, be careful
> not to touch more pages than you have available.
> You can improve things by putting your swap parition on a separate
> disc. Swap should *never* be on the same disc as your data and should
> preferably not be on the disc with your OS and programmes.
>

The machine didn't thrash at all. It quickly filled up all memory, and
then all the swap. Then it locked. What does the kernel do when it needs a
page, and all of the pages (including swap) are used ? I kinda assumed
that it would just kill the process ...

P.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu

\
 
 \ /
  Last update: 2005-03-22 13:42    [W:0.041 / U:0.056 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site