lkml.org 
[lkml]   [1998]   [Jun]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Q: memory management
Peter Horton writes:
>
> On Tue, 2 Jun 1998, Richard Gooch wrote:
>
> > Peter Horton writes:
> > >
> > > I have a machine with 72MB of RAM and 128MB of swap.
> > >
> > > If I try to do an allocation of 512MB using sbrk(), it fails. However, I
> > > can successfully do 512 sbrk()'s of 1MB. This causes a problem if I store
> > > the pointers to the 1MB blocks, and then try and write to the 512MB of
> > > memory I have just allocated ... the kernel seems to lock up solid. If I
> > > write to each 1MB as it is allocated, the allocation fails around 180MB
> > > (amount of free memory).
> > >
> > > If I understand correctly I can allocate more memory than I have because
> > > the physical memory pages are not allocated until you access a block
> > > (allowing sparse data structures).
> >
> > Yep.
> >
> > > So, my first question is why does the single sbrk(512MB) fail whilst the
> > > 512 x sbrk(1MB) succeeds ?
> >
> > It's a similar dumb-arse bug that you'll find with mips4_IRIX6.2
> > machines. On my Linux box I can allocate up to 2046 MBytes, but the
> > largest single block is only 128 MBytes. This box has 128 MBytes of
> > RAM and 128 MBytes of swap.
> >
> > On a certain mips4_IRIX6.2 machine (with 1 GByte RAM and 10 GBytes of
> > swap) I can malloc 6454 MBytes but the single largest block I can
> > allocate is only 2048 MBytes. This is if I allocate ever increasing
> > blocks. If the first thing my programme does is allocate 800 MBytes,
> > that will fail.
> > Allocating 800 MBytes by mmapping /dev/zero works straight away.
> >
> > Clearly, the Linux and IRIX implementations of sbrk(2) are flawed. I
> > note that Solaris 2 doesn't have these problems.
> >
> > > Secondly, does the kernel lock up, or did I just not wait long enough for
> > > it to come back ? ... I'm not keen to try it again, as it resulted in an
> > > hour of fsck'ing ... 8-)
> >
> > You could easily spend hours waiting for your machine to stop
> > thrashing. Heavy paging also wears down your discs. The solution is:
> > Don't Do That[tm].
> >
> > If you don't have sparse data, then don't use swap, just buy more
> > RAM. This prevents thrashing, since you can't over-commit VM.
> >
> > If you are stuck with large sparse arrays larger than RAM, be careful
> > not to touch more pages than you have available.
> > You can improve things by putting your swap parition on a separate
> > disc. Swap should *never* be on the same disc as your data and should
> > preferably not be on the disc with your OS and programmes.
> >
>
> The machine didn't thrash at all. It quickly filled up all memory, and
> then all the swap. Then it locked. What does the kernel do when it needs a
> page, and all of the pages (including swap) are used ? I kinda assumed
> that it would just kill the process ...

Nope, it keeps trying to free pages (like throwing away pages in the
buffer cache).

Regards,

Richard....

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu

\
 
 \ /
  Last update: 2005-03-22 13:42    [W:0.038 / U:1.856 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site