lkml.org 
[lkml]   [1997]   [Feb]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectBig malloc's.
Greetings Kernellers, (Kernelites, Kerns,...)

We have had the "I can crash ... by malloc'ing something huge and
touching every page" thread on this list a couple of times.

Now I want to ask the next step...

Suppose I want to move through a _lot_ of data quickly. A really big
file (up to several times the size of memory+swap). Typically I malloc() a
BIG array, read() the file into that array and off I go. If the array
is too big, I start swapping like mad. Slow.

Thus its better if I chop it into buffers. The question is, how big a
buffer. The smaller the buffer, the more read()'s and the more
fiddling with buffer boundary conditions I do. Too big a buffer and I
start swapping.

My question is then...

How do I write a program that will ask the kernel how much free
physical memory there is available to me without causing a lot of
swapping?

Now there may be no free physical memory available as buffers and idle
processes may be hogging everything. Thus I need something that says
"swap out all idle processes, flush and forget 90% of the file
buffers, then tell me how much physcial memory I have to play around
with."

Could I memory map the file directly? Would it be faster / better? If
I was modifying the file, could I tell the kernel, read untouched
pages from this file, write modified pages to that file?

Is there a better way to do this class of operation?

John Carter EMail: ece@dwaf-hri.pwv.gov.za
Telephone : 27-12-808-0374x194 Fax:- 27-12-808-0338

Founder of the Council for Unnatural Scientists.


\
 
 \ /
  Last update: 2005-03-22 13:38    [W:0.035 / U:1.612 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site