[lkml]   [1997]   [Feb]   [11]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Subjectre: Big mallocs, mmap sorrows and double buffering.
On Mon, 10 Feb 1997, Mark Hahn wrote:

> use mmap! it's _easier_ than malloc/read, and better yet, is also
> _optimal_ when confronted with changing memory loads. and of course
> it also happens to be significantly more efficient, since you get
> buffer-cache blocks mapped directly into your address space:
> your disk controller reads them into memory, and you get to see
> them, no silly copying from kernel to user-space.
> let me know if you need some code on mmap. I use it extensively.

Sounds like just what I need.

Hmm. I remember using a facility like this on the old vaxen. The
glitch was if I mapped an entire satellite image (+-400megabytes) the
page tables used up so much physical memory that the thing started
thrashing again. (There we just settled down to using two 32k buffers
and doing NO_WAIT read/writes.)

Side Question : On Vaxen they had fancy IO hardware so it paid to do double
buffering. (Ie Read first buffer, Read second buffer NO_WAIT, process
first while second buffer is reading, Write NO_WAIT first output
buffer, Read NO_WAIT into first input buffer, process second input
buffer, etc. etc.) Does it pay to do this on a PC under Linux?

I have just tried mmap() & memcpy() to copy a 19Mb file on a 24Mb ram
system. It started thrashing quite heavily but completed without

Sorrow! Woe! Misery! I tried it on a 38Mb file and it said
"output memory map : out of memory."
Is that a mmem() bug perhaps? Or has too many page descriptors
been created to fit into physical memory?

So back to my original question. How do I ask linux kernel how much
physical memory I can use without causing thrashing?

#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
#include <stdio.h>

int ifd, ofd;
struct stat st;
char * icp, * ocp;

ifd = open( "/home/john/pgm/tryout/temp", 0, 0);
if( ifd < 0)
perror( "input file");
return 1;

ofd = open( "/home/john/pgm/tryout/temp1", O_CREAT | O_TRUNC, O_WRONLY);
if( ofd < 0)
perror( "output file");
return 1;

fstat( ifd, &st);

icp = (char *)mmap( 0, st.st_size,

if( icp == (char *)-1)
perror( "input memory map");
return 1;

ocp = (char *)mmap( 0, st.st_size,

if( ocp == (char *)-1)
perror( "output memory map");
return 1;

memcpy( ocp, icp, st.st_size); // OK, so I don't actually change
// anything here, but in a real
// program I would.

munmap( icp, st.st_size);
munmap( ocp, st.st_size);

close( ifd);
close( ofd);

John Carter EMail:
Telephone : 27-12-808-0374x194 Fax:- 27-12-808-0338

Founder of the Council for Unnatural Scientists.

 \ /
  Last update: 2005-03-22 13:38    [W:0.081 / U:0.624 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site