Messages in this thread |  | | Date | Fri, 6 Sep 1996 15:00:43 +1000 | From | Richard Gooch <> | Subject | Buffer cache hints |
| |
Hi, all. I have a process which reads a large file (nearly half the size of RAM) and swap-copies it into a malloced array of the same size. This results in a lot of disc head movement as pages are read from the file and written to the swap partition (when I can afford it, I'll get that second SCSI disc and move swap to that). This behaviour appears to be due to the kernels attempt to keep bits of the file I have read in the buffer cache, so malloced pages tend to be thrown out. Since I know that my process will read the file sequentially and then close it, this seems to be sub-optimal. A few solutions come to mind:
1) give malloced pages a higher priority than disc buffer pages. This has the disadvantage of crippling disc access performance for frequently used pages (like libc :-)
2) providing a hint to the kernel that I will be accessing a file in a sequential fashion, and the kernel then reclaims the oldest pages in the file in preference to writing malloced pages to swap. Since I may either use read() or mmap() to read the file, this scheme should work for both those cases
3) mlocking the malloced pages, but that's very unfriendly (and good luck if my file happens to be bigger than RAM)
I like (2) the best (this does not appear to be implemented), but perhaps there are problems with this, or someone can see a better way to do it. The current MM scheme does not work well with big data.
Regards,
Richard....
|  |