[lkml]   [2008]   [Aug]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Quad core CPUs loaded at only 50% when running a CPU and mmap intensive multi-threaded task
On Mon, Aug 25, 2008 at 01:41:17PM +0200, Peter Zijlstra wrote:
> I would certainly consider this for small (< 1M?) files. With mmap the
> faults and pte overhead aren't free either, and the extra memcpy from
> pread() isn't that much.

Even for very big files, if you're only doing a single sequential pass
over a very large file (for example when converting a Canon raw image
file to TIFF format --- I know because I was trying to optimize dcraw
a while aback), you take the page fault for each 4k page, and so
simply using read/pread is faster. And that's on a single-threded
program. With a multithreaded program, the locking issues come on top
of that.

Maybe if I had used hugepages it would have been a win, I suppose, but
I never tried the experiment. And this was several years ago, on much
older hardware, so maybe the relative times of doing the memory copy
versus the page fault, but I wouldn't be surprised if it's even more
expensive to do the mmap, relatively speaking.

- Ted

 \ /
  Last update: 2008-08-25 15:51    [W:0.074 / U:2.252 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site