Messages in this thread | | | From | Jeff Moyer <> | Subject | Re: [-stable 3.8.1 performance regression] madvise POSIX_FADV_DONTNEED | Date | Wed, 03 Jul 2013 10:53:08 -0400 |
| |
Mel Gorman <mgorman@suse.de> writes:
>> > I just tried replacing my sync_file_range()+fadvise() calls and instead >> > pass the O_DIRECT flag to open(). Unfortunately, I must be doing >> > something very wrong, because I get only 1/3rd of the throughput, and >> > the page cache fills up. Any idea why ? >> >> Since O_DIRECT does not seem to provide acceptable throughput, it may be >> interesting to investigate other ways to lessen the latency impact of >> the fadvise DONTNEED hint. >> > > There are cases where O_DIRECT falls back to buffered IO which is why you > might have found that page cache was still filling up. There are a few > reasons why this can happen but I would guess the common cause is that > the range of pages being written was in the page cache already and could > not be invalidated for some reason. I'm guessing this is the common case > for page cache filling even with O_DIRECT but would not bet money on it > as it's not a problem I investigated before.
Even when O_DIRECT falls back to buffered I/O for writes, it will invalidate the page cache range described by the buffered I/O once it completes. For reads, the range is written out synchronously before the direct I/O is issued. Either way, you shouldn't see the page cache filling up.
Switching to O_DIRECT often incurs a performance hit, especially if the application does not submit more than one I/O at a time. Remember, you're not getting readahead, and you're not getting the benefit of the writeback code submitting batches of I/O.
HTH, Jeff
| |