[lkml]   [2004]   [Oct]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Buffered I/O slowness
Jesse Barnes <> wrote:
> ...
> o one thread on one large volume using buffered I/O + filesystem
> read (1 thread, one volume, 131072 blocks/request) avg: ~931 MB/s
> write (1 thread, one volume, 131072 blocks/request) avg: ~908 MB/s
> I'm intentionally issuing very large reads and writes here to take advantage
> of the striping, but it looks like both the readahead and regular buffered
> I/O code will split the I/O into page sized chunks?

No, the readahead code will assemble single BIOs up to the size of the
readahead window. So the single-page-reads in do_generic_mapping_read()
should never happen, because the pages are in cache from the readahead.

> The call chain is pretty
> long, but it looks to me like do_generic_mapping_read() will split the reads
> up by page and issue them independently to the lower levels. In the direct
> I/O case, up to 64 pages are issued at a time, which seems like it would help
> throughput quite a bit. The profile seems to confirm this. Unfortunately I
> didn't save the vmstat output for this run (and now the fc switch is
> misbehaving so I have to fix that before I run again), but iirc the system
> time was pretty high given that only one thread was issuing I/O.
> So maybe a few things need to be done:
> o set readahead to larger values by default for dm volumes at setup time
> (the default was very small)

Well possibly. dm has control of queue->backing_dev_info and is free to
tune the queue's default readahead.

> o maybe bypass readahead for very large requests?
> if the process is doing a huge request, chances are that readahead won't
> benefit it as much as a process doing small requests

Maybe - but bear in mind that this is all pinned memory when the I/O is in
flight, so some upper bound has to remain.

> o not sure about writes yet, I haven't looked at that call chain much yet
> Does any of this sound reasonable at all? What else could be done to make the
> buffered I/O layer friendlier to large requests?

I'm not sure that we know what's going on yet. I certainly don't. The
above numbers look good, so what's the problem???

Suggest you get geared up to monitor the BIOs going into submit_bio().
Look at their bi_sector and bi_size. Make sure that buffered I/O is doing
the right thing.
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2005-03-22 14:07    [W:0.742 / U:9.916 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site