Messages in this thread | | | Date | Wed, 25 Aug 2004 08:50:55 +0200 | From | Jens Axboe <> | Subject | Re: [PATCH] Speed up the cdrw packet writing driver |
| |
On Tue, Aug 24 2004, Peter Osterlund wrote: > Jens Axboe <axboe@suse.de> writes: > > > On Mon, Aug 23 2004, Peter Osterlund wrote: > > > Jens Axboe <axboe@suse.de> writes: > > > > > > > On Sat, Aug 14 2004, Peter Osterlund wrote: > > > > > > > > > > This patch replaces the pd->bio_queue linked list with an rbtree. The > > > > > list can get very long (>200000 entries on a 1GB machine), so keeping > > > > > it sorted with a naive algorithm is far too expensive. > > > > > > > > It looks like you are assuming that bio->bi_sector is unique which isn't > > > > necessarily true. In that respect, list -> rbtree conversion isn't > > > > trivial (or, at least it requires extra code to handle this). > > > > > > I don't think that is assumed anywhere. > > > > > > [...] > > > > You are right, the code looks fine indeed. The bigger problem is > > probably that a faster data structure is needed at all, having hundreds > > of thousands bio's pending for a packet writing device is not nice at > > all. > > Why is it not nice? If the VM has decided to create 400MB of dirty > data on a DVD+RW packet device, I don't see a problem with submitting > all bio's at the same time to the packet device.
It's not nice because flushing 400MB of dirty data to the packet device will take _ages_. That the vm will dirty that much data for you non-blocking is a problem in itself. It would still be a really good idea for the packet writing driver to "congest" itself, like it happens for struct request devices, to prevent build up of these huge queues.
> The situation happened when I dumped >1GB of data to a DVD+RW disc on > a 1GB RAM machine. For some reason, the number of pending bio's didn't > go much larger than 200000 (ie 400MB) even though it could probably > have gone to 800MB without swapping. The machine didn't feel > unresponsive during this test.
But it very well could have. If you dive into the bio mempool (or the biovec pool) and end up having most of those reserved entries built up for execution half an hour from now, you'll stall (or at least hinder) other processes from getting real work done.
Latencies are horrible, I don't think it makes sense to allow more than a few handful of pending zone writes in the packet writing driver.
-- Jens Axboe
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |