Messages in this thread | | | Date | Fri, 9 Sep 2005 18:47:42 -0700 | From | Nate Diller <> | Subject | Re: kernel 2.6.13 buffer strangeness |
| |
On 9/9/05, Anthony Wesley <awesley@acquerra.com.au> wrote: > Okay, just tested a couple of things, here's what I see... > > I tested the write speed to the usb2 hard disk and got 21MBytes/sec. It's a > laptop > hard drive, only 5400rpm so this is not surprising. > > I did this test with my video capture app running, but just displaying data > and not writing, > > I have another laptop usb2 hard drive here which I just tested and got > 17MBytes/sec - it's > a 4200rpm drive so again not surprising numbers. > > The video data is written as individual frames, so the efficiency is a bit > below the raw throughput, > but my tests were transferring 1.5Gb of data as raw frames - exactly the > same way that Coriander would > write its data. > > I'd bet a large sum of money that these hard disk figures are correct to > within a few percent. > > Also, when I am actually capturing I have timed by stopwatch how long the > disk activity light > is on, and reached about the same conclusion. > > Working the problem from the other direction, the only way to explain the > early throttling that > I see would be if *almost no data* is being written to the disk, and this is > plainly not the case. Even if > the disk were running at a greatly reduced rate of (say) 10MBytes/sec I > would still see 86 seconds > of buffering before the throttle kicks in, and so far I have managed only to > get to about 65 or 70. > > regards, Anthony > i need a vmstat trace ('vmstat 1 > logfile', then run the test, then ^c) before i can comment further. also, your filesystem and scheduler would be interesting to know. you can find out the scheduler with 'cat /sys/block/[drive]/queue/scheduler'.
NATE
> Nate Diller wrote: > > > >>Setting dirty_ratio and dirty_background_ratio to 90/10 puts me back at > >>around 50 seconds, i.e. where I started. > >> > > > > this setting should do the trick, so there's something going on here > > that isn't expected. > > > > > >>So as far as I can see there is *no way* to get 3 minutes worth of > buffering > >>by adjusting these parameters. > >> > >>Just to remind everyone - I have video data coming in at 25MBytes/sec and > I > >>am writing it out to a usb2 hard disk that can sustain 17MBytes/sec. I > want > >>my video capture to run at full speed as long as possible by having the > >>7MBytes/sec deficit slowly eat up the available RAM in the machine. I > have > >>1.5Gb of RAM, 1.3Gb available for buffers, so this should take 3 minutes > to > >>consume at 7MBytes/sec. > >> > >>So, I've tried all the combinations on dirty_ratio and > >>dirty_background_ratio and they *do not help*. > >> > > > > dirty_ratio is the tubable you want, if it's not working correctly, > > either there's a problem with your setup, or a bug > > > > > >>Can anyone suggest something else that I might try? The goal is to have > >>25MBytes/sec streaming video for about 3 minutes. > >> > > > > how sure are you that you're getting 17MB/s during this test? can you > > run "vmstat 1" while this is running to verify? which FS and > > scheduler? > > > > just for interest, what's the raw disk bandwidth (use hdparm, or run a > > dd, or something)? it would obviously be much better to sustain > > 25MB/s to disk > > > > > >>Or is this simply not possible with the current kernel I/O setup? Do I > have > >>to do something elaborate myself, like build a big RAM buffer, mount the > >>disk synchronous, do the buffering myself in userland...?? > >> > > > > this should be possible, although it could be considered a bit risky WRT > OOM. > > > > NATE > > -- > Anthony Wesley > Director and IT/Network Consultant > Smart Networks Pty Ltd > Acquerra Pty Ltd > > Anthony.Wesley@acquerra.com.au > Phone: (02) 62595404 or 0419409836 > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |