Messages in this thread | | | From | () | Subject | Ideas on promised allocation response via the kernel | Date | 28 Mar 1996 20:37:47 -0500 |
| |
Today I decided to start playing with recording from the radio on my SoundBlaster. Well, it's been fun (there's a new radio station here in NYC called WKTU that makes this a great experience).
I've been using srec with these parameters:
srec -S -s 44100 -b 16 -d /dev/dsp /jaz/Sound1 ^^^ See below for description of results based on various drive types.
My lowly machine is an Intel Triton P133 from Dell, with 24M; I tend to keep it busy (I run a small newsfeed, web server, etc.), and I'm running 1.3.80.
At first I was trying my 1G Conner (IDE), and would frequently get this announcement while recording:
Mar 28 17:17:14 Q facility=kern(0) priority=warn(4) priority=warning(4) kernel: Sound: Recording overrun
(It shows up as a skip while playing back, similar to a CD skipping.)
This is most likely due to the HD stopping every once in a while to do some sort of bookkeeping that it does (what is this termed again? I read in an article about writing write-once CDs that in order to make a proper CD with many of the drives they were testing, either the input HD must not do this or there must be sufficient buffering.)
However it also does this more often when making the machine busy (I did a kernel make, innd shutdown & restart, and lots of paging on fvwm (since I have lots of windows open that are swapped this makes the machine busy) at the same time to increase this). One time, there was a song I really love and wanted to record perfectly, so I killed cron, innd, and a few other processes. While the overruns were reduced, they did not go away entirely until I stopped killing processes.
I seem to remember my 1.6G Fujitsu (IDE) having similar behavoir (btw, it's *TRUE* that two IDE HD's on same interface degrades performance! I'm happy for them anyway -- the extra space makes breathing much easier.)
My IOMEGA Jaz drive is simply wonderful in this respect (via an ncr53c825 SCSI controller) -- it does not do this, and I don't get any of the recording overrun messages, even when doing a kernel compile, and even when using dd bs=512k or bs=4096k to copy the same file to another disk at the same time as writing to the disk.
While I'm thinking about running out and getting a whole bunch of Jaz media (they're $500 for 5-pack of 1G disks which is ok but I have to wait until I can afford it -- I should consider writable CDs for this application really since they're cheaper), I realized there's not really much reason why these overruns should happen at all if the computer knew what it was doing, even on slower or jittery or even busy disks.
Here's a method I thought of:
(I intentionally use the term "space" instead of "memory" since I don't want to limit my idea to devices that call themselves RAM -- this method could theoretically work for disk (such as swap space) and other capable devices, too, parameters and performance permitting.)
Anything would be able to request allocatable space with promised parameters of size & speed. A successfully satisfied request for this promisable allocatable space would not be guaranteed, of course -- an error should be returned when it can't happen, possibly with an option to do "as much as possible" and/or "as fast as possible", and the user program can inform the user and/or make other adjustments as necessary. Before making this promise the kernel *could* use unallocated space (on a capable device) but more importantly it would also use allocated space on a capable device by copying it to swap & then marking such space soft-read-only. When a request to turn promised allocations into real allocations comes in, the kernel would consider this soft-read-only space swapped & allocate it to the promisee. As soon as a write request comes in for a soft-read-only space, the kernel would find another capable device and copy to swap sufficient allocations from it and mark those allocations soft-read-only then release the original soft-read-only memory to read-write (for writing). When the kernel senses some (certain?) resource utilization ebb(s), it can go through allocations on capable devices preemptively swapping them out & marking soft-read-only so that it's easier to fulfill requests of any sort in the future (such as hard-swap-out or write-request or allocate promised amount).
This code could be invoked by the hard disk driver in order to achieve certain performance parameters, such as being able to write (or, perhaps, to read (i.e. prepared read-ahead)) 178685 bytes per second without interruption for one certain calling function (possibly incorporating this commitment into a bigger overall commitment for the device that includes all commitments made to that device at the moment) (about what my srec above requires; conceivably, the hard disk would build its needs specifications on the hierarchy of file function calls to it that would also be specifying their needs requests -- possibly via ioctl?). The sound driver could also use these promisable allocatable amounts to buffer, but it seems wrong that it should do so considering it's a disk limitation, however it *could* originate the specifications of performance parameters that would be passed all the way through the path to the output bottlenecks, making only its own required latency allowances in its own allocations. Initially, the user program (such as srec) could do this in place of the disk & sound driver, as well, so placement of the call to this feature, while some places more appropriate & flexible & advantegous than others, isn't extremely vital.
The code making these calls should always calculate the possible latency in task scheduling, etc. and incorporate this into its request for size & speed requirements (possibly an equation (with a range) would be appropriate ... size per speed or such).
This would eliminate the need to kill processes and use certain devices in order to succeed in these otherwise simple excersizes (e.g., recording and/or playing sound and/or video). Since audio & visual stuff is being done more, I think these are important considerations to make.
Does the Linux kernel already offer mechanisms that fulfill these ideas?
I would then only need to tweak the hard disk drivers and/or the sound drivers to make use of these items (as well as possibly srec and splay too).
This isn't only useful for today's common hard disks -- it's also useful for other things such as network communications, etc. (The requests would have to grow & shrink with network changes in behavior.) Also this stuff could slowly be adapted to make up for performance issues within the programs, tasking & kernel themselves regardless of hardware buffer excellence.
P.S. I'll never get time to put this in a separate message even though it's unrelated ... performance from SCSI vfat sucks when cpio'ing to another IDE hard disk -- I suspect it's more VFAT driver related since the equivalent operation from ext2 on the same (Jaz) SCSI disk to the same IDE ext2 was ~3000 blocks/s, about my peak performance anyway (I read the discussion in the ncr53c810 mailing list about how Linux has great room for improvement in even these numbers too). It seems turning on tagged command queuing on SCSI makes the performance on a straight read better: vvvv 6 2 0 20284 200 4232 160 392 5004 98 521 732 10 90 0 :) Sigh I should turn on tagged queuing permanently since I already increased timeouts to 60 seconds on all my media devices (they *all* spin down during non-use) (None of this caused any recording overruns ... is something else in the IDE driver interfering with the sound driver then? Or is the IOMEGA just that good? Buffered?)
| |