lkml.org 
[lkml]   [1998]   [Apr]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: CDROM jukebox filesystem using autofs
On Thu, 30 Apr 1998, Rik van Riel wrote:

#On Thu, 30 Apr 1998, Dan A. Dickey wrote:
#> Aaron Passey wrote:
#
#> > So far, I have made it move and mount disks on command and unmount and put away
#> > disks in an LRU fashion. For example, if I do an ls of /autofs/1, it'll get
#> > the disk out of slot 1 and put it in one of the drives, mount it and give me
#> > the ls.
#> >
#> > There are several problems I am running into with this approach. First
#> > of all, what do I do if I have 4 drives, all mounted and busy, and I get a
#> > request for a fifth disk?
#>
#> return EBUSY.
#
#Or you could:
#- let the user of the first started request wait, and switch
# drives again after XXX kilobytes (making sure that throughput
# isn't hurt too much, latency will be bad anyway)
#- have a caching HD partition and read the biggest file requested
# for the next X MB, then quickly switch disks to the requested
# one

Or, do it the way I've done it with a large quantity (12) of Pioneer 6
disc SCSI changers...

User1->Request File /cd3/images/boot.img
(1 Second)
User2->Request File /cd3/images/boot.img
(3 Seconds)
Server->Send File /cd3/images/boot.img /cdcache
(1 Second)
Server->Send File->User1
Server->Send File->User2

This is a very effecient way that I've found to deal with CD changers and
jukeboxes when there are files that are frequently accessed.

User1->Request File /cd16/system/XFree86-All.tgz
(1 Second)
User2->Request File /cd17/misc/games/quake-tf.tgz
(3 Seconds)
Server->Send File /cd16/system/XFree86-All.tgz /cdcache
(Wait for file to copy)
Server->Send File /cdcache/XFree86-All.tgz->User1
Server->Change CD17
(3 Seconds)
Server->Send File /cd17/misc/games/quake-tf.tgz->User2

This allows for multiple connection, with minimal slowdown and wait in a
changer environment.

Now for the fun part, a jukebox environment. My example will use a file
queueing system I used before with a machine that had 6 SCSI CD-ROMs, and
nothing more. Boot disk was IDE. You know the routine. Ran Linux. :)

User1->Request File /cd5/thisfile.tgz
User2->Request File /cd2/thatfile.txt
User5->Request File /cd1/allfiles.txt
User3->Request File /cd6/303wavs.gz
Server->Process Requests, sort by filesize
(File Sizes:
thisfile.tgz 53K
thatfile.txt 32K
allfiles.txt 10K
303wavz.gz 23M;)
Server->Send File->User5
(Wait for 10% of send before starting next)
Server->Send File->User2
(Wait again)
Server->Send File->User1
(Wait)
Server->Send File->User3
User4->Request File /cd3/bob/thepipes.tgz (33M)
Server->Send File->User4
User2->Request File /cd3/bob/thefrop.txt (60K)
Server->Communicate->User2 "Server is busy, your request has been queued"
(Wait till User3's file is 40% done)
Server->Send File->User2

In a high use environment, the above principle would apply as well. With
some modifications, of course. With the current technology, that kind of
machine would have 6 CD-ROMs on an UltraWide PCI, and two 6.4G UW drives.
One for the OS, the other for caching exclusively. It may sound a little
big, but YMMV, as always. :)

#For truly high-latency apps this might not be the case.
#Simply make sure that your readahead (on large files) is
#big enough (and maybe HD cached) so that no more than
#50% of the time/bandwidth is spent switching the disks.

The easiest way to catch this is user limits, and using the principle
outlined above, where it waits for a certain percentage of the file to be
sent. Another issue to consider is processor use for sorting files. In a
Perfect Environment(tm), you wouldn't need to worry about it, and could be
picky as all hell, down to a single bit of difference in size. In reality,
you're probably running on a single P200 or dual PPro180, which means you
do have limited processing power, and you need to keep it low. The easiest
way to do this is to classify filesizes in blocks. 1bit to 4M, 4.001M to
10M, and so on. This lets the machine run through a small list of sizes,
decide where the file fits, and throw it in there. (At this point, a
coworker distracted me to fix his bloody AIX box.) (And again, to fix
permissions. And again, to find X. And again, to fix something else) (Now
I try to find train of thought again after about an hour.) By using
filesize blocks, the machine can easily categorize. Utilize caching as
well, and it speeds up a lot. Either way, jukebox will always be faster
than changer, and I give up. I can't remember what I was going to say. :)

-Phil R. Jaenke (kernel@nls.net / prj@nls.net)
TheGuyInCharge(tm), Ketyra Designs - We get paid to break stuff :)
Linux pkrea.ketyra.INT 2.0.33 #15 Sat Apr 18 00:40:21 EDT 1998 i586
Linux eiterra.nls.net 2.0.33 #15 Fri Apr 17 00:22:13 EDT 1998 i586
- Linus says for 'brave people only.' I say 'keep a backup.' - :)


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu

\
 
 \ /
  Last update: 2005-03-22 13:42    [W:0.052 / U:0.868 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site