Messages in this thread | | | Date | Fri, 20 Feb 1998 22:19:07 -0500 | From | Martin Wahlen <> | Subject | Re: virtual disk system |
| |
At 03:35 PM 2/20/98 -0600, you wrote:
>2. It would be great if when my physical disk filled up I could just >start writing on emptier disks located either in my machine or across the >network.
Yes, that would be greate, or to a tape and or optical device. This would complement my ideas on a HSM system for Linux (almost).
>3. It might be difficult to get every filesystem to support the idea of >growing to accomodate more space.
Yes, this probably a though thing to do, a better idea is as you have stated bellow to use a lokal disk as cache or first tear storage and a Network, Tape or Optical Library as Near line storage.
>5. This is *not* a filesharing system. It would be implemented at the >*block* level. There would be an equivalent of a "page table" for looking >up information. Only the original writer is guaranteed to know where >their files were. Sharing would have to be done with NFS or something >else.
I would suggest making the system have a daemon running that would write to what ever mount point (ext2, NFS, smb) or would have the possibility of writing to tape device. This would make implementing HSM using the same mechanism much easier. This user space program could also be used to write two copies of the same data to two different devices ( local or remote) and thus we would have a bit of redundancy.
>8. Rather than having a dedicated, fixed size cache, the system would be >an all-cache system. When I requested a file to my workstation, if it was >not already local, it would be copied to my local hard drive over >something else, and then whoever I copied it from could mark that block as >"able to be written over" since I have a valid copy.
I would suggest using a slightly more complicated model. When a file is not used for a period of time and the system has reached a certain level files would be premigrated (ala Veritas). If more space is needed on the Virtual block device the local copy gets deleted. If the remote instance would only be marked free if the blocks were modified. If a read occured then we would have to leave the blocks as premigrated this would lead to reduced network traffic.
>10. A quota system would probably be desirable to keep it from going >completely out of hand, but then again, if you pick reasonable partition >sizes, it should be okay since you won't expand past what you decide the >partition be.
Quota would definatly be disireble, also the ability to mark files as non movable objects. I.E. stuff that has to remain in the local cache disk would be desireble.
>Anyway, it's an idea that I would like to persue if any of you think it's >viable. It would be alot of work to design, but hey, that's what senior >design is all about, right? Any comments would be greatly appreciated.
It is a viable method since people from Veritas, Netstor (CA) ,etc are doing it for UNIX/Windows NT so why not Linux. Btw. I like your ideas of doing HSM over a network in a clustered environment this is something I had not considred. You should have a look at the latest BYTE magazine and at the work of the IEEE SSS?G (I can't remember what they are called) who are trying to standardize storage systems.
>If you have any interest, please e-mail me, and maybe I can setup a list. >I've actually thought through the problem more than I can relate here.
If you don't mind I would like to try to help you with your endeavour. I am not a great programmer but I could help set up a web page/mailing list and try to help out with the programming/ideas. =========================================================================== Martin Wahlen Sound Foundation Inc. tel. (613) 563-2226 mwahlen@soundfound.com 152 Hawthorne Ave. fax. (613) 563-2228 www.soundfound.com K1S 0B3 ===========================================================================
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu
| |