Messages in this thread | | | Date | Thu, 18 Nov 1999 14:16:24 -0500 (EST) | From | William Stearns <> | Subject | Re: Transparent mounts (wishlist and design ideas, somewhat long) |
| |
Good day, Riley et al, Executive summary: The following offers a little in the way of design suggestions, but tends to be more focused on possible uses of a union/transparent approach from a non-kernel-programmer perspective.
On Thu, 18 Nov 1999, Riley Williams wrote:
> > BTW, as I stated, much of what this acomplishes can be got via a > > symlink farm. Why is that not enough here? > > >>> Many would like to mount a "live" filesystem from CD, and be > >>> able to "replace" files by ones on disk (i.e., / on CD, or the > >>> distribution; changes/updates on disk). > > OK, here's what I would like to see this end up with: > > 1. Any number of partitions can be mounted on top of each other, > with all files in all partitions visible.
I tend to like the idea that if a file exists on more than one level, the version on the topmost media that has it wins.
> 2. There can be any mixture of R/O and R/W partitions, with no > limit on the number of each.
Agreed.
> 3. Where a program attempts to modify an existing file found on > a R/W partition, it is directly modified on whichever of the > partitions it is currently on.
How about a simpler approach: Reading a file would pull it off the topmost media that had it. Writing a file would write it back to the topmost RW media - even if that was a different media than originally held the file. The administrator can then order the mounts so that the media most likely to have the preferred version of a file is on top. By writing to the topmost RW media, an administrator could even stick one last RO media on the _top_ of the stack (i.e., mounted last); any files in this media would automatically override any files on the RW media below - although one could successfully write to them, the OS would still use the version from the RO media on top. An alternate way of creating immutable files, essentially.
> 4. Where a program attempts to create a new file, it is created > on a designated read-write partition from those available.
As above, the topmost RW media. It logically follows that if all media are RO, the stack is also RO.
> 5. The designated partition can be different for different users.
Hmmm... That's a toughie. Does that imply that a userland app needs to be involved in the media decision, or could this requirement be satisfied with some list of pid's that write to media1, another list that writes to media2, etc? Even the latter might be problematic; does the kernel have access to the owner pid when the media choice needs to be made? What happens when chmod changes the owner?
> Note that I very specifically have NOT dealt with the problem of what > to do when a program attempts to modify an existing file on a R/O > partition. I can see valid arguments for both of the following > viewpoints: > > 6. Elsewhere, R/O partitions do not permit files to be modified > in any way, so why should it be different here. > > 7. Why can't I boot from a CD with a hard drive mounted over it, > and have the modifications made on the hard drive.
This is the one that holds an awe-inspiring amount of promise for me. I do my base install of distribution X, tar it up, and burn multiple CD's with the content. A few head off to other buildings for archive/backup. I mount one of the CD's and an empty ext2 drive on top as part of mounting the root filesystem (root=/dev/scd0,/dev/sda1). All of the the changes I make are on the hard drive; the CD holds the absolutely immutable pristine image. After a month or so, I tar up the contents of the hard drive only, and burn a new CD from the hard drive contents. Now the mount sequence is pristine, month1CD, empty HD. After another month, I tar up the hard drive+month1CD to get month2CD. The month1CD goes into storage and the boot sequence is pristine, month2CD, emptyHD. Lather, Rinse, repeat. What do I get out of this? - I _always_ have a pristine image to fall back on if I get hacked. In fact, I can just back up as many months as necessary. - I have a reasonably efficient method of making weekly or monthly backups; permanent ones that I can always go back to. Cheap backups, too. Live backups that can be directly mounted instead of being solely accessible through a backup application. - If I want to upgrade distributions or change to a completely different one, I just replace the lowest pristine image with another. (OK, the LSB isn't quite there yet, but the theory is still reasonably good...) - If I'm doing some testing of an app, or library upgrade, or major change to my system that might make it unusable afterwards, I make sure I have a CD burned just before I start. If something goes wrong, no problem. - I install a distribution and customize it to be a router/firewall. It gets burned to a CD. The next level up could be a floppy or pcmcia card or ramdisk (at startup and shutdown, the contents would need to be restored and backed up, respectively, to somewhere). - I do a base install of a distribution, burn 30 CD's of it, and carry the CD's to lab workstations. At the end of a class period, I go to single user mode, mke2fs the hard drive, and reboot; this could even be the default on shutdown. A network drive would be available for permanent storage of projects. - Any of the above references to CD could be replaced by coda, nfs, or samba mounted partition, or even another hard drive mounted read-only. - If some CD ends up with more than 680M of data, I just split the files onto two or more.
Please understand; I am _not_ advocating that the kernel suddenly takes responsibility for some individual's inability to manage his/her system. I _do_ understand that various uses of tripwire, rpm -V, tape backups, symlink farms, boot and shutdown time tars, additional hard drives, shell scripts, rdist/scp, common sense, etc. can very reasonably stand in for any of the above. These pre-union/transparent approaches _do_ work now and will continue to be available. I'm simply pointing out that the above union/transparent approach opens up a realm of possibilities that give the administrator an additional tool that, in some circumstances, might be "better" in that person's opinion.
> I believe this problem will prove to require a separate mount option > to allow users to specify which behaviour they require, and for this > reason, the ONLY decision I would make wwould be to specify that (6) > would be the default, with (7) chosen only if the other option is also > specified at mount time. See later... > > >>> OTOH, semantics are hard to come by... > > >>> - If I mount A, B, C; each containing file f. Which f do I see? > >>> Last one mounted, first one, individually selectable (how?)
Last one (=topmost) mounted would be my preference. Simpler to implement. Perhaps there should later be some kind of option similar to mounting an older image on a multi-write CD.
> >>> - What happens if I remount in another order later?
You change which file appears. Hey, the same problem exists if you swap scsi ID's and aren't using something like Ted's UUID's or Richard's devfs, so this isn't a new concept to admins. Please, no devfs wars; I immediately grant you that your opinion on this is absolutely correct.
> >>> - Assume the winning f above is the last one (from C). Now I > >>> delete f. What happens? Does f go away, do I see B's f? What > >>> if I unmount and then remount, does the change stick?
Interesting question. Would this require some kind of "negative dentry" that persists as long as the union is mounted? Should that negative dentry be stored on the topmost writable media or not?
> Additionally: > > 4. mount -o ro,trans,cow /dev/hda7 /mnt > > Mount a second or later partition transparently on top of > the existing ones, also mounted read-only, but specify > that if any files on this partition are modified, they are > to be copied to whatever is the current read-write mount > in this stack. > > Note that "cow" = "copy on write" as per Linux standards.
In my view of the process, cow would be the default. It would be understood that when first booting a system set up this way that there would be a lot of copying. For some far-in-the-future version, the concept of copy-on-read would be marvelous too; the hard drive could transparently act as a local cache to a remote nfs volume for only the files you actually access. One could start off with an nfs volume that had an _entire_ distribution installed and an empty local drive. As files get read, they get copied to the local hard drive. After a few weeks of priming the local cache, the union could theoretically get broken and have just a straight hard drive mount for, say, a laptop. What happens when the laptop user wants a file they didn't use before breaking the union, what happens when a newer version shows up on the nfs server, etc....
> >> The following restrictions could then be reasonably implemented: > > >> 1. The bottom partition MUST be read-only. Given these semantics, > >> this is not a problem. > > >> 2. In any stack of transparent mounts, AT MOST ONE may be > >> marked read-write. Any attempt to mount a second one as > >> read-write MUST fail. However, there is no problem with > >> a stack having no read-write partitions in it.
While I could certainly believe that there might be programming or kernel details that might require this, if we took the "read from the topmost media that has the file" and "write to the topmost RW media", the above two restrictions wouldn't _seem_ to be necessary. But maybe I've missed something. Thanks for your time. Cheers, - Bill
--------------------------------------------------------------------------- People standing in the middle of the road look like roadkill to me. -- Linus Torvalds, keynote speech at LWCE. -------------------------------------------------------------------------- William Stearns (wstearns@pobox.com). Mason, Buildkernel, named2hosts, and ipfwadm2ipchains are at: http://www.pobox.com/~wstearns/ --------------------------------------------------------------------------
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |