Messages in this thread |  | | Date | Mon, 21 Aug 2000 03:41:35 -0700 (PDT) | From | Clayton Weaver <> | Subject | Re: abstract file (multi-part one file or multiple files?) |
| |
On Sun, 20 Aug 2000, Albert D. Cahalan wrote:
> Clayton Weaver writes:
> A file has these properties:
> 1. it can not be the current working directory of a process > 2. it can be atomically unlinked > 3. it has a single ACL (or UNIX permission bits, or whatever) > 4. it can be hard-linked if the filesystem supports hard links > 5. it has exactly one inode number > 6. it has a visible name (ignoring hard links and deleted files) > 7. it is of type S_IFREG (but note: directories may have streams too) > 8. it has at least one stream, just as a process has threads
Nice list.
> A stream has these properties:
> 1. it is a blob of data w/o OS-interpreted structure > 2. it shared (inheritance WITH UPDATE) properties with others > 3. it can not be independently hard linked > 4. it has no normally visible name > 5. st_mode has no meaning, except to describe the parent type > 6. streams themselves do not have streams
[snip nesting in multi-part document files]
> One may implement complex document files using multi-streamed > files or, less efficiently, using single-streamed files. Using > multi-streamed files is a performance hack that may require some > sort of format-specific loadable module.
How is it a performance hack?
Why doesn't a sysV streams type of interface mapping to multi-stream storage object filesystem backends make more sense than trying to support them as normal files from the VFS point-of-view?
(Usual litany of "sysV strems were broken as defined", well maybe, but then that would apply to multi-stream files for the same reasons, no? Or was the much bewailed brokenness of sysV streams earned for reasons entirely independent of the semantics of streams described by Albert above, and something like sysV streams without the broken parts could be implemented for linux, and then used as an interface to multi-stream filesystems?)
Note: I can't say I really care one way or the other about the answers to these questions. My desire to access filesystems from linux that move multi-part storage object semantics from application space into kernel space is nil. But since the subject came up and at the very least this is a real issue for the maintainers of the HFS and NTFS filesystem drivers, starting with a multi-part abstract storage object and dropping the multi-part semantics when handling single bytestream files just seems intuitively simpler at the design level than pasting the multi-part semantics onto an implementation that assumes a single bytestream model of a file by default (when you have to do one or the other in the VFS).
The default multi-part design, however, inevitably costs space, branches, and probably dereferences when accessing single bytestream files that wouldn't be paid by an implementation that assumes traditional unix files by default.
The thing is, treating multi-part storage objects as merely an application level issue, where the VFS doesn't have to do anything special about them (because all the parts are merely normal directories and files at the system call level), is not broken now for those applications that implement it that way. ("If it ain't broke, ...")
I hate to see us muck up the VFS just because some idiot vendors failed to recognize the wisdom of leaving the association of multiple filesystem streams in multi-part storage objects entirely in user space (ie load a .so or .dll or whatever the MacOS equivalent is that defines the semantics and data structures and access functions for multi-part abstract storage objects, for the benefit of simpler, faster VFS code that will inevitably have less bugs). Another way to look at it is that you should only pay extra space/cycles/complexity overhead for multi-part storage objects if your application actually needs them for something, so it's a user space application issue, not a kernel issue.
You can #ifdef it all, of course, have default multi-part objects in VFS only if a filesystem backend that needs them is compiled in or as a module, but what a maintenance nightmare, holding the code details of and differences between both models in your head when changing something that affects VFS. I would think people would not desire to repeat the experience of implementing the uniprocessor/smp dichotomy to implement an #ifdef-branched traditional/multi-part default VFS storage object model. (Consider this a straw man if you want, but this is an inevitable suggestion to please both people that don't care about multi-part storage object filesystem support and people that do. I'm just noting that it does have costs, and not just at runtime.)
Simplicity wins. It is ready sooner, it runs faster, users that can live with what it provides are much happier than BSOD victims. It only lacks some features of the complex alternatives. Which is easier to work around, missing "automatic" whiz bang features in a kernel or one that has everything you could want on paper but won't stay up and running reliably?
Linus knows this, of course, "the simplest design that will do the job, as long as the job really gets done," is practically a personal signature in his code. So his earlier comments would probably evolve to the question of how to cull all of the necessary persistent streams/multi-part storage object support into one set of functions that has the abstract nature of VFS and can be called by VFS when accessing filesystems that support such multi-part storage objects natively, but without VFS itself having any default dependency on any part of it.
That is the question.
Regards,
Clayton Weaver <mailto:cgweav@eskimo.com> (Seattle)
"Everybody's ignorant, just in different subjects." Will Rogers
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/
|  |