[lkml]   [2000]   [Aug]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: abstract file (multi-part one file or multiple files?)
On Sun, 20 Aug 2000, Albert D. Cahalan wrote:

> Clayton Weaver writes:

> A file has these properties:

> 1. it can not be the current working directory of a process
> 2. it can be atomically unlinked
> 3. it has a single ACL (or UNIX permission bits, or whatever)
> 4. it can be hard-linked if the filesystem supports hard links
> 5. it has exactly one inode number
> 6. it has a visible name (ignoring hard links and deleted files)
> 7. it is of type S_IFREG (but note: directories may have streams too)
> 8. it has at least one stream, just as a process has threads

Nice list.

> A stream has these properties:

> 1. it is a blob of data w/o OS-interpreted structure
> 2. it shared (inheritance WITH UPDATE) properties with others
> 3. it can not be independently hard linked
> 4. it has no normally visible name
> 5. st_mode has no meaning, except to describe the parent type
> 6. streams themselves do not have streams

[snip nesting in multi-part document files]

> One may implement complex document files using multi-streamed
> files or, less efficiently, using single-streamed files. Using
> multi-streamed files is a performance hack that may require some
> sort of format-specific loadable module.

How is it a performance hack?

Why doesn't a sysV streams type of interface mapping to multi-stream
storage object filesystem backends make more sense than trying to support
them as normal files from the VFS point-of-view?

(Usual litany of "sysV strems were broken as defined", well maybe, but
then that would apply to multi-stream files for the same reasons, no?
Or was the much bewailed brokenness of sysV streams earned for reasons
entirely independent of the semantics of streams described by Albert
above, and something like sysV streams without the broken parts could be
implemented for linux, and then used as an interface to multi-stream

Note: I can't say I really care one way or the other about the
answers to these questions. My desire to access filesystems from
linux that move multi-part storage object semantics from application
space into kernel space is nil. But since the subject came up and
at the very least this is a real issue for the maintainers of the HFS and
NTFS filesystem drivers, starting with a multi-part abstract storage
object and dropping the multi-part semantics when handling single
bytestream files just seems intuitively simpler at the design level than
pasting the multi-part semantics onto an implementation that assumes a
single bytestream model of a file by default (when you have to do one or
the other in the VFS).

The default multi-part design, however, inevitably costs space, branches,
and probably dereferences when accessing single bytestream files that
wouldn't be paid by an implementation that assumes traditional unix files
by default.

The thing is, treating multi-part storage objects as merely an application
level issue, where the VFS doesn't have to do anything special about
them (because all the parts are merely normal directories and files at the
system call level), is not broken now for those applications that
implement it that way. ("If it ain't broke, ...")

I hate to see us muck up the VFS just because some idiot vendors
failed to recognize the wisdom of leaving the association of
multiple filesystem streams in multi-part storage objects entirely
in user space (ie load a .so or .dll or whatever the MacOS equivalent
is that defines the semantics and data structures and access functions
for multi-part abstract storage objects, for the benefit of simpler,
faster VFS code that will inevitably have less bugs). Another way to look
at it is that you should only pay extra space/cycles/complexity overhead
for multi-part storage objects if your application actually needs them for
something, so it's a user space application issue, not a kernel issue.

You can #ifdef it all, of course, have default multi-part objects in VFS
only if a filesystem backend that needs them is compiled in or as
a module, but what a maintenance nightmare, holding the code details of
and differences between both models in your head when changing something
that affects VFS. I would think people would not desire to repeat the
experience of implementing the uniprocessor/smp dichotomy to implement an
#ifdef-branched traditional/multi-part default VFS storage object model.
(Consider this a straw man if you want, but this is an inevitable
suggestion to please both people that don't care about multi-part storage
object filesystem support and people that do. I'm just noting that it does
have costs, and not just at runtime.)

Simplicity wins. It is ready sooner, it runs faster, users that can
live with what it provides are much happier than BSOD victims. It
only lacks some features of the complex alternatives. Which is easier
to work around, missing "automatic" whiz bang features in a kernel or one
that has everything you could want on paper but won't stay up and running

Linus knows this, of course, "the simplest design that will do the job,
as long as the job really gets done," is practically a personal signature
in his code. So his earlier comments would probably evolve to the
question of how to cull all of the necessary persistent streams/multi-part
storage object support into one set of functions that has the abstract
nature of VFS and can be called by VFS when accessing filesystems that
support such multi-part storage objects natively, but without VFS itself
having any default dependency on any part of it.

That is the question.


Clayton Weaver

"Everybody's ignorant, just in different subjects." Will Rogers

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
Please read the FAQ at

 \ /
  Last update: 2005-03-22 12:37    [W:0.057 / U:1.228 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site