[lkml]   [2004]   [Aug]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why
    The Idea

    You should be able to access metadata about a file the same way you
    access the file's data, but with a name based on the filename followed
    by a name to select the metadata of interest.


    cat song_of_silence/metas/owner
    cat song_of_silence/metas/permissions
    cat 10 > song_of_silence/metas/mixer_defaults/volume
    cat song_of_silence/metas/license

    But wait! This idea has flaws. Let us list them and address them:

    Problem: Often we want to access more than one metafile at a time, and
    a separate system call for each of them is less efficient than stat or

    Solution: Create a system call that can access more than one file at a
    time, whether it is a metafile or a data file should be irrelevant.

    Problem: Many kinds of metadata need to be updated atomically.

    Solution: Atomic filesystem, which allows all files, including data
    files, to be updated atomically.

    Problem: Many kinds of metadata need to be constrained in their
    allowed values.

    Solution: Allow general purpose constraints to constrain all files,
    including data files also.

    Problem: Many kinds of metadata should not be visible to the user
    unless he wants to see them.

    Solution: Allow all files to be hidden from the user unless he wants
    to see them.

    Example: metas subdirectory is invisible to readdir

    Problem: this approach can pollute the namespace

    Answer: choose obscure names

    Problem (all credit to Mr. Demidov for identifying this problem, I
    argued the other viewpoint, and I can only claim the wisdom to know
    that I lost the argument): names like "..metas" are ugly to new users,
    who don't really care for languages that use punctuation in their


    don't make them too obscure, experienced namespace developers know
    that the problem of polluting the namespace is not really as big a
    deal as beginners think it is, and Clearcase and the WAFL filesystem
    manage to get by just fine, whereas the problem of putting punctuation
    marks into names and syntax is a big deal for newbies to the system.
    Name it "metas" not "..metas", and users will never experience it as a
    real problem, and newbies will never be annoyed by a-rhythmic
    punctuation. Note: if Linus disagrees, it is not the most important
    thing in this design, "..metas" isn't the end of the world.

    Problem: Many kinds of metadata, such as acls, need to be inherited.
    Solution: Allow all files, metafiles and data files both, to be
    inherited from other files.

    The Idea 2:

    When implementing a file plugin, very often there are methods supplied
    by that plugin that are unique to that plugin. These methods are
    often well interfaced with as files that can be written to or read
    from. Allow file plugins to define pseudo files that perform these


    cat complex_document/pseudos/glued gives an archive of the directory
    "complex_document" and everything below it, suitable for emailing.

    cat filename/pseudos/backup gives a set of commands that a backup
    command can use to create the file "filename" with all of its state
    preserved, without the backup program having to understand the
    plugin's internals. (If there is going to be a proliferation of file
    plugins, then backup plugin methods are a necessity.

    cat password_directory/pseudos/cat gives a concatenation of all the
    files in the directory "password_directory".

    Why Is This Design Important?

    It is all about namespace fragmentation. If you can access everything
    in the OS from the same namespace, the OS is much more powerful.

    The expressive power of an OS is not proportional to how many objects
    are in the OS, but rather to how many different ways you can combine
    them. Separate namespaces for different types of objects is like
    having different railroad track gauges for the different peoples of the

    If there are different namespaces for 2 sets of objects, then the
    number of possible combinations of the objects is (N/2)^2, which is
    less than the number of combinations of the objects if they are in one
    namespace, which is N^2.

    Why Openat Sucks:

    because you can't go cat filenameA/metas/permissions > filenameB/permissions

    If cat doesn't work, then we are suffering exactly the problem with
    namespace fragmentation that this whole scheme was invented to avoid.

    The Origins Of This Idea

    I disliked stat() way back in 1984. This is just one small piece of
    my efforts to build tools to allow OS architects to unify namespaces
    throughout the OS. DARPA funded this under the CHATS program, and it
    was one of the main components of my proposal.


    One piece at a time, I would like to let this mature as a reiser4 only
    prototype throughout the 2.7 kernel series, and then have it reviewed
    for use by other filesystems after that. Let us not rush it, please.
    I have been studying namespace design since 1984, let someone who is
    deeply familiar with namespace design principles have a try at a
    coherent design with all the pieces in place first before committee
    redesign is attempted. Please consider that there are advantages to
    letting the guy who had the idea and did the work to make it happen be
    the maintainer.

    My approach is to first produce a working prototype, and only then to
    try to make a standard. It is not an unusual approach in our
    community. Let me take that path, it works better for persons like
    me. I have a half working prototype. A little more time please.
    After 30 years of Unix file system semantics stagnation, there is not
    a great rush for this little piece of mild innovation I think.


    Namespace fragmentation is the most important single determinant of OS
    expressive power, just as trade fragmentation is the most important
    single determinant of national wealth (see Adam Smith's The Wealth of
    Nations). Making objects combine easily is what it is all about.

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 14:05    [W:0.026 / U:210.544 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site