lkml.org 
[lkml]   [2000]   [Aug]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: NTFS-like streams?
I'll return this to the mailing list (I didn't press the right button)

Michael Rothwell <rothwell@holly-springs.nc.us>:
> Jesse Pollard wrote:
>
> > I'm not directly opposed to streams, though it isn't a posix type of file,
> > and does look more like it should be a data manager type of operation than
> > a file system.
>
> Linus and I are arguing for a standard way of supporting
> streams for filesystems that already support them. Currently,
> there are four:
>
> HTFS, HFS, BeFS, HPFS. Rumor has it that XFS supports EAs,
> which can be done on top of generic streams functionality.
>
> Can you think of a way to implement standardized access to
> streams/EAs on Linux? I want to stay clear of supporting
> streams on FSes that have no concept of them (like Ext2) --
> that argument is, at the moment, a black pit. Plus, if
> Linux gains a stackable filesystem interface, streams/EAs
> _can_ be added to existing filesystems without breaking them.

Actually ... I think so. - stackable file systems that would support it?
no I don't think so. The resulting file would only show up on the top
(last mounted) file system. Two different systems mounting from the
same file server will get different results from the file. That means
data IS lost, even if it is only the alternate streams. Most of this
thread keeps assuming that "the alternate streams are dropped" is a
valid thing to do. LOSING DATA IS NEVER A VALID ASSUMPTION. If you can't
keep the data.. then why use the unreliable system....

The following skeleton file system layout is a way to implement it:

So what everybody seems to be promoting is something like the following:

inode
+------------------+
| mode (complex) |
| owner |
| size |
| ... | +----------------+
| ptr->default |-------->| direct block |
| ptr->nam1: name |------+ | ptr->1st block |
| ptr->nam2: name2 |---+ | | ptr->2ed block |
| ptr->nam3: name3 |-+ | | | ... | +--------------------+
| ... | | | | | ptr->2ed level |----->| second level block |
| ptr->namn: namen |-+ | | +----------------+ | ptr->1st sec level |
| ptr->extend | | | | | ptr->2ed sec level |
+------------------+ | | | +----------------+ | .... |
| | | +->| direct block | | ptr->3rd level |
| | | | ptr->1st block | +--------------------+
| | | | ptr->2ed block |
| | | | ... | +--------------------+
| | | | ptr->2ed level |----->| second level block |
| | | +----------------+ | ptr->1st sec level |
| | | | ptr->2ed sec level |
| | | +----------------+ | .... |
| | +---->| direct block | | ptr->3rd level |
| | | ptr->1st block | +--------------------+
| | | ptr->2ed block |
| | | ... | +--------------------+
| | | ptr->2ed level |----->| second level block |
| | +----------------+ | ptr->1st sec level |
| | | ptr->2ed sec level |
| +-> and so forth | .... |
| | ptr->3rd level |
V +--------------------+
+----------------------+
| EA extended... |
| ptr->namn+1:name n+1 |---> (repeat of the first)
| ptr->namn+2:name n+2 |---> (repeat of the first)
| .... |
| (null) |
+----------------------+

First some notes:

1. This is only an outline.

2. the "ptr->nam1:name 1" construct is to be interpreted as:

a pointer to the first level direct data block pointers associated
with stream "name 1".

3. the "ptr->default" may or may not have a name associated with it. The
first pointer entry is always the default.

The major differences between the current UNIX file system is that the
default pointers, and EA pointers take up the space in the inode, where
small files used to go. The small files here would have to go in the
first direct block instead, pointed to by ptr->default. The same applies
to each EA. If the entire contents will fit in the direct block, then
it should be stored there.

NOTE: there should be SOME form of length stored in the "ptr->nam1:name 1"
construct. Possibly the bytelength and block count of the size of the stream.

This will force the file to occupy a minimum of two blocks: one inode,
one direct block.

After the default pointer (say 64 bit minimum) the rest of the inode
(except the last 64 bit... though this could, in reality be imediately
following the default pointer) is used for pointer:name pairs (just as
in a directory, but it's not a directory). When the end of the inode
is reached (if more data follows) then the ptr->extend pointer is used
to point to a "next EA block". These EA blocks could be searched linearly
since there isn't/shouldn't be that many, a file metadata tree shouldn't
be needed.

Each EA would point to a file metadata tree the same as the default pointer.
This would allow the file + EA to be as large as necessary.

Looking back at this, I suspect that ALL regular files should look the
same, whether there is an alternate data stream associated or not.

The problem with making the secondary data stream point to an inode is
that fsck will have no (nearly) way to repair a damaged file that occupies
two inodes. The second (and subsequent) inodes will almost always be seen
as a "lost" file.

The way diagramed above, there is no second inode.

No support for hard (or soft) links is provided (soft might be done... if
there are some "mode" bits associated with each EA...).

It would make things simpler if the EA name could be a fixed length
(8 or 16 bytes). It would make the search much simpler.

Side notes:

1 The VFS functions could lookup the stream name IF the path provided
("file/stream") construct is used. The moment the "file" inode is loaded,
it will be obvious that the "/stream" part refers to a stream name.
2 The namei function will actually get more complex - it must also be able
to search the EA list as well as be able to search directories. At a minimum
namei will have to invoke a "nameEA" funtion to finish the search.
3 There has to be some way to backup the entire filesystem - including all
streams. This capability must also be available to users.
4 ls will have to have another non-posix option - list EA references with
sizes? do we also need dates (access/modified/created) on each stream?
I assume here that a modification to a stream would only change the
modification time on the base file.

If nothing else, this may generate a LOT of additional discussion

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:58    [W:0.037 / U:1.252 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site