[lkml]   [1998]   [Sep]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: (reiserfs) Re: Implementing Meta File information in Linux (and a note at the end on current reiserfs status)
Hans Reiser writes:
> Richard Gooch wrote:
> > Can you explain why you think that keeping the metadata in a normal
> > file and splitting the data streams in userspace is going to be slow?
> Imagine a million small pieces of metadata. Much better to make a
> directory out of it,and let the FS optimize the storage and indexing
> and caching and everything else. The FS is not going to give your
> storage layer enough control via the VFS interface for your storage
> to do its job as well as the FS could.

Fine. Note that I did show that my assumptions were not many streams
and the secondary streams where small, for the case where I'm arguing
in favour of keeping it all in a single file with no kernel/FS tricks.

> > I would have expected the file would have a "header" which describes a
> > series of data streams, including file offsets. Putting the non-data
> > metadata at the front would allow the normal data component to grow
> > without any problems. Even if more metadata was added later, it would
> > likely be small, so it could be easily cached and appended as the
> > normal data grew. Accessing a particular stream is just a seek
> > operation.
> What about insertion? This is the killer issue...... Do you want to
> rewrite the entire1GB file to insert 100 bytes?

Again, I expressly ignored this scenario in my assumptions.

As I said:

> > Of course, if you have large metadata, then you're better off making
> > each stream as a file in a directory and letting the FS do the work.
> > The FS code already has one bunch of code to implement growing streams
> > (files), I think we'd want to leverage that instead of implementing
> > yet another level of what is effectively file/directory management
> > inside the FS.

To clarify: I see two situations. The first is where you have a small
number of secondary streams of small size which don't grow much. Here
a single file can suffice.
The second case is where these assumptions break down, and in that
case I think each stream belongs in a file. A userspace API is written
to allow you to deal with the dataset as a single object. For
consistency, even where the above assumptions hold, it's probably
better to use one file per stream.
I see directories being excellent placeholders for grouping
metadata. With reiserFS, this userspace implementation will be very
fast, so it's the logical way to go, IMO.



To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:44    [W:0.171 / U:0.968 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site