Messages in this thread | | | Date | Tue, 6 Jul 1999 16:48:36 +0200 | From | jan@gondor ... | Subject | Re: Another way |
| |
On Tue, Jul 06, 1999 at 09:16:24AM -0400, Lou Grinzo wrote: > If the system has enough memory available, the userland lib > can read the whole file into memory upon open, hold the file
This probably doesn't scale well enough. You will allways have files bigger than your (free) memory. And you will have writes of huge documents only to append a few bytes. (Look at mail readers)
> an explicit flush or close. This eliminates the waste of Word's > fast save and dead space, and can be quite quick.
What about a new kernel feature, touching all abstraction levels from fs implementation to user space API :-) :
Files that can grow (or shrink) in the middle.
Then, at application level, we can have a file that contains of several data streams, and we only have to care about the starting points (and sizes) of the streams - or whatever the application wants to store. And we still have a single stream file as a whole, that can be moved around by existing tools. And we can access this file on systems that do not support this scheme (but only with more application code and less performance)
I have some ideas how to implement this scheme, but they are rather long and probably not complete. If you don't like this scheme at all, you can stop reading here :-)
The only way to support this I can currently imagine is some kind of allocation list per file.
In this scheme, on the lowest level, a file is still a stream of bytes. But it has some kind of header saying (for example):
'Of this 10240 bytes long file, use only bytes 0-3999 and 5000-10000'.
When I write to the end of this file (that is, at file position 9200) this changes to 'Of this 10240 bytes long file, use only bytes 0-3999 and 5000-10200'. and finally 'Of this 11264 bytes long file, use only bytes 0-3999 and 5000-10300'.
That's easy. When I insert bytes after the 'hole' at 3999 it changes to: 'Of this 11264 bytes long file, use only bytes 0-4299 and 5000-10300'. and finally combines to: 'Of this 11264 bytes long file, use only bytes 0-10300'.
The difficult part is inserting at a position where no hole is available. But this can be done with the following steps: Imagine, we want to insert at position 1000. (That is not at a block boundary. For simplicity, I assume block boundaries at 1024 bytes)
First, we insert an unused block at 1024, and our usage pattern changes to:
'Of this 12280 bytes long file, use only bytes 0-1023, 2048-10300'.
This doesn't change the files contents, and doesn't need any copying. But we still cannot insert at 1000. So we have to move some bytes (but allways less than one allocation unit) and write: 'Of this 12280 bytes long file, use only bytes 0-999, 2024-10300'.
Now we have a hole at 1000 that we can fill like above.
That's it. Comments?
Bye, Jan
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |