Messages in this thread | | | Date | Fri, 5 Dec 2003 11:16:51 -0500 | From | Erez Zadok <> | Subject | Re: partially encrypted filesystem |
| |
In message <20031205112050.GA29975@wohnheim.fh-wedel.de>, =?iso-8859-1?Q?J=F6rn?= Engel writes: > On Thu, 4 December 2003 14:41:48 -0500, Erez Zadok wrote: > > > > Part of our stackable f/s project (FiST) includes a Gzipfs stackable > > compression f/s. There was a paper on it in Usenix 2001 and there's code in > > the latest fistgen package. See > > http://www1.cs.columbia.edu/~ezk/research/fist/ > > > > Performance of Gzipfs is another matter, esp. for writes in the middle of > > files. :-) > > You don't seriously treat big files as a single gzip stream, do you? > ;)
Of course not. We're not *that* bad. :-)
We compress each chunk separately; currently chunk==PAGE_CACHE_SIZE. For each file foo we keep an index file foo.idx that records the offsets in the main file of where you might find the decompressed data for page N. Then we hook it up into the page read/write ops of the VFS. It works great for the most common file access patterns: small files, sequential/random reads, and sequential writes. But, it works poorly for random writes into large files, b/c we have to decompress and re-compress the data past the point of writing. Our paper provides a lot of benchmarks results showing performance and resulting space consumption under various scenarios.
We've got some ideas on how to improve performance for writes-in-the-middle, but they may hurt performance for common cases. Essentially we have to go for some sort of O(log n)-like data structure, which'd make random writes much better. But since it may hurt performance for other access patterns, we've been thinking about some way to support both modes and be able to switch b/t the two modes on the fly (or at least let users "mark" a file as one for which you'd expect a lot of random writes to happen).
If anyone has some comments or suggestions, we'd love to hear them.
BTW, our support in the stackable templates (described in the paper) is more general than just for compression. We support any (monotonically continuous) size-changing algorithm: compression, encryption, etc. In our paper we also show a size-expanding file system called uuencodefs.
Cheers, Erez. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |