lkml.org 
[lkml]   [2003]   [Dec]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: partially encrypted filesystem
Matthew Wilcox wrote:
> On Fri, Dec 05, 2003 at 02:47:56PM -0500, Erez Zadok wrote:
>
>>Thanks for the info, Matthew. Yes, clearly a scheme that keeps some "holes"
>>in compressed files can help; one of our ideas was to leave sparse holes
>>every N blocks, exactly for this kind of expansion, and to update the index
>>file's format to record where the spaces are (so we can efficiently
>>calculate how many holes we need to consume upon a new write).

FYI, Acorn's scheme was described in "Compressed Executables: An
Exercise in Thinking Small" by Mark Taunton, in the Usenix Spring '91
conference, it doesn't seem to be online, but a search on google groups
for "group:comp.unix.internals taunton compressed executables" brings up
a description. I used to work with Mark Taunton at Acorn.

>
> But the genius is that you don't need to calculate anything. If the
> data block turns out to be incompressible (those damn .tar.bz2s!), you
> just write the block in-place. If it is compressible, you write as much
> into that block's entry as you need and leave a gap. The underlying
> file system doesn't write any data there. There's no need for an index
> file -- you know exactly where to start reading each block.
>

Of course this is all being done at the file level, which relies on
proper support of holes in the underlying filesystem (which Acorn's BSD
FFS filesystem did). FiST's scheme is much more how it would be
implemented without hole support, where you *have* to pack the data,
otherwise the "unused" space would physically consume disk blocks. In
this case an index to find the start of each compressed block is essential.

I'm guessing that FiST lacks support for holes or data insertion in the
filesystem model, which explains why on writing to the middle of a
file, the entire file from that point has to be re-written.

Of course, all this is at the logical file level, and ignores the
physical blocks on disk. All filesystems assume physical data blocks
can be updated in place. With compression it is possible a new physical
block has to be found, especially if blocks are highly packed and not
aligned to block boundaries. I expect this is at least partially why
JFFS2 is a log structured filesystem.

Phillip

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:59    [W:0.151 / U:0.832 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site