lkml.org 
[lkml]   [2003]   [Oct]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Transparent compression in the FS
On Thu, Oct 16, 2003 at 06:29:26PM +0200, Andrea Arcangeli wrote:
> Hi Jeff,
>
> On Wed, Oct 15, 2003 at 11:13:27AM -0400, Jeff Garzik wrote:
> > Josh and others should take a look at Plan9's venti file storage method
> > -- archival storage is a series of unordered blocks, all of which are
> > indexed by the sha1 hash of their contents. This magically coalesces
> > all duplicate blocks by its very nature, including the loooooong runs of
> > zeroes that you'll find in many filesystems. I bet savings on "all
> > bytes in this block are zero" are worth a bunch right there.
>
> I had a few ideas on the above.
>
> if the zero blocks are the problem, there's a tool called zum that nukes
> them and replaces them with holes. I use it sometime, example:

With the exception of NUL blocks i doubt there is much
savings at any level below the file. That is, only a tiny
fraction of files that are not fully duplicate of one
another will share many aligned blocks.

Now detecting files that are duplicates and linking them in
some way might be a useful in a low-priority daemon. But
the links created would have to be sure to preserve them as
seperate inodes so that overwrites break the loose link but
not the user-created hardlink.


--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: jw@pegasys.ws

Remember Cernan and Schmitt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:58    [W:0.175 / U:1.248 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site