Messages in this thread | | | Date | Sat, 7 Apr 2007 13:58:30 +0200 | From | Willy Tarreau <> | Subject | Re: compressing intermediate files with LZO on the fly |
| |
Hi Al,
On Sat, Apr 07, 2007 at 02:32:34PM +0300, Al Boldi wrote: > Willy Tarreau wrote: > > > > ... for some usages (temporary space), > > light compression can increase speed. For instance, when processing logs, > > I get better speed by compressing intermediate files with LZO on the fly. > > How can you do that on ext3? > > Also, can you do that on a partition block-io level?
No, sorry for the confusion. My scripts simply do :
$ lzop -cd file1.lzo | process | lzop -c3 > file2.lzo
With decent CPU, you can reach higher read/write data rates than what a single off-the-shelf disk can achieve. For this reason, I think that reiser4 would be worth trying for this particular usage. And in this case, I'm not interested at all in reliability. It's just temporary storage. If the disk fails, I throw it away and buy a new one.
Cheers, Willy
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |