lkml.org 
[lkml]   [2002]   [Sep]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
SubjectRe: ext3 throughput woes on certain (possibly heavily fragmented) files
Stephen C. Tweedie writes:
> Hi,
>
> On Tue, Sep 03, 2002 at 02:24:19AM -0700, Aaron Lehmann wrote:
>
> > [aaronl@vitelus:~]$ time cat mail/debian-legal > /dev/null
> > cat mail/debian-legal > /dev/null 0.00s user 0.02s system 0% cpu 5.565 total
> > [aaronl@vitelus:~]$ ls -l mail/debian-legal
> > -rw------- 1 aaronl mail 7893525 Sep 3 00:42 mail/debian-legal
> > [aaronl@vitelus:~]$ time cat /usr/src/linux-2.4.18.tar.bz2 > /dev/null
> > cat /usr/src/linux-2.4.18.tar.bz2 > /dev/null 0.00s user 0.10s system 16% cpu 0.616 total
> > [aaronl@vitelus:~]$ ls -l /usr/src/linux-2.4.18.tar.bz2
> > -rw-r--r-- 1 aaronl aaronl 24161675 Apr 14 11:53
> >
> > Both files were AFAIK not in any cache, and they are on the same
> > partition.
> >
> > My current uninformed theory is that this is caused by fragmentation,
> > since the linux tarball was downloaded all at once but the mailbox I'm
> > comparing it to has 1695 messages, each of which having been appended
> > seperately to the file. All of my mailboxes exhibit similarly awful
> > performance.
>
> Yep, both ext2 and ext3 can get badly fragmented by files which are
> closed, reopened and appended to frequently like that.
>
> > Do any other filesystems handle this type of thing more gracefully?
>
> There are some ideas from recent FFS changes. One thing they now do
> is to defragment things automatically as a file grows by effectively
> deleting and then reallocating the last 16 blocks of the file.
> Fragmentation will still occur, but less so, if we do that.
>

Another possible solution is to try to "defer" allocation. For example,
in reiser4 (and XFS, I believe) extents are allocated on the transaction
commit and as a result, if file was created by several writes, it will
still be allocated as one extent.

>
> Cheers,
> Stephen

Nikita.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:22    [W:0.110 / U:1.984 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site