[lkml]   [2002]   [Sep]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: ext3 throughput woes on certain (possibly heavily fragmented) files
On Sep 16, 2002  15:39 -0700, Simon Kirby wrote:
> We recently switched a large mail spool from ext2 to ext3 with default
> journalling, and we are now having huge problems with disk I/O load.
> We have fsync and friends disabled for performance reasons. With ext2,
> the machine would happily hum along with an average load of 0.2 and a
> usual 400 kB - 800 kB write every 5 seconds, with about 10 kB/sec read in
> every second.
> This box is primarily running a POP3 server (written in-house to cache
> mbox offsets, so that it can handle a huge volume of mail), and also
> exports the mail spool via NFS to other servers which run exim (-fsync).
> nfsd is exported async. Everything is mounted noatime, nodiratime. No
> applications should be calling sync/fsync/fdatasync or using O_SYNC.
> It's a mail server, so everything is fragmented.

Hmm, it seems strange (and rather unsafe) that you would run a mail
spool without using sync I/O. Unfortunate too, because sync I/O with
a large journal (and perhaps an external journal disk) would give you
very fast throughput on ext3.

> We're using dotlocking. Would this cause metadata journalling?
> I estimate about 200 - 300 dotlock files are created per second, but
> these should all be asynchronous.

Lots of it. So, that is 250 * (1 dir block + 1 inode bitmap + 1 inode
table block (+ 1 block bitmap + 1 data block, if there is data in the
dotlock file)) = 1250 blocks/second or so.

> Would switching to fctnl() locking (if this works over NFS) solve the
> problem?

Probably (no disk I/O generated), but I don't know the state of NFS locking.

> We had to hash the mail spool a long time ago do to system time eating all
> CPU (the ext2 linear directory scan to find a slot available in the spool
> directory to add the dotlock file).

One reason why we are adding hash-indexed directory support to ext3, so
that you don't have to implement it in each application.

Cheers, Andreas
Andreas Dilger

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:28    [W:0.126 / U:0.040 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site