lkml.org 
[lkml]   [2010]   [Jun]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: xfs, 2.6.27=>.32 sync write 10 times slowdown [was: xfs, aacraid 2.6.27 => 2.6.32 results in 6 times slowdown]
09.06.2010 03:18, Dave Chinner wrote:
> On Wed, Jun 09, 2010 at 12:34:00AM +0400, Michael Tokarev wrote:
[]
>> Simple test doing random reads or writes of 4k blocks in a 1Gb
>> file located on an xfs filesystem, Mb/sec:
>>
>> sync direct
>> read write write
>> 2.6.27 xfs 1.17 3.69 3.80
>> 2.6.32 xfs 1.26 0.52 5.10
>> ^^^^
>> 2.6.32 ext3 1.19 4.91 5.02
>>
>> Note the 10 times difference between O_SYNC and O_DIRECT writes
>> in 2.6.32. This is, well, huge difference, and this is where
>> the original slowdown comes from, apparently.
>
> Are you running on the raw block device, or on top of LVM/DM/MD to
> split up the space on the RAID drive? DM+MD have grown barrier
> support since 2.6.27, so it may be that barriers are now being
> passed down to the raid hardware on 2.6.32 and they never were on
> 2.6.27. Can you paste the output of dmesg when the XFS filesystem in

That's why I asked how to tell if barriers are actually hitting the
device in question.

No, this is the only machine where DM/MD is _not_ used. On all other
machines we use MD software raid, this machine comes with an onboard
raid controller that does not work in JBOD mode so I weren't able to
use linux software raid. This is XFS on top of Adaptec RAID card,
nothing in-between.

Also, as I mentioned in the previous email, remounting with nobarrier
makes no difference whatsoever.

(Another side note here - I discovered that unknown options are
silently ignored in "remount mode" while correctly rejected in
"plain mount" mode, -- it looks like a kernel bug actually, but
it's entirely different issue).

> question is mounted on both 2.6.27 and 2.6.32 so we can see if
> there is a difference in the use of barriers?
>
> Also, remember that O_DIRECT does not imply O_SYNC. O_DIRECT writes
> only write data, while O_SYNC will also write metadata and/or the
> log.

I know this. I also found osyncisosync and osyncisdsync mount
options, and when I try to use the latter, kernel tells it's the
default and hence deprecated. I don't need metadata updates, but
it _looks_ like the system is doing such updates (with barriers
or flushes?) anyway even when mounted with -o osyncisdsync it behaves
the same: very slow.

I also experimented with both O_SYNC|O_DIRECT: it is as slow as
without O_DIRECT, i.e. O_SYNC makes whole thing slow regardless
of other options.

I looked at the dmesg outputs, and there's no relevant differences
related to block devices or usage of barriers. For XFS it always
mounts like this:

SGI XFS with ACLs, security attributes, large block/inode numbers, no debug enabled
SGI XFS Quota Management subsystem
XFS mounting filesystem sda6

and for the device in question, it is always like

Adaptec aacraid driver 1.1-5[2456]-ms
aacraid 0000:03:01.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
AAC0: kernel 5.1-0[8832] Feb 1 2006
AAC0: monitor 5.1-0[8832]
AAC0: bios 5.1-0[8832]
AAC0: serial 267BE0
AAC0: Non-DASD support enabled.
AAC0: 64bit support enabled.
AAC0: 64 Bit DAC enabled
scsi0 : aacraid
scsi 0:0:0:0: Direct-Access Adaptec f0500 V1.0 PQ: 0 ANSI: 2
sd 0:0:0:0: [sda] 286715904 512-byte hardware sectors (146799 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 06 00 10 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
sda: sda1 sda2 sda3 < sda5 sda6 >

There are tons of other differences, but that is to be expected (like
format of CPU topology printing which is changed between .27 and .32).

Thanks!

/mjt


\
 
 \ /
  Last update: 2010-06-09 08:45    [W:0.073 / U:0.084 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site