[lkml]   [2008]   [Aug]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Subjectamd64 sata_nv (massive) memory corruption

    I'm seeing strong, easily reproducible (and silent) corruption on a
    disk drive on an amd64 board. It might be the disk itself, but I
    doubt it; googling
    suggests that its somehow iommu-related but I cannot confirm this.

    quickie summary:
    -- disk is a brand new WDC WD5000AAKS-00YGA0 500GB disk (well, it
    was brand new a few months ago -- unusued, at any rate)
    -- passes smartmon with flying colors, including many repeated short and long
    self-tests. Been passing for months. No hint of bad sectors or other errors
    in smartctl -a display
    -- no ide, sata errors in syslog -- no block device errors, no fs errors, etc.
    -- No oopses anywhere to be found
    -- system works flawlessly with an old PATA disk. (although I'm running it
    with dma turned off with hdparm, out of paranoia)
    -- system is amd64 dual core, ASUS M2N-E mobo, 4GB RAM
    Northbridge is nVidia Corporation MCP55 Memory Controller (rev a3)
    -- I tried moving the sata cable around to other ports, no effect; also tried
    reseating it on hard drive, no effect.

    corruption is *easily* observed copying files with cp or dd. Also, typically
    filesystem metadata is corrupted too. Creating even a small ext2 filesystem,
    say 1GB, then copying 300MB of files onto it, unmounting it, and running fsk
    will return many dozens of errors. Rerunning e2fsck over and over (as
    e2fsck -f -y /dev/sda6) will report new errors about 1 out of every 3 times
    (on small fs'es -- on big one's it will find new errors every time)

    This behaviour has been observed with two different kernels:
    with, compiled for 32-bit, and also 2.6.26 complied
    for 64-bit.

    Googling this uncovers some Dec 2006 LKML emails suggesting an
    iommu problem, which I explored:
    -- My default boot complains
    Your BIOS doesn't leave a aperture memory hole
    Please enable the IOMMU option in the BIOS setup
    This costs you 64 MB of RAM
    -- I cannot find any option in BIOS that even vaguely hints at IOMMU-like
    function; at best, I can assign interrupts to PCI slots, but
    that's it. There's
    a bunch of IO options for olde-fashioned superio-like stuff: serial,parallel
    ports, USB stuff, etc. but that's all.
    -- booting with iommu=soft does get rid of the aperature memory hole
    messsage, but does not solve the corruption problem.
    -- booting with iommu=force seems to have no effect.

    I'm running the powernow-k8 cpu frequency regulator. On a hunch,
    I wondered if this might be the source of the problem; however,
    using the "performance" regulator to keep the clock speed nailed
    at maximum had no effect on the corruption bug.

    Also of note:
    -- problem was observed earlier, when system had 3GB RAM in it.
    -- The integrated nvidia ethernet seems to work great, no errors, etc.
    -- A different PCI ethernet card works great too.
    -- I'm running graphics on an anceint matrox card in a PCI slot, and
    there's no hint of trouble there.
    -- I'm using this system as my day-to-day desktop, and there seem to
    be no other problems. This suggests that if its some pci iommu
    wackiness, it certainly not affecting anything that isn't sata.

    I really doubt the problem is the hard-drive; but I'll have to buy another
    one to rule this out. Its possible that there's some problem with the
    sata_nv driver, but there have been historical reports of corruption
    on amd64 with other sata controllers. I can buy another sata controller
    if needed, to experiment.

    Other than that, any ideas for any further experiments? What can
    I do to narrow the problem?

    -- Linas Vepstas

     \ /
      Last update: 2008-08-01 19:33    [W:0.248 / U:4.344 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site