[lkml]   [2007]   [Nov]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Subjectunionfs: several more problems
    I think it may be time to move away from that by now misleading
    "msync(2) bug(?), returns AOP_WRITEPAGE_ACTIVATE to userland" thread.
    And I've cut most out of the Cc list, but by all means add any back.

    On Saturday I said:
    > I'm glad to report that this unionfs, not the one in 2.6.24-rc2-mm1
    > but the one including those 9 patches you posted, now gets through
    > my testing with tmpfs without a problem. I do still get occasional
    > "unionfs: new lower inode mtime (bindex=0, name=<directory>)"
    > messages, but nothing worse seen yet: a big improvement.

    I have seen worse now: details below.

    > I deceived myself for a while that the danger of shmem_writepage
    > hitting its BUG_ON(entry->val) was dealt with too; but that's wrong,
    > I must go back to working out an escape from that one (despite never
    > seeing it).

    Once I tried a more appropriate test (fsx while swapping) I hit that
    easily. After some thought and testing, I'm happy with the mm/shmem.c
    +mm/swap_state.c fixes I've arrived at for that; but since it's not
    easy to reproduce in normal usage, and hasn't been holding you up,
    I'd prefer for the moment to hold on to that patch. I need to make
    changes around the same pagecache<->swapcache area to solve some mem
    cgroup issues: there might turn out to be some interaction, so I'd
    rather finalize both patches in the same series if I can.

    For a while I thought the further unionfs problems I'm seeing were
    tmpfs ones, but now I seem to be seeing all(?) of them with ext2:
    so I'd better spend no more time on them, hand them over to you.

    Please try running LTP (e.g. ltp-full-20071031.tgz) after something like
    mkfs -t ext2 /dev/sdb1
    mount -t ext2 /dev/sdb1 /mnt
    mount -t unionfs -o dirs=/mnt unionfs /tmp

    That throws up quite a number of errors which don't occur
    with a straightforward ext2 mounted on /tmp. Notably, when it
    mkdir seems to collapse into lots of
    mkdir: cannot create directory `dirN': No space left on device
    mkdir dirN FAILED
    (the same happened when I was using a tmpfs instead of an ext2).

    But perhaps before fixing up the several LTP tests, you'll want
    to concentrate on a more directed test. Please try this sequence:

    # Running with mem=512M, probably irrelevant
    swapoff -a # Merely to rule out one potential confusion
    mkfs -t ext2 /dev/sdb1
    mount -t ext2 /dev/sdb1 /mnt
    df /mnt # I have 2280 Used out of 1517920 KB
    cp -a 2.6.24-rc3 /mnt # Copy a kernel source tree into ext2
    rm -rf /mnt/2.6.24-rc3 # Delete the copy
    df /mnt # Again 2280 Used, just as you'd expect
    mount -t unionfs -o dirs=/mnt unionfs /tmp
    cp -a 2.6.24-rc3 /tmp # Copy a kernel source tree into unionfs
    rm -rf /tmp/2.6.24-rc3 # Generates 176 unionfs: filldir error messages
    df /mnt # Now 68380 Used (df /tmp shows the same)
    ls -a /mnt # Shows . .. .wh.2.6.24-rc3 lost+found
    echo 1 >/proc/sys/vm/drop_caches # to free pagecache
    df /mnt # Still 68380 Used (df /tmp shows the same)
    echo 2 >/proc/sys/vm/drop_caches # to free dentries and inodes
    df /mnt # Now 2280 Used as it should be (df /tmp same)
    ls -a /mnt # But still shows that .wh.2.6.24-rc3
    umount /tmp # Restore
    umount /mnt # Restore
    swapon -a # Restore

    Three different problems there:

    1. Whiteouts seem to get left behind (at this top level anyway):
    I'm getting an increasing number of files there.
    I'm not familiar with the correct behaviour for whiteouts (and it's
    not clear to me why whiteouts are needed at all in this degenerate
    case of a single directory in the union, but never mind).

    2. Why does copying then deleting a tree leave blocks allocated,
    which remain allocated indefinitely, until memory pressure or
    drop_caches removes them? Hmm, I should have done "df -i" instead,
    that would be more revealing. This may well be the same as the LTP
    mkdir problem - inodes remaining half-allocated after they're unlinked.

    3. The unionfs filldir error messages:
    unionfs: filldir: possible I/O error: a file is duplicated in the same
    branch 0: page_offset.h
    <... 174 more include/asm-* header files named until ...>
    unionfs: filldir: possible I/O error: a file is duplicated in the same
    branch 0: sfafsr.h
    It's tempting to suppose these are related to the unfreed inodes, but
    retrying again gives me 176 messages, whereas inodes fall from 2672
    to 30. And I didn't see those messages with tmpfs, just with ext2.

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2007-11-20 20:19    [W:0.066 / U:24.576 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site