lkml.org 
[lkml]   [2011]   [Feb]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: NFS Regression in commit 0b26a0bf6ff398
    On Tue, 15 Feb 2011, Trond Myklebust wrote:

    > On Tue, 2011-02-15 at 19:56 -0500, Eric B Munson wrote:
    > > While testing some 2.6.38 work my rsync backup script started consuming
    > > large amounts of memory (all available before dying with no more available
    > > memory). I have bisected the problem back to 0b26a0bf6ff398. I am
    > > unfamiliar with the NFS code so I don't know where to start looking for a
    > > possible fix. My backups files from my home directory to an NFS mounted
    > > directory. The NFS server is a Synology DS-411+ if it matters. Let me know
    > > if there is any other information I can provide.
    >
    > Exactly which 2.6.38 kernel are you running, and which NFS version?
    >
    > I'm having trouble seeing how the patch in question can be responsible
    > for what you are seeing, so please could you provide more details of
    > your test setup.
    >

    Trond,

    I just updated to 2.6.38-rc6 and I still see the regression. I will add some
    more information here that might be useful.

    The entries from ps -ef:

    user 3116 3096 0 14:45 pts/4 00:00:00 /bin/bash /home/emunson/bin/bu
    user 3117 3116 3 14:45 pts/4 00:00:05 rsync -avz --delete --exclude=*.
    user 3118 3117 26 14:45 pts/4 00:00:41 rsync -avz --delete --exclude=*.
    user 3119 3118 0 14:45 pts/4 00:00:01 rsync -avz --delete --exclude=*.

    strace from 3117:
    user@machine:~$ sudo strace -p 3117
    Process 3117 attached - interrupt to quit
    select(6, [5], [], NULL, {33, 382426}) = 0 (Timeout)
    select(6, [5], [], NULL, {60, 0}) = 0 (Timeout)
    select(6, [5], [], NULL, {60, 0}) = 0 (Timeout)
    select(6, [5], [], NULL, {60, 0}) = 0 (Timeout)
    select(6, [5], [], NULL, {60, 0}

    strace from 3118:
    ...
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    lstat("linux-2.6/net/netfilter/xt_dscp.c", {st_mode=S_IFREG|0644, st_size=2890, ...}) = 0
    ...

    These stream by very quickly.

    and finally from 3119:
    user@machine:~$ sudo strace -p 3119
    Process 3119 attached - interrupt to quit
    select(5, NULL, [4], [4], {15, 621031}) = 0 (Timeout)
    select(5, NULL, [4], [4], {60, 0}) = 0 (Timeout)
    select(5, NULL, [4], [4], {60, 0}) = 0 (Timeout)
    select(5, NULL, [4], [4], {60, 0}) = 0 (Timeout)
    select(5, NULL, [4], [4], {60, 0}

    Now from my /etc/fstab:

    /dev/mapper/isw_gbdfddifh_Volume02 / ext4 errors=remount-ro 0 1
    /dev/mapper/isw_gbdfddifh_Volume03 none swap sw 0 0

    #NFS
    192.168.1.50:/volume1/backup /mnt/backup nfs rsize=1048576,wsize=1048576,user 0 0
    192.168.1.50:/volume1/data /mnt/data nfs rsize=1048576,wsize=1048576,user 0 0
    192.168.1.50:/volume1/music /mp3 nfs rsize=1048576,wsize=1048576,user 0 0
    192.168.1.50:/volume1/video /video nfs rsize=1048576,wsize=1048576,user 0 0

    The backup script is reading from my home dir (locally mounted on partition 2 of a
    fake raid stripe). And writing to /mnt/backup/bert-ubuntu.

    And here is the error I get when rsync finally dies:

    ERROR: out of memory in flist_expand [generator]
    rsync error: error allocating core memory buffers (code 22) at util.c(117) [generator=3.0.7]
    rsync: connection unexpectedly closed (9324 bytes received so far) [sender]
    rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]

    At the point when this happens, rsync is consuming almost all of the free memory on the system.

    I am nost sure if there is anything else that might help, please let me know if you need more
    information.
    [unhandled content-type:application/pgp-signature]
    \
     
     \ /
      Last update: 2011-02-24 21:01    [W:0.027 / U:30.148 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site