lkml.org 
[lkml]   [2009]   [May]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: 2.6.30-rc deadline scheduler performance regression for iozone over NFS
    Date
    Trond Myklebust <trond.myklebust@fys.uio.no> writes:

    > On Thu, 2009-05-14 at 09:34 -0400, Jeff Moyer wrote:
    >> Trond Myklebust <trond.myklebust@fys.uio.no> writes:
    >>
    >> > On Wed, 2009-05-13 at 15:29 -0400, Jeff Moyer wrote:
    >> >> Hi, netdev folks. The summary here is:
    >> >>
    >> >> A patch added in the 2.6.30 development cycle caused a performance
    >> >> regression in my NFS iozone testing. The patch in question is the
    >> >> following:
    >> >>
    >> >> commit 47a14ef1af48c696b214ac168f056ddc79793d0e
    >> >> Author: Olga Kornievskaia <aglo@citi.umich.edu>
    >> >> Date: Tue Oct 21 14:13:47 2008 -0400
    >> >>
    >> >> svcrpc: take advantage of tcp autotuning
    >> >>
    >> >> which is also quoted below. Using 8 nfsd threads, a single client doing
    >> >> 2GB of streaming read I/O goes from 107590 KB/s under 2.6.29 to 65558
    >> >> KB/s under 2.6.30-rc4. I also see more run to run variation under
    >> >> 2.6.30-rc4 using the deadline I/O scheduler on the server. That
    >> >> variation disappears (as does the performance regression) when reverting
    >> >> the above commit.
    >> >
    >> > It looks to me as if we've got a bug in the svc_tcp_has_wspace() helper
    >> > function. I can see no reason why we should stop processing new incoming
    >> > RPC requests just because the send buffer happens to be 2/3 full. If we
    >> > see that we have space for another reply, then we should just go for it.
    >> > OTOH, we do want to ensure that the SOCK_NOSPACE flag remains set, so
    >> > that the TCP layer knows that we're congested, and that we'd like it to
    >> > increase the send window size, please.
    >> >
    >> > Could you therefore please see if the following (untested) patch helps?
    >>
    >> I'm seeing slightly better results with the patch:
    >>
    >> 71548
    >> 75987
    >> 71557
    >> 87432
    >> 83538
    >>
    >> But that's still not up to the speeds we saw under 2.6.29. The packet
    >> capture for one run can be found here:
    >> http://people.redhat.com/jmoyer/trond.pcap.bz2
    >>
    >> Cheers,
    >> Jeff
    >
    > Yes. Something is very wrong there...
    >
    > See for instance frame 1195, where the client finishes sending a whole
    > series of READ requests, and we go into a flurry of ACKs passing
    > backwards and forwards, but no data. It looks as if the NFS server isn't
    > processing anything, probably because the window size falls afoul of the
    > svc_tcp_has_wspace()...
    >
    > Does something like this help?

    Is this in addition to the previous patch or instead of it?

    Cheers,
    Jeff


    \
     
     \ /
      Last update: 2009-05-14 16:43    [W:2.883 / U:0.688 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site