[lkml]   [2008]   [Oct]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectIRQ balancing on a router

    I have a dual-CPU router/firewall with five gigabit NICs. Recently I have
    found that irqbalance (0.55 from Fedora 9/x86_64) gives a suboptimal
    IRQ to CPU mapping on this box:

    During traffic spikes, it assings two NICs to one CPU, and the
    other three to the second CPU. However, this does not account for
    the fact that packets coming from the uplink interface are way more
    expensive to handle than the rest of the traffic: most iptables rules
    apply to the packets received from the uplink interface. The result is
    that the CPU which receives IRQs for the uplink interface
    is 100 % busy (softirq mostly), while the other one is 90% idle.

    Setting the IRQ mapping by hand (uplink to one CPU, all the other
    NICs to the other CPU) makes a well balanced system (both CPUs 30-60 % busy).
    I am not sure whether my configuration is too special, but it might be
    worth trying to make irqbalance daemon cope also with this usage pattern.

    Another problem is that with one CPU 100 % busy in the kernel
    the system latency of user-space programs is _way_ too high. For example,
    MRTG graphs from my router used to have blank stripes (i.e. snmpd
    has failed to respond in time). Also the shell response time was bad,
    even though I was logged in using SSH over the interface, which at that
    time had its IRQ routed to the other CPU.

    With the same network load and manual IRQ to CPU assignment,
    MRTG works well and both snmpd and shell response time is good.


    | Jan "Yenya" Kasprzak <kas at { - work | - private}> |
    | GPG: ID 1024/D3498839 Fingerprint 0D99A7FB206605D7 8B35FCDE05B18A5E |
    | Journal: |
    >> If you find yourself arguing with Alan Cox, you’re _probably_ wrong. <<
    >> --James Morris in "How and Why You Should Become a Kernel Hacker" <<
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2008-10-03 15:23    [W:0.021 / U:5.576 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site