lkml.org 
[lkml]   [2008]   [Oct]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: IRQ balancing on a router
    On Fri, Oct 03, 2008 at 06:38:57AM -0700, Arjan van de Ven wrote:
    > > Hello,
    > >
    > > I have a dual-CPU router/firewall with five gigabit NICs. Recently I
    > > have found that irqbalance (0.55 from Fedora 9/x86_64) gives a
    > > suboptimal IRQ to CPU mapping on this box:
    > >
    > > During traffic spikes, it assings two NICs to one CPU, and the
    > > other three to the second CPU. However, this does not account for
    > > the fact that packets coming from the uplink interface are way more
    > > expensive to handle than the rest of the traffic: most iptables rules
    > > apply to the packets received from the uplink interface. The result is
    > > that the CPU which receives IRQs for the uplink interface
    > > is 100 % busy (softirq mostly), while the other one is 90% idle.
    > >
    > > Setting the IRQ mapping by hand (uplink to one CPU, all the
    > > other NICs to the other CPU) makes a well balanced system (both CPUs
    > > 30-60 % busy). I am not sure whether my configuration is too special,
    > > but it might be worth trying to make irqbalance daemon cope also with
    > > this usage pattern.
    > >
    >
    > one of the hard cases for irqbalance is that irqbalance doesn't have a
    > way to find out the actual cpu time spend in the handlers. For
    > networking it makes an estimate just based on the number of packets
    > (which is better than nothing)... but that breaks down if you have an
    > non-symmetry in CPU costs per packet like you have.
    >
    > The good news is that irqthreads at least have the potential to solve
    > this "lack of information"; if not, we could consider doing a form of
    > microaccounting for irq handlers....
    >
    >

    perhaps, this could be addressed using tracepoints. The currently
    proposed ones are at the beginning and end of 'handle_IRQ_event()'. See:
    http://marc.info/?l=linux-kernel&m=121616099830280&w=2

    thanks,

    -Jason


    \
     
     \ /
      Last update: 2008-10-03 16:33    [W:0.021 / U:0.564 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site