lkml.org 
[lkml]   [2010]   [Jul]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: Interrupt Affinity in SMP
    From
    On Sun, Jul 18, 2010 at 11:38 AM, Ciju Rajan K <ciju@linux.vnet.ibm.com> wrote:
    > Bryan Hundven wrote:
    >>
    >> On Sat, Jul 10, 2010 at 6:20 PM, Robert Hancock <hancockrwd@gmail.com>
    >> wrote:
    >>
    >>>
    >>> On Sat, Jul 10, 2010 at 1:46 PM, Bryan Hundven <bryanhundven@gmail.com>
    >>> wrote:
    >>>
    >>>>
    >>>> I was able to set eth0 and it's TxRx queues to cpu1, but it is my
    >>>> understanding that 0xFFFFFFFF should distribute the interrupts across
    >>>> all
    >>>> cpus, much like LOC in my output of /proc/interrupts.
    >>>>
    >>>> I don't have access to the computer this weekend, but I will provide
    >>>> more
    >>>> info on Monday.
    >>>>
    >>>
    >>> That may be chipset dependent, I don't think all chipsets have the
    >>> ability to distribute the interrupts like that. Round-robin interrupt
    >>> distribution for a given handler isn't optimal for performance anyway
    >>> since it causes the relevant cache lines for the interrupt handler to
    >>> be ping-ponged between the different CPUs.
    >>>
    >>>
    >>>>
    >>>> -bryan
    >>>>
    >>>> On Jul 9, 2010 5:48 PM, "Robert Hancock" <hancockrwd@gmail.com> wrote:
    >>>>
    >>>> On 07/09/2010 04:59 PM, Bryan Hundven wrote:
    >>>>
    >>>>>
    >>>>> Mauro, list,
    >>>>>
    >>>>> (please CC me in replies, I am not...
    >>>>>
    >>>>
    >>>> Tried changing these files to exclude CPU0?
    >>>>
    >>>> Have you tried running the irqbalance daemon? That's what you likely
    >>>> want to
    >>>> be doing anyway..
    >>>>
    >>>>
    >>>>>
    >>>>> =====8<=====8<=====8<=====8<=====8<=====8<=====8<=====8<=====8<=====
    >>>>>
    >>>>> =====8<=====8<=====8<==...
    >>>>>
    >>
    >> Please see the two attached examples.
    >>
    >> Notice on the 5410 example how we start with the affinity set to 0xff,
    >> and change it to 0xf0.
    >> This should spread the interrupts over the last 4 cores of this quad
    >> core - dual processor system.
    >>
    >> Also notice on the 5645 example, with the same commands we start with
    >> 0xffffff and change to 0xfff000 to spread the interrupts over the last
    >> 12 cores, but only the first of the last twelve cores receive
    >> interrupts.
    >>
    >> This is the inconsistency I was trying to explain before.
    >>
    >
    > What was the status of irqbalance daemon? Was it turned on? If it is
    > running, there is a chance that the interrupt count is within the threshold
    > limit and interrupts are not being routed to the other core.

    irqbalance daemon was not running on either setup.

    > Could you also try with increasing the interrupt load and see if the
    > distribution is happening among the cores?

    We use spirent testcenter l2/l3 test equipment and pushed 100%
    throughput with the same distribution. Nothing changed.

    This isn't affecting just ethernet drivers. I have also seen the same
    issues with hardware encryption devices and other hardware that gets a
    software interrupt.

    --Bryan

    >
    > -Ciju
    >>
    >> --Bryan
    >>
    >
    >



    --
    Bryan Hundven
    bryanhundven@gmail.com


    \
     
     \ /
      Last update: 2010-07-18 20:55    [W:0.029 / U:1.436 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site