[lkml]   [2008]   [Jun]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: SMP Affinity Issue with IRQ#
Khaled Al-Hamwi wrote:
> Hi list,
> I am trying to evaluate the performance of a Linux box when working as
> IP forwarder.
> I have a hardware traffic generator IXIA T400 on one end.
> On the other end, I have a Linux box with kernel 2.6.16 and quad-core xeon CPU.
> These two systems are connected through two crossover Gigabit cables and NICs.
> The traffic is sent from IXIA through one NIC to the forwarding machine
> and then it is forwarded back through the other NIC to IXIA.
> I have two issues related to SMP affinity.
> The first one is that when I set the SMP affinity through
> /proc/irq/<IRQ#>/smp_affinity, it changes dynamically.
> Is there any load balancing in the system that changes the affinity
> after some time or after some packets are received?
> Is there a way to set it permanently?

Likely the irqbalance daemon is doing it. You should be able to stop it.
(Or you have the kernel IRQ balancing config option enabled, which you
likely shouldn't.)

> The second issue is that changing the SMP affinity results in
> different delay and throughput measurements.
> I am assigning each NIC to a different CPU.
> If I used a different assignment but still each NIC is assigned to a
> different CPU, I am getting different performance results.
> I would expect that changing the assignment should yield the same
> performance results.
> The CPUs are identical and should have similar performance.
> Here, I have two examples of two different assignments:
> Example 1:
> /proc/irq/16/smp_affinity (eth0) -> CPU#1
> /proc/irq/20/smp_affinity (eth1) -> CPU#2
> Example 2:
> /proc/irq/16/smp_affinity (eth0) -> CPU#3
> /proc/irq/20/smp_affinity (eth1) -> CPU#2
> Which one of these two configurations can be used as a reference for
> performance evaluation?
> Any ideas??

I believe the current Intel quad core CPUs are really two dual-core CPU
dies in a common package. Splitting the network interrupts across the
two physical CPU chips will likely give you worse performance.

 \ /
  Last update: 2008-06-07 02:25    [W:0.023 / U:17.572 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site