[lkml]   [2012]   [Jan]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectWhy the number of /proc/interrupts doesn't change when nic is under heavy workload?
Hi All,

My nic of server is Intel Corporation 80003ES2LAN Gigabit Ethernet
Controller, the driver is e1000e, and my Linux version is 3.1.4. I
have a Memcached server running on this 8 core box, the weird thing is
that when my server is under heavy workload, the number of
/proc/interrupts doesn't change at all. Below are some details:
cat /proc/interrupts | grep eth0
68: 330887 330861 331432 330544 330346 330227
330830 330575 PCI-MSI-edge eth0
cat /proc/irq/68/smp_affinity

I know when network is under heavy load, NAPI will disable nic
interrupt and poll ring buffer in nic. My question is, when is nic
interrupt enabled again? It seems that it will never be enabled if the
heavy workload doesn't stop, simply because the number showed by
/proc/interrupts doesn't change at all. In my case, one of core is
saturated by ksoftirqd, because lots of softirqs are pending to that
core. I just want to distribute these softirqs to other cores. Even
RPS is enabled, that core is still occupied by ksoftirq, nearly 100%.

I dive into the codes and find these statements:
__napi_schedule ==>
____napi_schedule(&__get_cpu_var(softnet_data), n);

here "local_irq_save" actually invokes "cli" which disable interrupt
for the local core, is this the one that used in NAPI to disable nic
interrupt? Personally I don't think it is because it just disables
local cpu.

I also find "enable_irq/disable_irq/e1000_irq_enable/e1000_irq_disable"
under drivers/net/e1000e, are these used in NAPI to disable nic
interrupt, but I fail to get any clue that they are used in the code
path of NAPI?

My current situation is that, almost 60% of time of other 7 cores are
idle, while only one core which is occupied by ksoftirq is 100% busy.

Thanks very much!

 \ /
  Last update: 2012-01-15 21:55    [W:0.045 / U:11.748 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site