lkml.org 
[lkml]   [2011]   [Oct]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Regression in reading /proc/stat in the newer kernels with large SMP and NUMA configurations
On Fri, 14 Oct 2011, Oberman, Laurence (HAS GSE) wrote:

> On the 2.6.16.x series sles or 2.6.18.x RHEL kernels each read takes 84
> us.
>
> dl785sles:~/wrk # cat 2.6.16.60-0.87.1-smp.log Opened, read and closed 8640 times and read total of 13996800 bytes
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 99.41 0.725717 84 8641 read
> 0.38 0.002768 0 8642 open
> 0.21 0.001539 0 8642 close
> 0.00 0.000000 0 1 write
> 0.00 0.000000 0 3 fstat
> 0.00 0.000000 0 8 mmap
> 0.00 0.000000 0 2 mprotect
> 0.00 0.000000 0 1 munmap
> 0.00 0.000000 0 1 brk
> 0.00 0.000000 0 1 1 access
> 0.00 0.000000 0 1 madvise
> 0.00 0.000000 0 1 execve
> 0.00 0.000000 0 1 arch_prctl
> ------ ----------- ----------- --------- --------- ----------------
> 100.00 0.730024 25945 1 total
>
> On the 2.6.38.4 kernel each read takes > 6ms
>
> dl785sles:~/wrk # cat 2.6.38.4-smp.log
> Opened, read and closed 8640 times and read total of 86235840 bytes
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 100.00 59.021650 6830 8641 read
> 0.00 0.000520 0 8642 open
> 0.00 0.000425 0 8642 close
> 0.00 0.000000 0 1 write
> 0.00 0.000000 0 3 fstat
> 0.00 0.000000 0 9 mmap
> 0.00 0.000000 0 2 mprotect
> 0.00 0.000000 0 1 munmap
> 0.00 0.000000 0 1 brk
> 0.00 0.000000 0 1 1 access
> 0.00 0.000000 0 1 madvise
> 0.00 0.000000 0 1 execve
> 0.00 0.000000 0 1 arch_prctl
> ------ ----------- ----------- --------- --------- ----------------
> 100.00 59.022595 25946 1 total
>

The overhead is probably in kstat_irqs_cpu() which is called for each
possible irq for each of the 32 possible cpus, and /proc/stat actually
does the sum twice. You would see the same type of overhead with
/proc/interrupts if it wasn't masked by the locking that it requires to
safely read irq_desc. "dmesg | grep nr_irqs" will show how many percpu
variables are being read for every cpu twice.


\
 
 \ /
  Last update: 2011-10-14 12:11    [W:1.960 / U:0.024 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site