lkml.org 
[lkml]   [2007]   [Jul]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[patch] Avoid too many remote cpu references due to /proc/stat
    On Thu, Jul 12, 2007 at 05:06:15PM -0700, Ravikiran G Thirumalai wrote:
    > Too many remote cpu references due to /proc/stat.
    >
    > On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem.
    > On every call to kstat_irqs, the process brings in per-cpu data from all
    > online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS
    > results in (256+32*63) * 63 remote cpu references on a 64 cpu config.
    > /proc/stat is parsed by common commands like top, who etc, causing
    > lots of cacheline transfers
    >
    > This statistic seems useless. Other 'big iron' arches disable this.
    > Can we disable computing/reporting this statistic? This piece of
    > statistic is not human readable on x86_64 anymore,
    >
    > If not, can we optimize computing this statistic so as to avoid
    > too many remote references (patch to follow)

    Optimize show_stat to collect per-irq information just once.

    On x86_64, with newer kernel versions, kstat_irqs is a bit of a problem.
    On every call to kstat_irqs, the process brings in per-cpu data from all
    online cpus. Doing this for NR_IRQS, which is now 256 + 32 * NR_CPUS
    results in (256+32*63) * 63 remote cpu references on a 64 cpu config.
    Considering the fact that we already compute this value per-cpu, we can
    save on the remote references as below.

    Signed-off-by: Alok N Kataria <alok.kataria@calsoftinc.com>
    Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>

    Index: linux-2.6.22/fs/proc/proc_misc.c
    ===================================================================
    --- linux-2.6.22.orig/fs/proc/proc_misc.c 2007-07-11 14:32:33.013197741 -0700
    +++ linux-2.6.22/fs/proc/proc_misc.c 2007-07-12 16:28:24.871389279 -0700
    @@ -443,6 +443,11 @@ static int show_stat(struct seq_file *p,
    unsigned long jif;
    cputime64_t user, nice, system, idle, iowait, irq, softirq, steal;
    u64 sum = 0;
    + unsigned int *per_irq_sum;
    +
    + per_irq_sum = kzalloc(sizeof(unsigned int)*NR_IRQS, GFP_KERNEL);
    + if (!per_irq_sum)
    + return -ENOMEM;

    user = nice = system = idle = iowait =
    irq = softirq = steal = cputime64_zero;
    @@ -461,8 +466,11 @@ static int show_stat(struct seq_file *p,
    irq = cputime64_add(irq, kstat_cpu(i).cpustat.irq);
    softirq = cputime64_add(softirq, kstat_cpu(i).cpustat.softirq);
    steal = cputime64_add(steal, kstat_cpu(i).cpustat.steal);
    - for (j = 0 ; j < NR_IRQS ; j++)
    - sum += kstat_cpu(i).irqs[j];
    + for (j = 0 ; j < NR_IRQS ; j++) {
    + unsigned int temp = kstat_cpu(i).irqs[j];
    + sum += temp;
    + per_irq_sum[j] += temp;
    + }
    }

    seq_printf(p, "cpu %llu %llu %llu %llu %llu %llu %llu %llu\n",
    @@ -500,7 +508,7 @@ static int show_stat(struct seq_file *p,

    #if !defined(CONFIG_PPC64) && !defined(CONFIG_ALPHA) && !defined(CONFIG_IA64)
    for (i = 0; i < NR_IRQS; i++)
    - seq_printf(p, " %u", kstat_irqs(i));
    + seq_printf(p, " %u", per_irq_sum[i]);
    #endif

    seq_printf(p,
    @@ -515,6 +523,7 @@ static int show_stat(struct seq_file *p,
    nr_running(),
    nr_iowait());

    + kfree(per_irq_sum);
    return 0;
    }

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/
    \
     
     \ /
      Last update: 2007-07-13 02:11    [W:0.030 / U:0.828 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site