[lkml]   [2003]   [Jan]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRE: [2.5] IRQ distribution in the 2.5.52 kernel
    Hi Andrew,
    Your benchmark results are very impressive. Thanks for trying it out.
    I have some thoughts after seeing the results.

    > Nitin,
    > I got a chance to run the NetBench benchmark with your patch on
    > mjb2
    > kernel. NetBench measures SMB/CIFS performance by using several SMB
    > clients
    > (in this case 44 Windows 2000 systems), sending SMB requests to a
    > server running Samba 2.2.3a+sendfile. Result is in throughput, Mbps.
    > Generally the network traffic on the server is 60% recv, 40% tx.
    > I believe we have very similar systems. Mine is a 4 x 1.6 GHz, 1 MB
    L3 P4
    > Xeon with 4 GB DDR memory (3.2 GB/sec I believe). The chipset is
    > "Summit".
    > I also have more than one Intel e1000 adapters.
    > I decided to run a few configurations, first with just one adapter,
    > and
    > without HT support in the kernel (acpi=off), then add another adapter
    > test again with/without HT.
    > Here are the results:
    > 4P, no HT, 1 x e1000, no kirq: 1214 Mbps, 4% idle
    > 4P, no HT, 1 x e1000, kirq: 1223 Mbps, 4% idle,
    [NK] It is surprising to see single e1000 is giving bandwidth more than
    1Gbps. What can be the reason for this extra bandwidth? ... Maybe
    compression is happening somewhere.

    > I suppose we didn't see much of an improvement here because we never
    > into
    > the situation where more than one interrupt with a high rate is routed
    > a
    > single CPU on irq_balance.
    > 4P, HT, 1 x e1000, no kirq: 1214 Mbps, 25% idle
    > 4P, HT, 1 x e1000, kirq: 1220 Mbps, 30% idle,
    > Again, not much of a difference just yet, but lots of idle time. We
    > have
    > reached the limit at which one logical CPU can process interrupts for
    > e1000 adapter. There are other things I can probably do to help this,
    > like
    > int delay, and NAPI, which I will get to eventually.
    > 4P, HT, 2 x e1000, no kirq: 1269 Mbps, 23% idle
    > 4P, HT, 2 x e1000, kirq: 1329 Mbps, 18% idle
    [NK] It can be a case that throughput is getting limited by the network
    infrastructure or total load of clients. If we know the theoretical
    desired maximum throughput then we will get a better idea about the
    bottleneck. It would be interesting to see the results, after adding one
    more e1000 card to the server.

    > OK, almost 5% better!
    [NK] It's a pretty good number!

    Probably has to do with a couple of things; the
    > fact
    > that your code does not route two different interrupts to the same
    > core/different logical cpus (quite obvious by looking at
    > /proc/interrupts),
    > and that more than one interrupt does not go to the same cpu if
    > I
    > suspect irq_balance did some of those [bad] things some of the time,
    > we
    > observed a bottleneck in int processing that was lower than with kirq.
    > I don't think all of the idle time is because of a int processing
    > bottleneck.
    > I'm just not sure what it is yet :) Hopefully something will become
    > obvious
    > to me...
    > Overall I like the way it works, and I believe it can be tweaked to
    > with
    > NUMA when necessary.
    [NK] I also believe so.

    I hope to have access to a specweb system on a NUMA
    > box
    > soon, so we can verify that.
    > -Andrew Theurer
    Thanks & regards,
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:32    [W:0.025 / U:38.876 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site