lkml.org 
[lkml]   [2012]   [Jul]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Subject[RFC] net: further seperate dst_entry.__refcnt from cache contention
Date
After some investigation on large machines I found that
dst_entry.__refcnt particpates in false cache sharing issues that show
when scaling past 12 threads who communicate via tcp with loopback addresses.
I adjusted refcnt to be on its own cache line and that helped quite a bit.
But perhaps a bit of a waste of space? Is there some better way?

Here is some preliminary data I had gathered. It shows nicely improved scaling.

Threads baseline afterchange
2 1328.03 1340.67
4 2430.31 2282.09
6 3087.65 3258.12
8 3560.34 4165.88
10 3900.34 4962.28
12 3933.38 5613.76
14 3876.98 6113.85
16 3706.01 6338.00
18 3742.48 6634.77
20 3670.15 6641.25
22 3660.98 6799.55
24 3503.13 6613.45
26 3525.73 6702.67
28 3440.16 6801.27
30 3497.59 6911.52
32 3498.25 6540.06

I should say something about this test. It is a dead simple test in which a
pair of threads simply pass data to each other. They were placed in the same
socket to avoid cross node overhead.

CC: "David S. Miller" <davem@davemloft.net>
Signed-off-by: Nathan Zimmer <nzimmer@sgi.com>

---
include/net/dst.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/include/net/dst.h b/include/net/dst.h
index 8197ead..3898643 100644
--- a/include/net/dst.h
+++ b/include/net/dst.h
@@ -84,7 +84,7 @@ struct dst_entry {
* input/output/ops or performance tanks badly
*/
atomic_t __refcnt; /* client references */
- int __use;
+ int __use ____cacheline_aligned;
unsigned long lastuse;
union {
struct dst_entry *next;
--
1.6.0.2


\
 
 \ /
  Last update: 2012-07-20 22:21    [W:0.031 / U:0.604 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site