lkml.org 
[lkml]   [2010]   [Mar]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: Network performance - iperf
From
Date
Le lundi 29 mars 2010 à 13:33 +0200, Michal Simek a écrit :
> Hi All,
>
> I am doing several network benchmarks on Microblaze cpu with MMU.
> I am seeing one issue which is weird and I would like know where the
> problem is.
> I am using the same hw design and the same Linux kernel. I have done
> only change in memory size (in DTS).
>
> 32MB: 18.3Mb/s
> 64MB: 15.2Mb/s
> 128MB: 10.6Mb/s
> 256MB: 3.8Mb/s
>
> There is huge difference between systems with 32MB and 256MB ram.
>
> I am running iperf TCP tests with these commands.
> On x86: iperf -c 192.168.0.105 -i 5 -t 50
> On microblaze: iperf -s
>
> I look at pte misses which are the same on all configurations which
> means that the number of do_page_fault exceptions is the same on all
> configurations.
> I added some hooks to low level kernel code to be able to see number of
> tlb misses. There is big differences between number of misses on system
> with 256MB and 32MB. I measured two kernel settings. First column is
> kernel with asm optimized memcpy/memmove function and the second is
> without optimization. (Kernel with asm optimized lib functions is 30%
> faster than system without optimization)
>
> 32MB: 12703 13641
> 64MB: 1021750 655644
> 128MB: 1031644 531879
> 256MB: 1011322 430027
>
> Most of them are data tlb misses. Microblaze MMU doesn't use any LRU
> mechanism to find TLB victim that's why we there is naive TLB
> replacement strategy based on incrementing counter. We using 2 tlbs for
> kernel itself which are not updated that's why we can use "only" 62 TLBs
> from 64.
>

This probably has nothing to do with tcp stack, but trashing tlb on some
pathological cases (you have 62 entries, thats good for working size up
to 248 Kbytes, all included (program stack, program static & dynamic
data), given microblaze 4Kbytes page size.

You could try :

echo "4096 8192 32768" >/proc/sys/net/ipv4/tcp_rmem

to reduce memory footprint of iperf (or use iperf parameters)

Of course, I suppose kernel memory is 32 MB max, if you use only two
tlbs (16 Mbytes each) for kernel...



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2010-03-29 22:09    [from the cache]
©2003-2011 Jasper Spaans