lkml.org 
[lkml]   [1999]   [May]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: Likely cause of EAGAIN in connect() in 2.2.8??
Date
From

Andi Kleen wrote:

> This happens when you run out of local ports. Earlier 2.2 had some leaks
> in this regard, but they should be fixed in 2.2.8. You can try to increase
> the local port range in /proc/sys/net/ipv4/ip_local_port_range.
> Setting SO_REUSEADDR on connected sockets may help too.

Increased range to 16,000 ports. It still dies at around 4,000 simultaneous
connections/addresses, with ENOBUFS rather than EAGAIN. (this is 2.2.8)
Is that a documented return code too? (not in Debian 2.1 man page, but maybe
in some later one it is)

Anyway then I increased /proc/sys/net/ipv4/route/route-max to 16384, and
now it works (up to 7000 simultaneous at least - which is where the CPU
saturates, and falls back to around 4000 simultaneous...2.0.36 on a P120
was doing better than 2.2.8 is on a P233, with half the memory, almost all
other hardware identical). I would have expected this hardware to have done
at least 12,000 simultaneous and sustained if 2.0.36 would run on it, but
unfortunately I can't test since it won't :-(.

Are there any other things I should be tuning to run something which
attempts a lot of connect()s quickly? Or should I be forcing some of
these values down to prevent growth of some internal tables?

So far I've done
inode-max -> 32k
file-max -> 16k
ip_local_port_range -> 1k 17k
route-max -> 16k

It's nowhere near running out of memory or anything like that; CPU is
saturating.

here's vmstat 10 10 as it starts up

libretto:/proc/sys/fs# vmstat 10 10
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
1 0 0 4732 3056 1664 37152 0 0 4 0 114 11 0 1 99
1 0 0 4732 2804 1664 37152 0 0 0 2 885 60 18 36 46
2 0 0 4732 2344 1664 37152 0 0 0 3 1050 77 13 41 47
1 0 0 4732 2384 1664 37152 0 0 0 1 1115 81 2 20 78
0 0 0 4732 2380 1664 37152 0 0 0 1 1118 92 7 28 65
0 0 0 4732 2364 1664 37152 0 0 0 0 1100 91 0 15 85
0 0 0 4732 2364 1664 37152 0 0 0 1 1101 82 2 15 84
0 0 0 4732 2364 1664 37152 0 0 0 0 1109 83 0 14 86
2 0 0 4732 2252 1664 37152 0 0 0 2 1136 95 9 30 61
2 0 0 4732 2252 1664 37152 0 0 0 2 1131 95 13 50 36

and a little later when its running

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
4 0 0 4732 1812 1664 37152 0 0 4 0 115 11 0 1 99
4 0 0 4732 1732 1664 37152 0 0 0 4 1144 118 11 50 39
23 0 0 4732 1764 1664 36768 0 0 0 4 1099 118 34 66 0
16 0 0 4716 2200 1664 36632 0 0 1 3 946 107 32 68 0
16 0 0 4716 2420 1664 36632 0 0 0 2 1003 107 20 80 0
13 0 0 4716 2380 1664 36632 0 0 0 3 1089 110 16 84 0
13 0 0 4716 2036 1664 36632 0 0 0 3 996 108 15 85 0
12 0 0 4716 2000 1668 36632 0 0 0 0 1098 98 28 72 0
11 0 0 4716 1992 1668 36632 0 0 0 1 1048 90 15 85 0
14 0 0 4716 1956 1668 36632 0 0 0 0 1083 99 17 83 0

and so it continues

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
11 0 0 4716 1988 1668 36632 0 0 4 0 117 12 0 1 99
13 0 0 4716 1988 1668 36632 0 0 0 1 1041 99 18 82 0
12 0 0 4716 1896 1668 36632 0 0 0 1 1058 99 16 84 0
11 0 1 4716 1892 1668 36632 0 0 0 0 999 90 27 73 0
14 0 0 4716 1876 1668 36632 0 0 0 1 1084 95 19 81 0
11 0 0 4716 1888 1668 36632 0 0 0 0 1004 90 17 83 0
14 0 0 4716 1900 1668 36632 0 0 0 0 993 84 26 74 0
11 0 0 4716 2076 1668 36632 0 0 0 0 940 88 19 81 0
0 0 0 4716 3132 1668 36632 0 0 0 0 433 45 3 21 75
0 0 0 4716 3324 1668 36632 0 0 0 1 260 31 0 0 100

David.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:51    [W:0.475 / U:2.176 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site