Messages in this thread | | | Subject | Re: Likely cause of EAGAIN in connect() in 2.2.8?? | Date | Thu, 13 May 1999 17:09:14 +0800 | From | David Luyer <> |
| |
Andi Kleen wrote:
> This happens when you run out of local ports. Earlier 2.2 had some leaks > in this regard, but they should be fixed in 2.2.8. You can try to increase > the local port range in /proc/sys/net/ipv4/ip_local_port_range. > Setting SO_REUSEADDR on connected sockets may help too.
Increased range to 16,000 ports. It still dies at around 4,000 simultaneous connections/addresses, with ENOBUFS rather than EAGAIN. (this is 2.2.8) Is that a documented return code too? (not in Debian 2.1 man page, but maybe in some later one it is)
Anyway then I increased /proc/sys/net/ipv4/route/route-max to 16384, and now it works (up to 7000 simultaneous at least - which is where the CPU saturates, and falls back to around 4000 simultaneous...2.0.36 on a P120 was doing better than 2.2.8 is on a P233, with half the memory, almost all other hardware identical). I would have expected this hardware to have done at least 12,000 simultaneous and sustained if 2.0.36 would run on it, but unfortunately I can't test since it won't :-(.
Are there any other things I should be tuning to run something which attempts a lot of connect()s quickly? Or should I be forcing some of these values down to prevent growth of some internal tables?
So far I've done inode-max -> 32k file-max -> 16k ip_local_port_range -> 1k 17k route-max -> 16k
It's nowhere near running out of memory or anything like that; CPU is saturating.
here's vmstat 10 10 as it starts up
libretto:/proc/sys/fs# vmstat 10 10 procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 0 0 4732 3056 1664 37152 0 0 4 0 114 11 0 1 99 1 0 0 4732 2804 1664 37152 0 0 0 2 885 60 18 36 46 2 0 0 4732 2344 1664 37152 0 0 0 3 1050 77 13 41 47 1 0 0 4732 2384 1664 37152 0 0 0 1 1115 81 2 20 78 0 0 0 4732 2380 1664 37152 0 0 0 1 1118 92 7 28 65 0 0 0 4732 2364 1664 37152 0 0 0 0 1100 91 0 15 85 0 0 0 4732 2364 1664 37152 0 0 0 1 1101 82 2 15 84 0 0 0 4732 2364 1664 37152 0 0 0 0 1109 83 0 14 86 2 0 0 4732 2252 1664 37152 0 0 0 2 1136 95 9 30 61 2 0 0 4732 2252 1664 37152 0 0 0 2 1131 95 13 50 36
and a little later when its running
procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 4 0 0 4732 1812 1664 37152 0 0 4 0 115 11 0 1 99 4 0 0 4732 1732 1664 37152 0 0 0 4 1144 118 11 50 39 23 0 0 4732 1764 1664 36768 0 0 0 4 1099 118 34 66 0 16 0 0 4716 2200 1664 36632 0 0 1 3 946 107 32 68 0 16 0 0 4716 2420 1664 36632 0 0 0 2 1003 107 20 80 0 13 0 0 4716 2380 1664 36632 0 0 0 3 1089 110 16 84 0 13 0 0 4716 2036 1664 36632 0 0 0 3 996 108 15 85 0 12 0 0 4716 2000 1668 36632 0 0 0 0 1098 98 28 72 0 11 0 0 4716 1992 1668 36632 0 0 0 1 1048 90 15 85 0 14 0 0 4716 1956 1668 36632 0 0 0 0 1083 99 17 83 0
and so it continues
procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 11 0 0 4716 1988 1668 36632 0 0 4 0 117 12 0 1 99 13 0 0 4716 1988 1668 36632 0 0 0 1 1041 99 18 82 0 12 0 0 4716 1896 1668 36632 0 0 0 1 1058 99 16 84 0 11 0 1 4716 1892 1668 36632 0 0 0 0 999 90 27 73 0 14 0 0 4716 1876 1668 36632 0 0 0 1 1084 95 19 81 0 11 0 0 4716 1888 1668 36632 0 0 0 0 1004 90 17 83 0 14 0 0 4716 1900 1668 36632 0 0 0 0 993 84 26 74 0 11 0 0 4716 2076 1668 36632 0 0 0 0 940 88 19 81 0 0 0 0 4716 3132 1668 36632 0 0 0 0 433 45 3 21 75 0 0 0 4716 3324 1668 36632 0 0 0 1 260 31 0 0 100
David.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |