Messages in this thread | | | Date | Thu, 6 May 1999 18:55:58 +1000 | From | Richard Gooch <> | Subject | Re: Overscheduling DOES happen with high web server load. |
| |
davem@redhat.com writes: > Date: Thu, 6 May 1999 10:42:18 +1000 > From: Richard Gooch <rgooch@atnf.csiro.au> > > Indeed. As a separate question, we may wonder why so many > processes/threads are being used, and whether that number > could/should be reduced. Perhaps the server is doing something > silly. But that's an aside. Instead, I'd like to explore ways of > reducing the (already low) scheduler overhead. > > No, it is more beneficial to find out why so many tasks are waking up > so much in the first place, _THIS_ is the bug. > > This smells of galloping herd phenomenon to me. And my nose says it's > accept() in this case, and my nose further states that implementing > wake-one semantics in accept() might even make this part of the > profiling go away and increase our TCP server performance > dramatically. Several people have suggested this to me in private > discussions, and I was skeptical at first, but now I'm starting to > believe it. > > This has nothing to do with how many threads are being used, it has to > do with threads getting woken up only to find they have no work to do, > which is the only way the run queues can grow large enough for these > sorts of user processes.
I agree that the thundering herd problem needs to be resolved. Implementing wake-one for select() might be a good idea, although I prefer the approach of using a POSIX.4 signal which gives the fd to the signal handler/waiter. It's more scalable, so it's what should be used for best performance. Still, that's not to say it isn't worth optimising select() and poll() for "legacy" code. When 2.3 comes out, I'll redo my select()/poll() optimisation patch (factor 3x speedup), which will also help.
However, just now I'm interested in seeing what effect my scheduler optimisation patch has. This seems like a good test environment, so I'm curious to see the result.
But you're welcome to care more about improving select() :-)
Regards,
Richard....
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |