Messages in this thread | | | From | Orin Eman <> | Subject | Re: High Performance I/O stuff (more) | Date | Wed, 15 Sep 1999 01:17:19 -0700 (PDT) |
| |
> I think that both of these models are basically the same. They both have > an event queue that you pick up events from. The only way that they differ > is in what they call an event. Completion ports take asynchronous opperations > and queue an event when the opperation completes (hence the name). Synchronous > events do the opposite: they queue an event when an opperation is possible > and then the synchronous (usually, non-blocking) opperation is performed. > >From this, you can decouple and event queue from what you call an event. > > >From what I can see either model will give roughly the same performance, > as they both do roughly the same amount of work. The one benefit that seems > to exist for the Completion Ports model is that there are fewer contex > switches.
Nice summary. I've been watching this thread and do have some experience with NT's completion ports, along with some event based (WSAEventSelect) code. The following is meant to be food for thought. It seems to have come out as thoughts of how I would use the two schemes.
One useful feature of the Win32 completion ports is the instance data you get back. There is no need to map a file descriptor to your instance data for the connection. If what you get back from the API is the fd, then you must either keep an array with one entry per fd, or search thru your active connections for the fd.
There was some comment about what happens if you can't get buffers or other kernel resources. From what I can tell, when you do overlapped IO on win32, there isn't a problem with being unable to allocate memory to queue the completion - the overlapped structure given with the original async call is used. Ie. if the async call doesn't fail immediately, the kernel should have enough buffers etc to complete it. Unfortunately, this pretty much means that you will be allocating a buffer per connection and posting a read per connection. This doesn't seem to be a good solution of a lot of your connections are idle! I believe that this can be somewhat mitigated by setting socket options such that the stack uses the buffers posted in the async call... it gets messy - the Winsock2 mailing list occasionally has such tidbits of information.
With the event model proposed, I would would keep a pool of buffers, such that when an event comes in indicating there is data to be read, I would grab a buffer and issue the read (recv). I would then hold onto this buffer until I got enough data for further processing. You need one buffer per active connection rather than the buffer per connection for the completion port solution. (You may run around thru several events keeping the same buffer until you got a complete packet/request/whatever you are interested in.) What happens if you can't get a buffer? Then you wait until you can. In the case of TCP, the TCP window drops to zero and the other end gets throttled back...
This brings up the question of disabling events once they are delivered. Say you deliver an event indicating data is available. It isn't read due to lack of buffers. Now you get some more incoming data. A new event probably shouldn't be delivered in this case - I would delay any new event generation at least until after some data has been read. (Winsock does this, though it could smarter - it will generate a new event if there is data to be read at the end of a recv. I would wait until all available data was read before allowing an event to be generated on the arrival of data.)
One final point. The completion port model potentially has lower latency - once you get the completion, the data is in your buffer. With the event based scheme, you have to issue the recv and wait while it is copied from the system buffer. Probably not significant unless a completion port scheme is using the user supplied buffers, in which case, a copy might be avoided and only then for a very large number of connections.
BTW, I saw Gideon Glass's proposed API. I'd change the name - I see it as an event queue, not a completion port.
Orin.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |