lkml.org 
[lkml]   [1999]   [Sep]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: High Performance I/O stuff (more)
Date
> I think that both of these models are basically the same.  They both have
> an event queue that you pick up events from. The only way that they differ
> is in what they call an event. Completion ports take asynchronous opperations
> and queue an event when the opperation completes (hence the name). Synchronous
> events do the opposite: they queue an event when an opperation is possible
> and then the synchronous (usually, non-blocking) opperation is performed.
> >From this, you can decouple and event queue from what you call an event.
>
> >From what I can see either model will give roughly the same performance,
> as they both do roughly the same amount of work. The one benefit that seems
> to exist for the Completion Ports model is that there are fewer contex
> switches.

Nice summary. I've been watching this thread and do have some experience
with NT's completion ports, along with some event based (WSAEventSelect) code.
The following is meant to be food for thought. It seems to have
come out as thoughts of how I would use the two schemes.

One useful feature of the Win32 completion ports is the instance data
you get back. There is no need to map a file descriptor to your
instance data for the connection. If what you get back from the API
is the fd, then you must either keep an array with one entry per fd,
or search thru your active connections for the fd.

There was some comment about what happens if you can't get buffers
or other kernel resources. From what I can tell, when you do
overlapped IO on win32, there isn't a problem with being
unable to allocate memory to queue the completion - the overlapped
structure given with the original async call is used. Ie. if the
async call doesn't fail immediately, the kernel should have enough
buffers etc to complete it. Unfortunately, this pretty much means
that you will be allocating a buffer per connection and posting
a read per connection. This doesn't seem to be a good solution
of a lot of your connections are idle! I believe that this can
be somewhat mitigated by setting socket options such that
the stack uses the buffers posted in the async call... it gets messy -
the Winsock2 mailing list occasionally has such tidbits of information.

With the event model proposed, I would would keep a pool
of buffers, such that when an event comes in indicating there is
data to be read, I would grab a buffer and issue the read (recv).
I would then hold onto this buffer until I got enough data
for further processing. You need one buffer per active connection
rather than the buffer per connection for the completion port solution.
(You may run around thru several events keeping the same buffer
until you got a complete packet/request/whatever you are interested in.)
What happens if you can't get a buffer? Then you wait until you can.
In the case of TCP, the TCP window drops to zero and the other
end gets throttled back...

This brings up the question of disabling events once they are
delivered. Say you deliver an event indicating data is available.
It isn't read due to lack of buffers. Now you get some more
incoming data. A new event probably shouldn't be delivered in
this case - I would delay any new event generation at least until after
some data has been read. (Winsock does this, though it could
smarter - it will generate a new event if there is data to be
read at the end of a recv. I would wait until all available
data was read before allowing an event to be generated on
the arrival of data.)

One final point. The completion port model potentially has lower
latency - once you get the completion, the data is in your buffer.
With the event based scheme, you have to issue the recv and wait
while it is copied from the system buffer. Probably not significant
unless a completion port scheme is using the user supplied buffers,
in which case, a copy might be avoided and only then for a very large
number of connections.

BTW, I saw Gideon Glass's proposed API. I'd change the name - I see it as
an event queue, not a completion port.

Orin.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:54    [W:0.310 / U:0.144 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site