lkml.org 
[lkml]   [2007]   [Feb]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: threadlets as 'naive pool of threads', epoll, some measurements
    On Mon, 26 Feb 2007, Ingo Molnar wrote:

    >
    > update:
    >
    > > i have tried the one Evgeniy provided in the URL:
    > >
    > > http://tservice.net.ru/~s0mbre/archive/kevent/evserver_epoll.c
    > >
    > > and 'ab -k -c8000 -n80000' almost always aborts with:
    > >
    > > apr_socket_recv: Connection reset by peer (104)
    > >
    > > in the few cases it finishes, i got the following epoll result, over
    > > gigabit ethernet, on an UP Athlon64 box:
    > >
    > > eserver_epoll: 7800 reqs/sec
    >
    > eserver_epoll.c had a number of bugs. The most serious one was the
    > apparently buggy use of "EPOLLET" (edge-triggered events). Removing that
    > and moving epoll to level-triggered (which is slower but does not result
    > in missed events) gives:
    >
    > eserver_epoll: 9400 reqs/sec
    >
    > > the same with the most naive implementation of the same, using
    > > threadlets:
    > >
    > > eserver_threadlet: 5800 reqs/sec
    >
    > eserver_epoll_threadlet: 9400 reqs/sec
    >
    > as expected, the level of extra blocking triggered by this is low - even
    > if the full request function runs without nonblock assumptions.

    That looks pretty good. I started (spare time allowing) laying down the
    based for a more realistic test as far webserver-like benchmarking.
    I want to compare three solutions, that uses the same internal code (as
    far as request parsing and content delivery).

    1) A fully thread classical web "server". This does the trivial
    accept+pthread_create and in there does open+fstat+sendhdrs+sendfile.
    This is already done:

    http://www.xmailserver.org/thrhttp.c

    Do a `gcc -o thrhttp thrhttp.c -lpthread` and you're done.

    2) A coronet (coroutine+epoll library) handling for network
    events/dispatch, plus GUASI (Generic Userspace Asyncronous Syscall
    Interface) to handle generic IO.

    libpcl: http://www.xmailserver.org/libpcl.html
    coronet: http://www.xmailserver.org/coronet-lib.html
    GUASI: http://www.xmailserver.org/guasi-lib.html

    cghttpd: http://www.xmailserver.org/cghttpd-home.html

    All these are configure+make+make_install installable.
    The cghttpd server has the same parsing/content-delivery features of
    thrhttp, but it uses coroutine+epoll to handle network events, and
    GUASI (an old pthread-based userspace implementation of async execution)
    to give open/fstat/sendfile an async behaviour. It implements the
    epoll_wait() (conet_events_wait() acutally, but that maps directly to
    epoll_wait()) hosted inside an async GUASI request, and the GUASI
    dispatch loop handle the case with:

    if (cookie == conet_events_wait_cookie)
    handle_network_events();

    3) Finally, a very similar to cghttpd implementation, but this time using
    the *syslets* to handle async requests. That should be a prety easy
    change to cghttpd, by the mean of replacing the CGHTTPD_SYSCALL() macro
    with proper syslet code. The big advantage of the syslets here, is that
    they do have the cachehit optimization, while GUASI always have to
    trigger a queueing.
    Of course, since the only machine I have with enough RAM to keep many
    thousands of session active is a dual Opteron, I'd need to have an
    x86-64 version of the patch. That shouldn't be a big problem though.


    This hopefully may prove two points. First, a fully threaded solution does
    not scale well when dealing with thousands and thousands of sessions.
    Second, the cachehit syslet trick is The Man in the syslet code, and kicks
    userspace solution #2 ass.
    In the meantime, I think Jens tests are more meaningfull, as far of usage
    field goes, than any network based test.




    - Davide


    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2007-02-26 22:25    [W:3.211 / U:0.012 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site