[lkml]   [2007]   [Feb]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3
On Wed, Feb 21, 2007 at 10:13:55PM +0100, Ingo Molnar wrote:
> this is the v3 release of the syslet/threadlet subsystem:
> This release came a few days later than i originally wanted, because
> i've implemented many fundamental changes to the code. The biggest
> highlights of v3 are:
> - "Threadlets": the introduction of the 'threadlet' execution concept.
> - syslets: multiple rings support with no kernel-side footprint, the
> elimination of mlock() pinning, no async_register/unregister() calls
> needed anymore and more.
> "Threadlets" are basically the user-space equivalent of syslets: small
> functions of execution that the kernel attempts to execute without
> scheduling. If the threadlet blocks, the kernel creates a real thread
> from it, and execution continues in that thread. The 'head' context (the
> context that never blocks) returns to the original function that called
> the threadlet. Threadlets are very easy to use:
> long my_threadlet_fn(void *data)
> {
> char *name = data;
> int fd;
> fd = open(name, O_RDONLY);
> if (fd < 0)
> goto out;
> fstat(fd, &stat);
> read(fd, buf, count)
> ...
> out:
> return threadlet_complete();
> }
> main()
> {
> done = threadlet_exec(threadlet_fn, new_stack, &user_head);
> if (!done)
> reqs_queued++;
> }
> There is no limitation whatsoever about how a threadlet function can
> look like: it can use arbitrary system-calls and all execution will be
> procedural. There is no 'registration' needed when running threadlets
> either: the kernel will take care of all the details, user-space just
> runs a threadlet without any preparation and that's it.
> Completion of async threadlets can be done from user-space via any of
> the existing APIs: in threadlet-test.c (see the async-test-v3.tar.gz
> user-space examples at the URL above) i've for example used a futex
> between the head and the async threads to do threadlet notification. But
> select(), poll() or signals can be used too - whichever is most
> convenient to the application writer.
> Threadlets can also be thought of as 'optional threads': they execute in
> the original context as long as they do not block, but once they block,
> they are moved off into their separate thread context - and the original
> context can continue execution.
> Threadlets can also be thought of as 'on-demand parallelism': user-space
> does not have to worry about setting up, sizing and feeding a thread
> pool - the kernel will execute the workload in a single-threaded manner
> as long as it makes sense, but once the context blocks, a parallel
> context is created. So parallelism inside applications is utilized in a
> natural way. (The best place to do this is in the kernel - user-space
> has no idea about what level of parallelism is best for any given
> moment.)
> I believe this threadlet concept is what user-space will want to use for
> programmable parallelism.
> [ Note that right now there's a pair of system-calls: sys_threadlet_on()
> and sys_threadlet_off() that demarks the beginning and the end of a
> syslet function, which enter the kernel even in the 'cached' case -
> but my plan is to do these two system calls via a vsyscall, without
> having to enter the kernel at all. That will reduce cached threadlet
> execution NULL-overhead to around 10 nsecs - making it essentially
> zero. ]
> Threadlets share much of the scheduling infrastructure with syslets.
> Syslets (small, kernel-side, scripted "syscall plugins") are still
> supported - they are (much...) harder to program than threadlets but
> they allow the highest performance. Core infrastructure libraries like
> glibc/libaio are expected to use syslets. Jens Axboe's FIO tool already
> includes support for v2 syslets, and the following patch updates FIO to

Ah, glad to see that - I was wondering if it was worthwhile to try adding
syslet support to aio-stress to be able to perform some comparisons.
Hopefully FIO should be able to generate a similar workload, but I haven't
tried it yet so am not sure. Are you planning to upload some results
(so I can compare it with patterns I am familiar with) ?


> the v3 API:
> Furthermore, the syslet code and API has been significantly enhanced as
> well:
> - support for multiple completion rings has been added
> - there is no more mlock()ing of the completion ring(s)
> - sys_async_register()/unregister() has been removed as it is not
> needed anymore. sys_async_exec() can be called straight away.
> - there is no kernel-side resource used up by async completion rings at
> all (all the state is in user-space), so an arbitrary number of
> completion rings are supported.
> plus lots of bugs were fixed and a good number of cleanups were done as
> well. The v3 code is ABI-incompatible with v2, due to these fundamental
> changes.
> As always, comments, suggestions, reports are welcome.
> Ingo

Suparna Bhattacharya (
Linux Technology Center
IBM Software Lab, India

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2007-02-22 10:59    [W:0.289 / U:3.100 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site