[lkml]   [2009]   [Oct]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [RFC PATCHSET] workqueue: implement concurrency managed workqueue
On Thu, Oct 01 2009, Tejun Heo wrote:
> Hello, all.
> This rather large patchset implements concurrency managed workqueue.
> It's not complete yet. Singlethread workqueue handling needs more
> work and workqueue users need to be audited and simplified and async
> and slow-work should be reimplemented in terms of workqueue. Although
> this patchset currently adds ~2000 lines of code, I'm fairly
> optimistic that after the whole conversion is done, it would be a net
> decrease in lines of code.
> This patchset reimplements workqueue such that it auto-regulates
> concurrency and thus relieves its users from the managing duty. It
> works by managing single shared pool of per-cpu workers and hooking
> into the scheduler to get notifications about workers going to sleep
> and waking up. Using the mechanism, workqueue implementation keeps
> track of the current level of concurrency and schedules only the
> necessary number of workers to keep the cpu occupied.
> Concurrency managed workqueue has the following benefits.
> * Workqueue users no longer have to worry about managing concurrency
> and, in most cases, deadlocks. The workqueue will manage it
> automatically and unless the deadlock chain involves many (currently
> 127) works, it won't happen.
> * There's one single shared pool of workers per cpu and one rescuer
> for each workqueue which requires it, so there are far fewer number
> of kthreads.
> * More efficient. Although it adds considerable amount of code, the
> code added to hot path isn't big and works will be executed on the
> local cpu and in batch as much as possible using minimal number of
> kthreads leading to fewer task switches and lower cache
> * As concurrency is no longer a problem, most types of asynchronous
> jobs can be done using generic workqueue and other async mechanisms,
> including slow-work, async and adhoc subsystem custom ones, can be
> removed. ie. It can serve as the unified async thread pool
> mechanism.

Awesome work so far Tejun, I have high hopes for this patchset! I'll
take some time to review this when I have it, just consider this so far
a big encouragement to crank away on this. It's always annoyed me that
we have various methods for doing async work, this promises to unify
that very nicely.

Jens Axboe

 \ /
  Last update: 2009-10-01 10:27    [W:0.338 / U:2.944 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site