[lkml]   [2009]   [Nov]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: newidle balancing in NUMA domain?
> Quite a few being one test case, and on a program with a horrible
> parallelism design (rapid heavy weight forks to distribute small
> units of work).

> If x264 is declared dainbramaged, that's fine with me too.

We did multiple benchmarks using a thread pool and it did not help.
If you want to declare our app "braindamaged", feel free, but pooling
threads to avoid re-creation gave no benefit whatsoever. If you think
the parallelism methodology is wrong as a whole, you're basically
saying that Linux shouldn't be used for video compression, because
this is the exact same threading model used by almost every single
video encoder ever made. There are actually a few that use
slice-based threading, but those are actually even worse from your
perspective, because slice-based threading spawns mulitple threads PER
FRAME instead of one per frame.

Because of the inter-frame dependencies in video coding it is
impossible to efficiently get a granularity of more than one thread
per frame. Pooling threads doesn't change the fact that you are
conceptually creating a thread for each frame--it just eliminates the
pthread_create call. In theory you could do one thread per group of
frames, but that is completely unrealistic for real-time encoding
(e.g. streaming), requires a catastrophically large amount of memory,
makes it impossible to track the bit buffer, and all other sorts of
bad stuff.

Jason Garrett-Glaser

 \ /
  Last update: 2009-11-24 18:27    [W:0.066 / U:6.684 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site