Messages in this thread | | | Date | Mon, 30 Nov 2009 09:19:18 +0100 | From | Nick Piggin <> | Subject | Re: newidle balancing in NUMA domain? |
| |
On Tue, Nov 24, 2009 at 09:24:26AM -0800, Jason Garrett-Glaser wrote: > > Quite a few being one test case, and on a program with a horrible > > parallelism design (rapid heavy weight forks to distribute small > > units of work). > > > If x264 is declared dainbramaged, that's fine with me too. > > We did multiple benchmarks using a thread pool and it did not help. > If you want to declare our app "braindamaged", feel free, but pooling > threads to avoid re-creation gave no benefit whatsoever. If you think > the parallelism methodology is wrong as a whole, you're basically > saying that Linux shouldn't be used for video compression, because > this is the exact same threading model used by almost every single > video encoder ever made. There are actually a few that use > slice-based threading, but those are actually even worse from your > perspective, because slice-based threading spawns mulitple threads PER > FRAME instead of one per frame. > > Because of the inter-frame dependencies in video coding it is > impossible to efficiently get a granularity of more than one thread > per frame. Pooling threads doesn't change the fact that you are > conceptually creating a thread for each frame--it just eliminates the > pthread_create call. In theory you could do one thread per group of > frames, but that is completely unrealistic for real-time encoding > (e.g. streaming), requires a catastrophically large amount of memory, > makes it impossible to track the bit buffer, and all other sorts of > bad stuff.
If you can scale to N threads by having 1 frame per thread, then you can scale to N/2 threads and have 2 frames per thread. Can't you?
Is your problem in scaling to a large N?
| |