Messages in this thread | | | Date | Tue, 21 May 2013 09:54:59 -0700 | From | "Paul E. McKenney" <> | Subject | Re: [PATCH tip/core/rcu 6/7] rcu: Drive quiescent-state-forcing delay from HZ |
| |
On Tue, May 21, 2013 at 11:45:31AM +0200, Peter Zijlstra wrote: > On Thu, May 16, 2013 at 06:22:10AM -0700, Paul E. McKenney wrote: > > > But somehow I imagined making a CPU part of the GP would be easier than taking > > > it out. After all, taking it out is dangerous and careful work, one is not to > > > accidentally execute a callback or otherwise end a GP before time. > > > > > > When entering the GP cycle there is no such concern, the CPU state is clean > > > after all. > > > > But that would increase the overhead of GP initialization. Right now, > > GP initialization touches only the leaf rcu_node structures, of which > > there are by default one per 16 CPUs (and can be configured up to one per > > 64 CPUs, which it is on really big systems). So on busy mixed-workload > > systems, this approach increases GP initialization overhead for no > > good reason -- and on systems running these sorts of workloads, there > > usually aren't "sacrificial lamb" timekeeping CPUs whose utilization > > doesn't matter. > > Right, so I read through some of the fqs code to get a better feel for > things and I suppose I see what you're talking about :-) > > The only thing I could come up with is making fqslock a global/local > style lock, so that individual CPUs can adjust their own state without > bouncing the lock around.
Maybe... The current design uses bitmasks at each level, and avoiding the upper-level locks would mean making RCU work with out-of-date bitmasks at the upper levels. Might be possible, but it is not clear to me that this would be a win.
I could also maintain yet another bitmask at the bottom level to record the idle CPUs, but it is not clear that this is a win, either, especially on systems with frequent idle/busy transitions.
> It would make the fqs itself a 'bit' more expensive but ideally those > don't happen that often, ha!. > > But yeah, every time you let the fqs propagate 'idle' state up the tree > your join becomes more expensive too.
Yep! :-/
Thanx, Paul
| |