Messages in this thread | | | Subject | per-cpu operation madness vs validation | From | Peter Zijlstra <> | Date | Tue, 26 Jul 2011 23:06:50 +0200 |
| |
Hi all,
so recently our per-cpu ops have exploded.. who can say (without looking) what the difference is between percpu_read() and this_cpu_read() ?
Now Thomas recently did a fresh -rt and ran into the fun problem of trying to reconstruct the requirements on a lot of the per-cpu grub we've grown over the past few releases.
Does it require preempt-disable, bh disable, irq-disable, is it perhaps covered by a lock, what!? I'm sure Thomas can point you to a few gems if you're interested in specifics.
Now the reason is of course that -rt changes some things, even when using migrate_disable() it might be multiple processes might try and access the per-cpu variable, needing serialization.
The idea was to do something similar to rcu-lockdep, where each rcu_dereference() was paired with a conditional that expresses the exact conditions under which that dereference was good -- typically either when holding the rcu_read_lock() or the lock used to serialize the modifying side of things.
Even for mainline such validation is useful, suppose you intended the variable to only be accessed by task context, and thus disabling preemption is plenty to fully serialize things. However unbeknown to the original author someone adds usage from IRQ context, and voila things start breaking.
The point of course is, how are we going to go about doing this, I'm sure adding a proper conditional to each and every per-cpu op is going to be a herculean task, and most of it utterly boring.
One of the thing we could do is create a lock type that doesn't actually generate any lock code on mainline (but does on rt) but is instead used to annotate the serialization requirements of various of these things. If such a lock isn't IRQ safe but used from IRQ context lockdep will complain etc..
| |