lkml.org 
[lkml]   [2009]   [Mar]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Question about x86/mm/gup.c's use of disabled interrupts
Jeremy Fitzhardinge wrote:
>>>
>>> Well, no, not deferring. Making xen_flush_tlb_others() spin waiting
>>> for "doing_gup" to clear on the target cpu. Or add an explicit
>>> notion of a "pte update barrier" rather than implicitly relying on
>>> the tlb IPI (which is extremely convenient when available...).
>>
>> Pick up a percpu flag from all cpus and spin on each? Nasty.
>
> Yeah, not great. Each of those flag fetches is likely to be cold, so
> a bunch of cache misses. The only mitigating factor is that cross-cpu
> tlb flushes are expected to be expensive, but some workloads are
> apparently very sensitive to extra latency in that path.

Right, and they'll do a bunch more cache misses, so in comparison it
isn't too bad.

> And the hypercall could result in no Xen-level IPIs at all, so it
> could be very quick by comparison to an IPI-based Linux
> implementation, in which case the flag polling would be particularly
> harsh.

Maybe we could bring these optimizations into Linux as well. The only
thing Xen knows that Linux doesn't is if a vcpu is not scheduled; all
other information is shared.

>
> Also, the straightforward implementation of "poll until all target
> cpu's flags are clear" may never make progress, so you'd have to "scan
> flags, remove busy cpus from set, repeat until all cpus done".
>
> All annoying because this race is pretty unlikely, and it seems a
> shame to slow down all tlb flushes to deal with it. Some kind of
> global "doing gup_fast" counter would get flush_tlb_others bypass the
> check, at the cost of putting a couple of atomic ops around the
> outside of gup_fast.

The nice thing about local_irq_disable() is that it scales so well.

>
>> You could use the irq enabled flag; it's available and what native
>> spins on (but also means I'll need to add one if I implement this).
>
> Yes, but then we'd end up spuriously polling on cpus which happened to
> disable interrupts for any reason. And if the vcpu is not running
> then we could end up polling for a long time. (Same applies for
> things in gup_fast, but I'm assuming that's a lot less common than
> disabling interrupts in general).

Right.

--
error compiling committee.c: too many arguments to function



\
 
 \ /
  Last update: 2009-03-19 10:49    [from the cache]
©2003-2011 Jasper Spaans