Messages in this thread | | | Date | Tue, 19 Jan 2010 12:11:29 -0500 | From | Mathieu Desnoyers <> | Subject | Re: [RFC PATCH] introduce sys_membarrier(): process-wide memory barrier (v5) |
| |
* Peter Zijlstra (peterz@infradead.org) wrote: [...] > > It's this scenario that is causing problem. Let's consider this > > execution: > > > > CPU 0 (membarrier) CPU 1 (another mm -> our mm) > > <kernel-space> <kernel-space> > > switch_mm() > > smp_mb() > > clear_mm_cpumask() > > set_mm_cpumask() > > smp_mb() (by load_cr3() on x86) > > switch_to() > > mm_cpumask includes CPU 1 > > rcu_read_lock() > > if (CPU 1 mm != our mm) > > skip CPU 1. > > rcu_read_unlock() > > current = next (1) > > OK, so on x86 current uses esp and will be flipped somewhere in the > switch_to() magic, cpu_curr(cpu) as used by CPU 0 uses rq->curr, which > will be set before context_switch() and that always implies a mb() for > non matching ->mm's [*]
Hi Peter,
Please refer to the discussion with Steven further down this thread (http://lkml.org/lkml/2010/1/14/319), which I update the scenario when I figured out that "current" and rq->curr are indeed two different things. It's rq->curr we are interested into here, not "current" as I previously thought. (sorry about the mixup)
> > > <switch back to user-space> > > read-lock() > > read gp, store local gp > > barrier() > > access critical section (2) > > > > So if we don't have any memory barrier between (1) and (2), the memory > > operations can be reordered in such a way that CPU 0 will not send IPI > > to a CPU that would need to have it's barrier() promoted into a > > smp_mb(). > > OK, so I'm utterly failing to make sense of the above, do you need more > than the 2 cpus discussed to make it go boom? > > > Replacing these kernel rcu_read_lock/unlock() by rq locks ensures that > > when the scheduler runs concurrently on another CPU, _all_ the scheduling > > code is executed atomically wrt the spin lock taken on cpu 0. > > Sure, but taking the rq->lock is fairly heavy handed. > > > When x86 uses iret to return to user-space, then we have a serializing > > instruction. But if it uses sysexit, or if we are on a different > > architecture, are we sure that a memory barrier is issued before > > returning to user-space ? > > [*] and possibly also for matching ->mm's, because: > > OK, so I had a quick look at the switch_to() magic, and from what I can > make of it it implies an mb, if only because poking at the segment > registers implies LOCK semantics.
Can you have a look at the updated scenario and reply with questions that might arise ?
Thanks!
Mathieu
> >
-- Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
| |