Messages in this thread | | | Date | Sun, 10 Jan 2010 11:03:14 -0500 | From | Mathieu Desnoyers <> | Subject | Re: [RFC PATCH] introduce sys_membarrier(): process-wide memory barrier |
| |
* Steven Rostedt (rostedt@goodmis.org) wrote: > On Sat, 2010-01-09 at 21:25 -0800, Paul E. McKenney wrote: > > On Sat, Jan 09, 2010 at 09:12:58PM -0500, Steven Rostedt wrote: > > > > > < user space > > > > > > > > > < misses that CPU 2 is in rcu section > > > > > > > > > > If the TLB flush misses that CPU 2 has a threaded task, and does not > > > flush CPU 2s TLB, it can also risk the same type of crash. > > > > But isn't the VM's locking helping us out in that case? > > > > > > [CPU 2's ->curr update now visible] > > > > > > > > [CPU 2's rcu_read_lock() store now visible] > > > > > > > > free(obj); > > > > > > > > use_object(obj); <=== crash! > > > > > > > > > > Think about it. If you change a process mmap, say you updated a mmap of > > > a file by flushing out one page and replacing it with another. If the > > > above missed sending to CPU 2, then CPU 2 may still be accessing the old > > > page of the file, and not the new one. > > > > > > I think this may be the safe bet. > > > > You might well be correct that we can access that bitmap locklessly, > > but there are additional things (like the loading of the arch-specific > > page-table register) that are likely to be helping in the VM case, but > > not necessarily helping in this case. > > > Then perhaps the sys_membarrier() should just do a flush_tlb()? That > should guarantee the synchronization, right? >
The way I see it, TLB can be seen as read-only elements (a local read-only cache) on the processors. Therefore, we don't care if they are in a stale state while performing the cpumask update, because the fact that we are executing switch_mm() means that these TLB entries are not being used locally anyway and will be dropped shortly. So we have the equivalent of a full memory barrier (load_cr3()) _after_ the cpumask updates.
However, in sys_membarrier(), we also need to flush the write buffers present on each processor running threads which belong to our current process. Therefore, we would need, in addition, a smp_mb() before the mm cpumask modification. For x86, cpumask_clear_cpu/cpumask_set_cpu implies a LOCK-prefixed operation, and hence does not need any added barrier, but this could be different for other architectures.
So, AFAIK, doing a flush_tlb() would not guarantee the kind of synchronization we are looking for because an uncommitted write buffer could still sit on the remote CPU when we return from sys_membarrier().
Thanks,
Mathieu
-- Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
| |