Messages in this thread | | | Date | Sat, 9 Jan 2010 21:25:08 -0800 | From | "Paul E. McKenney" <> | Subject | Re: [RFC PATCH] introduce sys_membarrier(): process-wide memory barrier |
| |
On Sat, Jan 09, 2010 at 09:12:58PM -0500, Steven Rostedt wrote: > On Sat, 2010-01-09 at 20:44 -0500, Mathieu Desnoyers wrote: > > > > So what if we have a linear decrease in performance with the number of > > > threads on the write side? > > > > Hrm, looking at arch/x86/include/asm/mmu_context.h > > > > switch_mm(), which is basically called each time the scheduler needs to > > change the current task, does a > > > > cpumask_clear_cpu(cpu, mm_cpumask(prev)); > > > > and > > > > cpumask_set_cpu(cpu, mm_cpumask(next)); > > > > which precise goal is to stop the flush ipis for the previous mm. The > > 100$ question is : why do we have to confirm that the thread is indeed > > on the runqueue (taking locks and everything) when we could simply just > > bluntly use the mm_cpumask for our own IPIs ? > > I was just looking at that code, and was thinking the same thing ;-) > > > cpumask_clear_cpu and cpumask_set_cpu translate into clear_bit/set_bit. > > cpumask_next does a find_next_bit on the cpumask. > > > > clear_bit/set_bit are atomic and not reordered on x86. PowerPC also uses > > ll/sc loops in bitops.h, so I think it should be pretty safe to assume > > that mm_cpumask is, by design, made to be used as cpumask to send a > > broadcast IPI to all CPUs which run threads belonging to a given > > process. > > > > So, how about just using mm_cpumask(current) for the broadcast ? Then we > > don't even need to allocate our own cpumask neither. > > > > Or am I missing something ? I just sounds too simple. > > I think we can use it. If for some reason it does not satisfy what you > need then I also think the TLB flushing is also broken. > > IIRC, (Paul help me out on this), what Paul said earlier, we are trying > to protect against this scenario: > > (from Paul's email:) > > > > > > CPU 1 CPU 2 > > ----------- ------------- > > > > <user space> <kernel space, switching to task> > > > > ->curr updated > > > > <long code path, maybe mb?> > > > > <user space> > > > > rcu_read_lock(); [load only] > > > > obj = list->next > > > > list_del(obj) > > > > sys_membarrier(); > > < kernel space > > > > > if (task_rq(task)->curr != task) > > < but load to obj reordered before store to ->curr > > > > > < user space > > > > > < misses that CPU 2 is in rcu section > > > > If the TLB flush misses that CPU 2 has a threaded task, and does not > flush CPU 2s TLB, it can also risk the same type of crash.
But isn't the VM's locking helping us out in that case?
> > [CPU 2's ->curr update now visible] > > > > [CPU 2's rcu_read_lock() store now visible] > > > > free(obj); > > > > use_object(obj); <=== crash! > > > > Think about it. If you change a process mmap, say you updated a mmap of > a file by flushing out one page and replacing it with another. If the > above missed sending to CPU 2, then CPU 2 may still be accessing the old > page of the file, and not the new one. > > I think this may be the safe bet.
You might well be correct that we can access that bitmap locklessly, but there are additional things (like the loading of the arch-specific page-table register) that are likely to be helping in the VM case, but not necessarily helping in this case.
Thanx, Paul
| |