[lkml]   [2015]   [Mar]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Alternative to signals/sys_membarrier() in liburcu

* Mathieu Desnoyers <> wrote:

> ----- Original Message -----
> > From: "Linus Torvalds" <>
> > To: "Mathieu Desnoyers" <>
> > Cc: "Michael Sullivan" <>,, "LKML" <>, "Paul E.
> > McKenney" <>, "Peter Zijlstra" <>, "Ingo Molnar" <>,
> > "Thomas Gleixner" <>, "Steven Rostedt" <>
> > Sent: Thursday, March 12, 2015 5:47:05 PM
> > Subject: Re: Alternative to signals/sys_membarrier() in liburcu
> >
> > On Thu, Mar 12, 2015 at 1:53 PM, Mathieu Desnoyers
> > <> wrote:
> > >
> > > So the question as it stands appears to be: would you be comfortable
> > > having users abuse mprotect(), relying on its side-effect of issuing
> > > a smp_mb() on each targeted CPU for the TLB shootdown, as
> > > an effective implementation of process-wide memory barrier ?
> >
> > Be *very* careful.
> >
> > Just yesterday, in another thread (discussing the auto-numa TLB
> > performance regression), we were discussing skipping the TLB
> > invalidates entirely if the mprotect relaxes the protections.

We have such code already in mm/mprotect.c, introduced in:

10c1045f28e8 mm: numa: avoid unnecessary TLB flushes when setting NUMA hinting entries

which does:

/* Avoid TLB flush if possible */
if (pte_protnone(oldpte))

> > Because if you *used* to be read-only, and them mprotect()
> > something so that it is read-write, there really is no need to
> > send a TLB invalidate, at least on x86. You can just change the
> > page tables, and *if* any entries are stale in the TLB they'll
> > take a microfault on access and then just reload the TLB.
> >
> > So mprotect() to a more permissive mode is not necessarily
> > serializing.
> The idea here is to always mprotect() to a more restrictive mode,
> which should trigger the TLB shootdown.

So what happens if a CPU comes around that integrates TLB shootdown
management into its cache coherency protocol? In such a case IPI
traffic can be skipped: the memory bus messages take care of TLB
flushes in most cases.

It's a natural optimization IMHO, because TLB flushes are conceptually
pretty close to the synchronization mechanisms inherent in data cache
coherency protocols:

This could be implemented for example by a CPU that knows about ptes
and handles their modification differently: when a pte is modified it
will broadcast a MESI invalidation message not just for the cacheline
belonging to the pte's physical address, but also an 'invalidate TLB'
MESI message for the pte value's page.

The TLB shootdown would either be guaranteed within the MESI
transaction, or there would either be a deterministic timing
guarantee, or some explicit synchronization mechanism (new
instruction) to make sure the remote TLB(s) got shot down.

Every form of this would be way faster than sending interrupts. New
OSs could support this by the hardware telling them in which cases the
TLBs are 'auto-flushed', while old OSs would still be compatible by
sending (now pointless) TLB shootdown IPIs.

So it's a relatively straightforward hardware optimization IMHO:
assuming TLB flushes are considered important enough to complicate the
cacheline state machine (which I think they currently aren't).

So in this case there's no interrupt and no other interruption of the
remote CPU's flow of execution in any fashion that could advance the
RCU state machine.

What do you think?



 \ /
  Last update: 2015-03-13 09:21    [W:0.040 / U:9.648 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site