`On Mon, Oct 01, 2007 at 11:44:25AM -0700, Davide Libenzi wrote:> On Sun, 30 Sep 2007, Paul E. McKenney wrote:> > > On Sun, Sep 30, 2007 at 04:02:09PM -0700, Davide Libenzi wrote:> > > On Sun, 30 Sep 2007, Oleg Nesterov wrote:> > > > > > > Ah, but I asked the different question. We must see CPU 1's stores by> > > > definition, but what about CPU 0's stores (which could be seen by CPU 1)?> > > > > > > > Let's take a "real life" example,> > > > > > > >                 A = B = X = 0;> > > >                 P = Q = &A;> > > > > > > > CPU_0           CPU_1           CPU_2> > > > > > > > P = &B;         *P = 1;         if (X) {> > > >                 wmb();                  rmb();> > > >                 X = 1;                  BUG_ON(*P != 1 && *Q != 1);> > > >                                 }> > > > > > > > So, is it possible that CPU_1 sees P == &B, but CPU_2 sees P == &A ?> > > > > > That can't be. CPU_2 sees X=1, that happened after (or same time at most - > > > from a cache inv. POV) to *P=1, that must have happened after P=&B (in > > > order for *P to assign B). So P=&B happened, from a pure time POV, before > > > the rmb(), and the rmb() should guarantee that CPU_2 sees P=&B too.> > > > Actually, CPU designers have to go quite a ways out of their way to> > prevent this BUG_ON from happening.  One way that it would happen> > naturally would be if the cache line containing P were owned by CPU 2,> > and if CPUs 0 and 1 shared a store buffer that they both snooped.  So,> > here is what could happen given careless or sadistic CPU designers:> > Ohh, I misinterpreted that rmb(), sorry. Somehow I gave it for granted> that it was a cross-CPU sync point (ala read_barrier_depends). If that's a> local CPU load ordering only, things are different, clearly. But ...> > > o	CPU 0 stores &B to P, but misses the cache, so puts the> > 	result in the store buffer.  This means that only CPUs 0 and 1> > 	can see it.> > > > o	CPU 1 fetches P, and sees &B, so stores a 1 to B.  Again,> > 	this value for P is visible only to CPUs 0 and 1.> > > > o	CPU 1 executes a wmb(), which forces CPU 1's stores to happen> > 	in order.  But it does nothing about CPU 0's stores, nor about CPU> > 	1's loads, for that matter (and the only reason that POWER ends> > 	up working the way you would like is because wmb() turns into> > 	"sync" rather than the "eieio" instruction that would have been> > 	used for smp_wmb() -- which is maybe what Oleg was thinking of,> > 	but happened to abbreviate.  If my analysis is buggy, Anton and> > 	Paulus will no doubt correct me...)> > If a store buffer is shared between CPU_0 and CPU_1, it is very likely > that a sync done on CPU_1 is going to sync even CPU_0 stores that are > held in the buffer at the time of CPU_1's sync.That would indeed be one approach that CPU designers could take toavoid being careless or sadistic.  ;-)Another approach would be to make CPU 1 refrain from snooping CPU 0'sentries in the shared store queue.							Thanx, Paul-To unsubscribe from this list: send the line "unsubscribe linux-kernel" inthe body of a message to majordomo@vger.kernel.orgMore majordomo info at  http://vger.kernel.org/majordomo-info.htmlPlease read the FAQ at  http://www.tux.org/lkml/`