Messages in this thread Patch in this message | | | Date | Thu, 24 Aug 2017 14:33:04 +0200 | From | Peter Zijlstra <> | Subject | Re: [PATCH] rwsem: fix missed wakeup due to reordering of load |
| |
On Thu, Aug 24, 2017 at 01:29:27PM +0200, Peter Zijlstra wrote: > > WTH did you not Cc the people that commented on your patch last time? > > On Wed, Aug 23, 2017 at 04:58:55PM +0530, Prateek Sood wrote: > > If a spinner is present, there is a chance that the load of > > rwsem_has_spinner() in rwsem_wake() can be reordered with > > respect to decrement of rwsem count in __up_write() leading > > to wakeup being missed. > > > spinning writer up_write caller > > --------------- ----------------------- > > [S] osq_unlock() [L] osq > > spin_lock(wait_lock) > > sem->count=0xFFFFFFFF00000001 > > +0xFFFFFFFF00000000 > > count=sem->count > > MB > > sem->count=0xFFFFFFFE00000001 > > -0xFFFFFFFF00000001 > > RMB > > This doesn't make sense, it appears to order a STORE against something > else. > > > spin_trylock(wait_lock) > > return > > rwsem_try_write_lock(count) > > spin_unlock(wait_lock) > > schedule()
Is this what you wanted to write?
--- kernel/locking/rwsem-xadd.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c index 02f660666ab8..813b5d3654ce 100644 --- a/kernel/locking/rwsem-xadd.c +++ b/kernel/locking/rwsem-xadd.c @@ -613,6 +613,33 @@ struct rw_semaphore *rwsem_wake(struct rw_semaphore *sem) DEFINE_WAKE_Q(wake_q); /* + * __rwsem_down_write_failed_common(sem) + * rwsem_optimistic_spin(sem) + * osq_unlock(sem->osq) + * ... + * atomic_long_add_return(&sem->count) + * + * - VS - + * + * __up_write() + * if (atomic_long_sub_return_release(&sem->count) < 0) + * rwsem_wake(sem) + * osq_is_locked(&sem->osq) + * + * And __up_write() must observe !osq_is_locked() when it observes the + * atomic_long_add_return() in order to not miss a wakeup. + * + * This boils down to: + * + * [S.rel] X = 1 [RmW] r0 = (Y += 0) + * MB RMB + * [RmW] Y += 1 [L] r1 = X + * + * exists (r0=1 /\ r1=0) + */ + smp_rmb(); + + /* * If a spinner is present, it is not necessary to do the wakeup. * Try to do wakeup only if the trylock succeeds to minimize * spinlock contention which may introduce too much delay in the
| |