lkml.org 
[lkml]   [2016]   [May]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH] locking/rwsem: Optimize write lock slowpath
    On 05/09/2016 03:16 PM, Jason Low wrote:
    > When acquiring the rwsem write lock in the slowpath, we first try
    > to set count to RWSEM_WAITING_BIAS. When that is successful,
    > we then atomically add the RWSEM_WAITING_BIAS in cases where
    > there are other tasks on the wait list. This causes write lock
    > operations to often issue multiple atomic operations.
    >
    > We can instead make the list_is_singular() check first, and then
    > set the count accordingly, so that we issue at most 1 atomic
    > operation when acquiring the write lock and reduce unnecessary
    > cacheline contention.
    >
    > Signed-off-by: Jason Low<jason.low2@hp.com>
    > ---
    > kernel/locking/rwsem-xadd.c | 20 +++++++++++++-------
    > 1 file changed, 13 insertions(+), 7 deletions(-)
    >
    > diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c
    > index df4dcb8..23c33e6 100644
    > --- a/kernel/locking/rwsem-xadd.c
    > +++ b/kernel/locking/rwsem-xadd.c
    > @@ -258,14 +258,20 @@ EXPORT_SYMBOL(rwsem_down_read_failed);
    > static inline bool rwsem_try_write_lock(long count, struct rw_semaphore *sem)
    > {
    > /*
    > - * Try acquiring the write lock. Check count first in order
    > - * to reduce unnecessary expensive cmpxchg() operations.
    > + * Avoid trying to acquire write lock if count isn't RWSEM_WAITING_BIAS.
    > */
    > - if (count == RWSEM_WAITING_BIAS&&
    > - cmpxchg_acquire(&sem->count, RWSEM_WAITING_BIAS,
    > - RWSEM_ACTIVE_WRITE_BIAS) == RWSEM_WAITING_BIAS) {
    > - if (!list_is_singular(&sem->wait_list))
    > - rwsem_atomic_update(RWSEM_WAITING_BIAS, sem);
    > + if (count != RWSEM_WAITING_BIAS)
    > + return false;
    > +
    > + /*
    > + * Acquire the lock by trying to set it to ACTIVE_WRITE_BIAS. If there
    > + * are other tasks on the wait list, we need to add on WAITING_BIAS.
    > + */
    > + count = list_is_singular(&sem->wait_list) ?
    > + RWSEM_ACTIVE_WRITE_BIAS :
    > + RWSEM_ACTIVE_WRITE_BIAS + RWSEM_WAITING_BIAS;
    > +
    > + if (cmpxchg_acquire(&sem->count, RWSEM_WAITING_BIAS, count) == RWSEM_WAITING_BIAS) {
    > rwsem_set_owner(sem);
    > return true;
    > }

    Acked-by: Waiman Long<Waiman.Long@hpe.com>

    \
     
     \ /
      Last update: 2016-05-10 04:41    [W:5.266 / U:0.008 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site