lkml.org 
[lkml]   [2008]   [Jan]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: runqueue locks in schedule()
    Nick,

    On Jan 18, 2008 3:07 AM, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
    >
    > On Friday 18 January 2008 00:24, Peter Zijlstra wrote:
    > > [ At the very least CC'ing the scheduler maintainer would be
    > > helpful :-) ]
    > >
    > > On Wed, 2008-01-16 at 16:29 -0800, stephane eranian wrote:
    > > > Hello,
    > > >
    > > > As suggested by people on this list, I have changed perfmon2 to use
    > > > the high resolution timers as the interface to allow timeout-based
    > > > event set multiplexing. This works around the problems I had with
    > > > tickless-enabled kernels.
    > > >
    > > > Multiplexing is supported in per-thread as well. In that case, the
    > > > timeout measures virtual time. When the thread is context switched
    > > > out, we need to save the remainder of the timeout and cancel the
    > > > timer. When the thread is context switched in, we need to reinstall
    > > > the timer. These timer save/restore operations have to be done in the
    > > > switch_to() code near the end of schedule().
    > > >
    > > > There are situations where hrtimer_start() may end up trying to
    > > > acquire the runqueue lock. This happens on a context switch where the
    > > > current thread is blocking (not preempted) and the new timeout happens
    > > > to be either in the past or just expiring. We've run into such
    > > > situations with simple tests.
    > > >
    > > > On all architectures, but IA-64, it seems thet the runqueue lock is
    > > > held until the end of schedule(). On IA-64, the lock is released
    > > > BEFORE switch_to() for some reason I don't quite remember. That may
    > > > not even be needed anymore.
    > > >
    > > > The early unlocking is controlled by a macro named
    > > > __ARCH_WANT_UNLOCKED_CTXSW. Defining this macros on X86 (or PPC) fixed
    > > > our problem.
    > > >
    > > > It is not clear to me why the runqueue lock needs to be held up until
    > > > the end of schedule() on some platforms and not on others. Not that
    > > > releasing the lock earlier does not necessarily introduce more
    > > > overhead because the lock is never re-acquired later in the schedule()
    > > > function.
    > > >
    > > > Question:
    > > > - is it safe to release the lock before switch_to() on all
    > > > architectures?
    > >
    > > I had similar problem when using hrtimers from the scheduler, I extended
    > > the HRTIMER_CB_IRQSAFE_NO_SOFTIRQ time type to run with cpu_base->lock
    > > unlocked.
    > >
    > > http://git.kernel.org/?p=linux/kernel/git/mingo/linux-2.6-sched-devel.git;a
    > >=commitdiff;h=7e7cbd617833dde5b442e03f69aac39d17d02ec7
    > > http://git.kernel.org/?p=linux/kernel/git/mingo/linux-2.6-sched-devel.git;a
    > >=commitdiff;h=45d10aad580a5cdd376e80848aeeaaaf1f97cc18
    > > http://git.kernel.org/?p=linux/kernel/git/mingo/linux-2.6-sched-devel.git;a
    > >=commitdiff;h=5ae5d6c5850d4735798bc0e4526d8c61199e9f93
    > >
    > > As for your __ARCH_WANT_UNLOCKED_CTXSW question I have to defer to Ingo,
    > > as I'm unaware of the arch ramifications there.
    >
    > It is arch specific. If an architecture wants interrupts on during context
    > switch, or runqueue unlocked, then they set it (btw INTERRUPTS_ON_CTXSW
    > also implies UNLOCKED_CTXSW).
    >
    Yes , I noticed that. I am only interested in UNLOCKED_CTXSW.
    But it appears that the approach suggested my Peter does work. We are
    running some tests.

    > Although, eg on x86, you would hold off interrupts and runqueue lock for
    > slightly less time if you defined those, it results in _slightly_ more
    > complicated context switching... although I did once find a workload
    > where the reduced runqueue contention improved throughput a bit, it is
    > not much problem in general to hold the lock.
    >
    By complicated you mean that now you'd have to make sure you don't
    need to access runqueue data?

    Thanks.


    \
     
     \ /
      Last update: 2008-01-18 07:37    [W:0.032 / U:0.080 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site