[lkml]   [2010]   [Jun]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Patch in this message
    SubjectRe: [PATCH] use unfair spinlock when running on hypervisor.
    On Wed, Jun 02, 2010 at 12:00:27PM +0300, Avi Kivity wrote:
    > There are two separate problems: the more general problem is that
    > the hypervisor can put a vcpu to sleep while holding a lock, causing
    > other vcpus to spin until the end of their time slice. This can
    > only be addressed with hypervisor help.

    Fyi - I have a early patch ready to address this issue. Basically I am using
    host-kernel memory (mmap'ed into guest as io-memory via ivshmem driver) to hint
    host whenever guest is in spin-lock'ed section, which is read by host scheduler
    to defer preemption.

    Guest side:

    static inline void spin_lock(spinlock_t *lock)
    + __get_cpu_var(gh_vcpu_ptr)->defer_preempt++;

    static inline void spin_unlock(spinlock_t *lock)
    + __get_cpu_var(gh_vcpu_ptr)->defer_preempt--;

    [similar changes to other spinlock variants]

    Host side:

    @@ -860,6 +866,17 @@ check_preempt_tick(struct cfs_rq *cfs_rq
    ideal_runtime = sched_slice(cfs_rq, curr);
    delta_exec = curr->sum_exec_runtime - curr->prev_sum_exec_runtime;
    if (delta_exec > ideal_runtime) {
    + if ((sched_feat(DEFER_PREEMPT)) && (rq_of(cfs_rq)->curr->ghptr)) {
    + int defer_preempt = rq_of(cfs_rq)->curr->ghptr->defer_preempt;
    + if (((defer_preempt & 0xFFFF0000) == 0xfeed0000) && ((defer_preempt & 0x0000FFFF) != 0)) {
    + if ((rq_of(cfs_rq)->curr->grace_defer++ < sysctl_sched_preempt_defer_count)) {
    + rq_of(cfs_rq)->defer_preempt++;
    + return;
    + } else
    + rq_of(cfs_rq)->force_preempt++;
    + }
    + }
    * The current task ran long enough, ensure it doesn't get
    [similar changes introduced at other preemption points in sched_fair.c]

    Note that guest can only request preemption to be deferred (and not disabled via
    this mechanism). I have seen good improvement (~15%) in kern compile benchmark
    with sysctl_sched_preempt_defer_count set to a low value of just 2 (i.e we can
    defer preemption by maximum two ticks). I intend to cleanup and post the patches
    pretty soon for comments.

    One pathological case where this may actually hurt is routines in guest like
    flush_tlb_others_ipi() which take a spinlock and then enter a while() loop
    waiting for other cpus to ack something. In this case, deferring preemption just
    because guest is in critical section actually hurts! Hopefully the upper bound
    for deferring preemtion and the fact that such routines may not be frequently
    hit should help alleviate such situations.
    - vatsa

     \ /
      Last update: 2010-06-03 06:23    [W:0.023 / U:2.944 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site