lkml.org 
[lkml]   [2010]   [Apr]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
SubjectRE: VM performance issue in KVM guests.
Avi Kivity wrote:
> On 04/14/2010 06:24 AM, Zhang, Xiantao wrote:
>>
>>>>> Spin loops need to be addressed first, they are known to kill
>>>>> performance in overcommit situations.
>>>>>
>>>>>
>>>> Even in overcommit case, if vcpu threads of one qemu are not
>>>> scheduled or pulled to the same logical processor, the performance
>>>> drop is tolerant like Xen's case today. But for KVM, it has to
>>>> suffer from additional performance loss, since host's scheduler
>>>> actively pulls these vcpu threads together.
>>>>
>>>>
>>>>
>>> Can you quantify this loss? Give examples of what happens?
>>>
>> For example, one machine is configured with 2 pCPUs and there are
>> two Windows guests running on the machine, and each guest is
>> cconfigured with 2 vcpus and one webbench server runs in it.
>> If use host's default scheduler, webbench's performance is very bad,
>> but if pin each geust's vCPU0 to pCPU0 and vCPU1 to pCPU1, we can
>> see 5-10X performance improvement with same CPU utilization.
>> In addition, we also see kvm's perf scalability is also impacted in
>> large systems, for some performance experiments, kvm's perf begins
>> to drop when vCPU is overcommitted and pCPU are saturated, but once
>> the wake_up_affine feature is switched off in scheduler, kvm's perf
>> can keep rising in this case.
>>
>
> Ok. This is probably due to spinlock contention.

Yes, exactly.

> When vcpus are pinned to pcpus, there is a 50% chance that a guest's
> vcpus will be co-scheduled and spinlocks will perform will.
>
> When vcpus are not pinned, but affine wakeups are disabled, there is a
> 33% chance that vcpus will be co-scheduled.
>
> When vcpus are not pinned and affine wakeups are enabled there is a 0%
> chance that vcpus will be co-scheduled.
>
> Keeping both vcpus on the same core actually makes sense since they
> can communicate through the local cache faster than across cores.
> What we need is to make sure that they don't spin.
>
> Windows 2008 can report spinlock spinning through a hypercall. Can
> you hook to that interface and see if it happens regularly?
> Altenatively use a PLE capable host and trace the kvm_vcpu_on_spin()
> function.
We only tried windows 2003 for the experiments, and have no data related to windows 2008.
But maybe we can have a try later. Anyway, the key point is we have to enhance the scheduler to let it
Know which threads are vcpu threads to avoid perf loss in this case.
Xiantao


\
 
 \ /
  Last update: 2010-04-16 04:31    [W:0.108 / U:0.056 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site