lkml.org 
[lkml]   [2012]   [Mar]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH 3/4 changelog-v2] KVM: Switch to srcu-less get_dirty_log()
On Wed, Mar 07, 2012 at 05:07:45PM +0900, Takuya Yoshikawa wrote:
> Marcelo Tosatti <mtosatti@redhat.com> wrote:
>
> > > Partly yes: my method mainly depends on the number of dirty pages,
> > > not slot size.
> > >
> > > But it is not a new problem: traversing all shadow pages for that
> > > also takes linearly increasing time.
> >
> > It was not necessary to read the bitmap under mmu_lock previously.
>
> Let's check actual data!
>
> Below I pasted a simple test result to show that reading bitmap is not
> a problem at all compared to traversing shadow pages.
>
> ** During doing the same live migration test as:
>
> For real workloads, both VGA and live migration, we have observed pure
> improvements: when the guest was reading a file during live migration,
> we originally saw a few ms of latency, but with the new method the
> latency was less than 200us.
>
> I measured how long the current method takes to just write protect sptes
> with the mmu_lock held - kvm_mmu_slot_remove_write_access() time.
>
>
> You can see many ms order of protection times from this result: for me this
> is more problematic than downtime problem many people like to improve.
>
> In contrast my method only took 200us in the worst case: actually what I
> measured for that was the entire kvm_vm_ioctl_get_dirty_log() time which
> contained more extra tasks, e.g. copy_to_user().
>
> FYI: changing the guest memory size from 4GB to 8GB did not show any
> siginificant change to my method, but, as you can guess, traversing
> shadow pages will need more time for increased shadow pages.
>
>
> If we have 4K shadow pages in the slot, kvm_mmu_slot_remove_write_access()
> have to traverse all of them, checking all 512 entries in them.
>
> Compared to that, the bitmap size of 4GB memory slot is 1M bits = 128KB.
> Reading this 8 pages is negligible.

Right, thanks for checking it.

> My unit-test experiment has also showed that xchg overheads is not so much,
> compared to others:
>
> 493900.4 15911.9 60.2 125% 5% 8K
> 760268.2 5929.6 46.4 63% 199% 16K
> 1238709.6 7123.5 37.8 23% 173% 32K
> 2359523.6 3121.7 36.0 -9% 87% 64K
> 4540780.6 10155.6 34.6 -27% 30% 128K
> 8747274.0 10973.3 33.3 -31% -3% 256K
>
> Note that these cases need to xchg the entire dirty bitmap because at least
> one bit is set for each unsigned-long-word.
>
> The big difference came from the number of sptes to protect alone.
>
> Takuya

What is worrying are large memory cases: think of the 50GB slot case.
100ms hold time is pretty bad (and reacquiring the lock is relatively
simple).

>
> ===
> funcgraph_entry: + 25.123 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 35.746 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 922.886 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 20.153 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 20.424 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 17.595 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 20.240 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 9783.060 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1992.718 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1312.128 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2028.900 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1455.889 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1382.795 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2030.321 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1407.248 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2189.321 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1444.344 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2291.976 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1801.848 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1993.104 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1531.858 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2394.283 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1613.203 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1699.472 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2416.467 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1566.451 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1772.670 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1700.544 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1590.114 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2311.419 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1923.888 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2534.780 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2083.623 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1664.170 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2867.553 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2684.615 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1706.371 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2655.976 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1720.777 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2993.758 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1924.842 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3091.190 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1776.427 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2808.984 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2669.008 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2359.525 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2703.617 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2623.198 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1942.833 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1906.551 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2981.093 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2168.301 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1949.932 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2992.925 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3360.511 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1993.321 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3187.857 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 1989.417 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2001.865 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2047.220 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3107.808 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2039.732 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2057.575 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2417.748 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2076.445 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2308.323 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3216.713 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2148.263 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2269.673 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 2133.566 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3757.388 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3372.302 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3679.316 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3516.200 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 630.067 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 3191.830 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: ! 658.717 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 66.683 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: + 31.027 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.274 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.568 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.460 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.358 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.197 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.306 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.259 us | kvm_mmu_slot_remove_write_access();
> funcgraph_entry: 0.181 us | kvm_mmu_slot_remove_write_access();


\
 
 \ /
  Last update: 2012-03-08 00:31    [W:0.159 / U:0.596 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site