lkml.org 
[lkml]   [2015]   [Apr]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH 0/2] powerpc/kvm: Enable running guests on RT Linux
On 21.04.2015 03:52, Scott Wood wrote:
> On Mon, 2015-04-20 at 13:53 +0300, Purcareata Bogdan wrote:
>> There was a weird situation for .kvmppc_mpic_set_epr - its corresponding inner
>> function is kvmppc_set_epr, which is a static inline. Removing the static inline
>> yields a compiler crash (Segmentation fault (core dumped) -
>> scripts/Makefile.build:441: recipe for target 'arch/powerpc/kvm/kvm.o' failed),
>> but that's a different story, so I just let it be for now. Point is the time may
>> include other work after the lock has been released, but before the function
>> actually returned. I noticed this was the case for .kvm_set_msi, which could
>> work up to 90 ms, not actually under the lock. This made me change what I'm
>> looking at.
>
> kvm_set_msi does pretty much nothing outside the lock -- I suspect
> you're measuring an interrupt that happened as soon as the lock was
> released.

That's exactly right. I've seen things like a timer interrupt occuring right
after the spinlock_irqrestore, but before kvm_set_msi actually returned.

[...]

>> Or perhaps a different stress scenario involving a lot of VCPUs
>> and external interrupts?
>
> You could instrument the MPIC code to find out how many loop iterations
> you maxed out on, and compare that to the theoretical maximum.

Numbers are pretty low, and I'll try to explain based on my observations.

The problematic section in openpic_update_irq is this [1], since it loops
through all VCPUs, and IRQ_local_pipe further calls IRQ_check, which loops
through all pending interrupts for a VCPU [2].

The guest interfaces are virtio-vhostnet, which are based on MSI
(/proc/interrupts in guest shows they are MSI). For external interrupts to the
guest, the irq_source destmask is currently 0, and last_cpu is 0 (unitialized),
so [1] will go on and deliver the interrupt directly and unicast (no VCPUs loop).

I activated the pr_debugs in arch/powerpc/kvm/mpic.c, to see how many interrupts
are actually pending for the destination VCPU. At most, there were 3 interrupts
- n_IRQ = {224,225,226} - even for 24 flows of ping flood. I understand that
guest virtio interrupts are cascaded over 1 or a couple of shared MSI interrupts.

So worst case, in this scenario, was checking the priorities for 3 pending
interrupts for 1 VCPU. Something like this (some of my prints included):

[61010.582033] openpic_update_irq: destmask 1 last_cpu 0
[61010.582034] openpic_update_irq: Only one CPU is allowed to receive this IRQ
[61010.582036] IRQ_local_pipe: IRQ 224 active 0 was 1
[61010.582037] IRQ_check: irq 226 set ivpr_pr=8 pr=-1
[61010.582038] IRQ_check: irq 225 set ivpr_pr=8 pr=-1
[61010.582039] IRQ_check: irq 224 set ivpr_pr=8 pr=-1

It would be really helpful to get your comments regarding whether these are
realistical number for everyday use, or they are relevant only to this
particular scenario.

- Can these interrupts be used in directed delivery, so that the destination
mask can include multiple VCPUs? The MPIC manual states that timer and IPI
interrupts are supported for directed delivery, altough I'm not sure how much of
this is used in the emulation. I know that kvmppc uses the decrementer outside
of the MPIC.

- How are virtio interrupts cascaded over the shared MSI interrupts?
/proc/device-tree/soc@e0000000/msi@41600/interrupts in the guest shows 8 values
- 224 - 231 - so at most there might be 8 pending interrupts in IRQ_check, is
that correct?

Looking forward to your feedback.

[1] http://lxr.free-electrons.com/source/arch/powerpc/kvm/mpic.c#L454
[2] http://lxr.free-electrons.com/source/arch/powerpc/kvm/mpic.c#L303
[3]
https://www-01.ibm.com/chips/techlib/techlib.nsf/techdocs/F27971551C9EED8E8525774A0048770A/$file/mpic_db_05_16_2011.pdf

Best regards,
Bogdan P.


\
 
 \ /
  Last update: 2015-04-22 14:41    [W:0.139 / U:1.012 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site