Messages in this thread | | | From | Sunil Muthuswamy <> | Subject | RE: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS | Date | Wed, 4 Aug 2021 20:10:43 +0000 |
| |
On Wednesday, August 4, 2021 2:21 AM, Marc Zyngier <maz@kernel.org> wrote: > > On Tue, 03 Aug 2021 09:35:12 +0100, > Robin Murphy <robin.murphy@arm.com> wrote: > > > > On 2021-08-03 03:11, Sunil Muthuswamy wrote: > > > On Saturday, July 31, 2021 2:52 AM, > > > Marc Zyngier <maz@kernel.org> wrote: > > >> > > >> [...] > > >> > > >>>> I also want to understand *how* you are going to plumb this into both > > >>>> ACPI and DT, given that neither understand how to link a PCI endpoint > > >>>> to a set of RDs. > > >>>> > > >>>> M. > > >>> > > >>> One way to do this for NUMA-aware systems would be to use the NUMA > > >>> related information that is available with PCI endpoints or root complex, to > > >>> pick a Redistributor/CPU that is in the NUMA node, as specified by the PCI > > >>> endpoint/root complex. In DT PCI devices can specify this using > > >>> 'numa-node-id' and in ACPI using the '_PXM (Proximity)'. For systems that > > >>> are not NUMA-aware, we can go with *any* Redistributor/CPU. > > >> > > >> This makes zero sense. From the point of view of a device, all the RDs > > >> should be reachable, and firmware has no say in it. Dealing with > > >> interrupt affinity is the responsibility of the endpoint driver, and > > >> NUMA affinity is only a performance optimisation. > > >> > > >>> Is there any additional information we would be able to gather from ACPI > > >>> or DT that's not there currently, that would be useful here? > > >> > > >> You will need some firmware information describing that a given set of > > >> devices must use the RDs for their MSIs. Just like we currently > > >> describe it in IORT for the ITS. You cannot /assume/ things. At the > > >> moment, there is nothing at all, because no-one (including Microsoft) > > >> thought it would be a good idea not to have an ITS, which is also why > > >> ACPI doesn't describe MBIs as a potential MSI provider. > > >> > > > I am a little bit confused by your above comment. Maybe you can help me > > > understand the ask. You indicate that from the point of the view of the > > > device, all the RDs should be reachable. But, then if we define a mapping > > > between PCI endpoint and RD in the firmware, we would be doing exactly > > > the opposite. i.e. restricting the RDs that are reachable by the device. Can > > > you please clarify? > > Not at all. What I am saying is that you need to describe that the > MSIs have to be routed to the RDs, and there is no way to express this > at the moment. > > > > > > > Is your concern that the device should be able to only DMA to a subset of > > > GIC Redistributor, for the MSIs? If so, in the IORT, there is "memory address > > > size limit" for both device and root complex nodes. In the implementation, > > > we can enforce that the GICR is within that range. And, if a device deviates > > > further than that (ex: by having accessibility gaps within the GICR range), > > > then that is out of scope for support. > > > > No, please don't try to abuse the Memory Address Size Limit - that has > > far more chance of adversely affecting normal DMA operation than of > > being any use here. > > > > I believe the point Marc was trying to make is that firmware should > > not associate a device with any one *specific* redistributor, however > > ACPI currently has no way to describe that MSIs can target > > redistributors *at all*, only ITS groups - there is no such concept as > > a "redistributor group". > > Thanks Robin. > > That is exactly my point. There is no linkage whatsoever between a > device generating MSIs and the redistributors. In that respect, this > is the same issue we have with DT, as none of the firmware interfaces > can currently describe MSIs directly targeting the RDs. The only way > to describe MSIs with GICv3 in ACPI is to describe an ITS, and you > cannot use the *absence* of an ITS to decide and use DirectLPIs. > > Given the state of things, using DirectLPI means that you need to > extend the firmware interfaces. This means both an extension to the DT > binding, and an update to the ACPI spec. There is no way around this. > > Thanks, > > M. > Thanks Marc and Robin for clarifying. I see and understand the point about having explicit MSI mappings in the firmware specification for Direct LPIs for generic hardware support.
Hey Mark, would you be willing to consider a scoped down implementation of GIC Direct LPI with just an IRQ chip implementation and no Direct LPI PCI-MSI IRQ chip. This will allow a MSI provider (such as Hyper-V vPCI) to provide a PCI-MSI IRQ chip on top of the Direct LPI IRQ chip and enable PCI-MSI scenarios, and avoid building in assumptions in other cases (like PCI) where firmware specification is not available. - This scoped down implementation would allow Microsoft to build virtual PCI on top, to enable our business needs. - If there's a need for a generic support for a Direct LPI PCI MSI IRQ, that could drive firmware revision efforts, and we are happy to assist there.
Looking forward to hearing back.
Thanks, Sunil
| |