Messages in this thread | | | Date | Thu, 11 May 2023 20:59:28 +0100 | From | Jean-Philippe Brucker <> | Subject | Re: [PATCH v1 4/5] iommu/arm-smmu-v3: Keep track of attached ssids |
| |
On Thu, May 11, 2023 at 11:26:48PM +0800, Michael Shavit wrote: > > You should be getting rid of mm->pasid in this series as well. > > > > When each domain keeps track of what STE/CD entries that point to it then > > *ALL* invalidation should iterate over the list of pointing entires > > and generate the correct invalidation for that pointer. > > > > Completely agree. The arm_smmu_atc_inv_domain_ssid function introduced > by this patch is a stopgap to decompose this patch from the SVA > refactor that's required to stop using ssid in these calls. > I also agree that such a refactoring probably belongs in the same > patch series. @Jean-Philippe Brucker and others: is there any way I > can about testing or at least exercising the SVA flow without physical > hardware that supports SVA?
Yes, there is a model with a simple test device that supports PASID and I/O page faults. It's not completely straightforward to setup and the driver needs to be rewritten from scratch, but it's the best we have at the moment. I'd like to do something equally useful for QEMU, so we can have proper regression tests, but that requires a lot of preliminary work to add PASID+PRI to PCI, virtio and IOMMUs.
You'll need a kernel with the driver and a rootfs with the smmute tool [1]; the RevC model [2] and a boot-wrapper [3].
$ ${BOOTWRAPPER}/configure --host=aarch64-linux-gnu --with-dtb=${KERNEL}/arch/arm64/boot/dts/arm/fvp-base-revc.dts --with-kernel-dir=${KERNEL} --with-initrd=${BUILDROOT}/images/rootfs.cpio --with-cmdline="console=ttyAMA0" --enable-psci --enable-gicv3 $ make # produces linux-system.axf
Run the model: $ FVP_Base_RevC-2xAEMvA -C bp.secure_memory=0 -C 'pctl.startup=0.*.*.*' -C bp.refcounter.non_arch_start_at_default=1 -C cache_state_modelled=0 -C bp.vis.disable_visualisation=1 -C bp.virtio_net.enabled=1 -C bp.virtio_net.hostbridge.userNetPorts=8022=22 -C bp.virtio_net.hostbridge.userNetworking=1 -C pci.pci_smmuv3.mmu.SMMU_IDR0=135100351 -C pci.pci_smmuv3.mmu.SMMU_IDR3=4116 -C pci.pci_smmuv3.mmu.SMMU_IDR5=8389749 -C cluster0.NUM_CORES=4 -C cluster1.NUM_CORES=4 -a 'cluster*.cpu*=linux-system.axf'
Then run a job using the tool. The process allocates two buffers and passes their VA to the device (via the kernel driver). The device memcpies one buffer to the other:
$ smmute -u mmap ... Success
With smmu and iommu trace events enabled, a trace should contain smmu_mm_invalidate and dev_fault/dev_page_response events.
It's not entirely representative of SVA flow, where an assignable device interface is mapped into the process and the process launches jobs directly without going through the kernel (that would now use drivers/misc/uacce), but it does exercise IOMMU SVA: sva_bind(), device accessing the process address space with PASID and some IOPFs, which I think is what you're looking for. However this model doesn't have a PCI test device so you won't be able to test ATC invalidations with PASID.
Other useful tests would be enabling lockdep (some intricate locking between IOPF, the driver and mm), killing bound processes (-k), triggering invalid accesses to verify TLB invalidations (-f tlb, I think). There is a lot more to test, like thp and oom, but I don't have those in this branch.
Thanks, Jean
[1] https://jpbrucker.net/git/linux/log/?h=sva/smmute-revc [2] https://developer.arm.com/downloads/-/arm-ecosystem-models [3] https://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git/
| |