Messages in this thread | | | From | Michael Shavit <> | Date | Tue, 23 May 2023 15:57:48 +0800 | Subject | Re: [PATCH v1 4/5] iommu/arm-smmu-v3: Keep track of attached ssids |
| |
Oh nice, this is exactly what I was looking for (minus the missing ATC inv, but that's somewhat easier to reason from code)! Thanks for the detailed guide Jean! I finally got around to trying it out and was able to see the page fault followed by invalidations on this patch-series as it is. This will be super useful to start refactoring SVA for a v2 of this patch.
On Fri, May 12, 2023 at 3:59 AM Jean-Philippe Brucker <jean-philippe@linaro.org> wrote: > > On Thu, May 11, 2023 at 11:26:48PM +0800, Michael Shavit wrote: > > > You should be getting rid of mm->pasid in this series as well. > > > > > > When each domain keeps track of what STE/CD entries that point to it then > > > *ALL* invalidation should iterate over the list of pointing entires > > > and generate the correct invalidation for that pointer. > > > > > > > Completely agree. The arm_smmu_atc_inv_domain_ssid function introduced > > by this patch is a stopgap to decompose this patch from the SVA > > refactor that's required to stop using ssid in these calls. > > I also agree that such a refactoring probably belongs in the same > > patch series. @Jean-Philippe Brucker and others: is there any way I > > can about testing or at least exercising the SVA flow without physical > > hardware that supports SVA? > > Yes, there is a model with a simple test device that supports PASID and > I/O page faults. It's not completely straightforward to setup and the > driver needs to be rewritten from scratch, but it's the best we have at > the moment. I'd like to do something equally useful for QEMU, so we can > have proper regression tests, but that requires a lot of preliminary work > to add PASID+PRI to PCI, virtio and IOMMUs. > > You'll need a kernel with the driver and a rootfs with the smmute > tool [1]; the RevC model [2] and a boot-wrapper [3]. > > $ ${BOOTWRAPPER}/configure --host=aarch64-linux-gnu > --with-dtb=${KERNEL}/arch/arm64/boot/dts/arm/fvp-base-revc.dts > --with-kernel-dir=${KERNEL} > --with-initrd=${BUILDROOT}/images/rootfs.cpio > --with-cmdline="console=ttyAMA0" > --enable-psci --enable-gicv3 > $ make # produces linux-system.axf > > Run the model: > $ FVP_Base_RevC-2xAEMvA > -C bp.secure_memory=0 > -C 'pctl.startup=0.*.*.*' > -C bp.refcounter.non_arch_start_at_default=1 > -C cache_state_modelled=0 > -C bp.vis.disable_visualisation=1 > -C bp.virtio_net.enabled=1 > -C bp.virtio_net.hostbridge.userNetPorts=8022=22 > -C bp.virtio_net.hostbridge.userNetworking=1 > -C pci.pci_smmuv3.mmu.SMMU_IDR0=135100351 > -C pci.pci_smmuv3.mmu.SMMU_IDR3=4116 > -C pci.pci_smmuv3.mmu.SMMU_IDR5=8389749 > -C cluster0.NUM_CORES=4 > -C cluster1.NUM_CORES=4 > -a 'cluster*.cpu*=linux-system.axf' > > Then run a job using the tool. The process allocates two buffers and > passes their VA to the device (via the kernel driver). The device memcpies > one buffer to the other: > > $ smmute -u mmap > ... Success > > With smmu and iommu trace events enabled, a trace should contain > smmu_mm_invalidate and dev_fault/dev_page_response events. > > It's not entirely representative of SVA flow, where an assignable device > interface is mapped into the process and the process launches jobs > directly without going through the kernel (that would now use > drivers/misc/uacce), but it does exercise IOMMU SVA: sva_bind(), device > accessing the process address space with PASID and some IOPFs, which I > think is what you're looking for. However this model doesn't have a PCI > test device so you won't be able to test ATC invalidations with PASID. > > Other useful tests would be enabling lockdep (some intricate locking > between IOPF, the driver and mm), killing bound processes (-k), triggering > invalid accesses to verify TLB invalidations (-f tlb, I think). There is a > lot more to test, like thp and oom, but I don't have those in this branch. > > Thanks, > Jean > > [1] https://jpbrucker.net/git/linux/log/?h=sva/smmute-revc > [2] https://developer.arm.com/downloads/-/arm-ecosystem-models > [3] https://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git/
| |