Messages in this thread | | | Date | Mon, 28 Nov 2016 13:31:55 +0800 | From | zhoucm1 <> | Subject | Re: Enabling peer to peer device transactions for PCIe devices |
| |
+Qiang, who is working on it.
On 2016年11月27日 22:07, Christian König wrote: > Am 27.11.2016 um 15:02 schrieb Haggai Eran: >> On 11/25/2016 9:32 PM, Jason Gunthorpe wrote: >>> On Fri, Nov 25, 2016 at 02:22:17PM +0100, Christian König wrote: >>> >>>>> Like you say below we have to handle short lived in the usual way, >>>>> and >>>>> that covers basically every device except IB MRs, including the >>>>> command queue on a NVMe drive. >>>> Well a problem which wasn't mentioned so far is that while GPUs do >>>> have a >>>> page table to mirror the CPU page table, they usually can't recover >>>> from >>>> page faults. >>>> So what we do is making sure that all memory accessed by the GPU >>>> Jobs stays >>>> in place while those jobs run (pretty much the same pinning you do >>>> for the >>>> DMA). >>> Yes, it is DMA, so this is a valid approach. >>> >>> But, you don't need page faults from the GPU to do proper coherent >>> page table mirroring. Basically when the driver submits the work to >>> the GPU it 'faults' the pages into the CPU and mirror translation >>> table (instead of pinning). >>> >>> Like in ODP, MMU notifiers/HMM are used to monitor for translation >>> changes. If a change comes in the GPU driver checks if an executing >>> command is touching those pages and blocks the MMU notifier until the >>> command flushes, then unfaults the page (blocking future commands) and >>> unblocks the mmu notifier. >> I think blocking mmu notifiers against something that is basically >> controlled by user-space can be problematic. This can block things like >> memory reclaim. If you have user-space access to the device's queues, >> user-space can block the mmu notifier forever. > Really good point. > > I think this means the bare minimum if we don't have recoverable page > faults is to have preemption support like Felix described in his > answer as well. > > Going to keep that in mind, > Christian. > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel
| |