Messages in this thread | | | From | Jason Gunthorpe <> | Subject | Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma | Date | Wed, 30 Jan 2019 17:44:31 +0000 |
| |
On Wed, Jan 30, 2019 at 09:02:08AM +0100, Christoph Hellwig wrote: > On Tue, Jan 29, 2019 at 08:58:35PM +0000, Jason Gunthorpe wrote: > > On Tue, Jan 29, 2019 at 01:39:49PM -0700, Logan Gunthorpe wrote: > > > > > implement the mapping. And I don't think we should have 'special' vma's > > > for this (though we may need something to ensure we don't get mapping > > > requests mixed with different types of pages...). > > > > I think Jerome explained the point here is to have a 'special vma' > > rather than a 'special struct page' as, really, we don't need a > > struct page at all to make this work. > > > > If I recall your earlier attempts at adding struct page for BAR > > memory, it ran aground on issues related to O_DIRECT/sgls, etc, etc. > > Struct page is what makes O_DIRECT work, using sgls or biovecs, etc on > it work. Without struct page none of the above can work at all. That > is why we use struct page for backing BARs in the existing P2P code. > Not that I'm a particular fan of creating struct page for this device > memory, but without major invasive surgery to large parts of the kernel > it is the only way to make it work.
I don't think anyone is interested in O_DIRECT/etc for RDMA doorbell pages.
.. and again, I recall Logan already attempted to mix non-CPU memory into sgls and it was a disaster. You pointed out that one cannot just put iomem addressed into a SGL without auditing basically the entire block stack to prove that nothing uses iomem without an iomem accessor.
I recall that proposal veered into a direction where the block layer would just fail very early if there was iomem in the sgl, so generally no O_DIRECT support anyhow.
We already accepted the P2P stuff from Logan as essentially a giant special case - it only works with RDMA and only because RDMA MR was hacked up with a special p2p callback.
I don't see why a special case with a VMA is really that different.
If someone figures out the struct page path down the road it can obviously be harmonized with this VMA approach pretty easily.
Jason
| |