lkml.org 
[lkml]   [2017]   [Jan]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Enabling peer to peer device transactions for PCIe devices
    On Thu, Jan 05, 2017 at 03:19:36PM -0500, Jerome Glisse wrote:

    > > Always having a VMA changes the discussion - the question is how to
    > > create a VMA that reprensents IO device memory, and how do DMA
    > > consumers extract the correct information from that VMA to pass to the
    > > kernel DMA API so it can setup peer-peer DMA.
    >
    > Well my point is that it can't be. In HMM case inside a single VMA
    > you
    [..]

    > In the GPUDirect case the idea is that you have a specific device vma
    > that you map for peer to peer.

    [..]

    I still don't understand what you driving at - you've said in both
    cases a user VMA exists.

    From my perspective in RDMA, all I want is a core kernel flow to
    convert a '__user *' into a scatter list of DMA addresses, that works no
    matter what is backing that VMA, be it HMM, a 'hidden' GPU object, or
    struct page memory.

    A '__user *' pointer is the only way to setup a RDMA MR, and I see no
    reason to have another API at this time.

    The details of how to translate to a scatter list are a MM subject,
    and the MM folks need to get

    I just don't care if that routine works at a page level, or a whole
    VMA level, or some combination of both, that is up to the MM team to
    figure out :)

    > a page level. Expectation here is that the GPU userspace expose a special
    > API to allow RDMA to directly happen on GPU object allocated through
    > GPU specific API (ie it is not regular memory and it is not accessible
    > by CPU).

    So, how do you identify these GPU objects? How do you expect RDMA
    convert them to scatter lists? How will ODP work?

    > > We have MMU notifiers to handle this today in RDMA. Async RDMA MR
    > > Invalidate like you see in the above out of tree patches is totally
    > > crazy and shouldn't be in mainline. Use ODP capable RDMA hardware.
    >
    > Well there is still a large base of hardware that do not have such
    > feature and some people would like to be able to keep using those.

    Hopefully someone will figure out how to do that without the crazy
    async MR invalidation.

    Jason

    \
     
     \ /
      Last update: 2017-01-06 00:23    [W:4.426 / U:0.164 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site