lkml.org 
[lkml]   [2018]   [Dec]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions
    On Wed, Dec 12, 2018 at 04:37:03PM -0700, Jason Gunthorpe wrote:
    > On Wed, Dec 12, 2018 at 04:53:49PM -0500, Jerome Glisse wrote:
    > > > Almost, we need some safety around assuming that DMA is complete the
    > > > page, so the notification would need to go all to way to userspace
    > > > with something like a file lease notification. It would also need to
    > > > be backstopped by an IOMMU in the case where the hardware does not /
    > > > can not stop in-flight DMA.
    > >
    > > You can always reprogram the hardware right away it will redirect
    > > any dma to the crappy page.
    >
    > That causes silent data corruption for RDMA users - we can't do that.
    >
    > The only way out for current hardware is to forcibly terminate the
    > RDMA activity somehow (and I'm not even sure this is possible, at
    > least it would be driver specific)
    >
    > Even the IOMMU idea probably doesn't work, I doubt all current
    > hardware can handle a PCI-E error TLP properly.

    What i saying is reprogram hardware to crappy page ie valid page
    dma map but that just has random content as a last resort to allow
    filesystem to reuse block. So their should be no PCIE error unless
    hardware freak out to see its page table reprogram randomly.

    >
    > On some hardware it probably just protects DAX by causing data
    > corruption for RDMA - I fail to see how that is a win for system
    > stability if the user obviously wants to use DAX and RDMA together...

    The question is who do you want to punish ? RDMA user that pin stuff
    and expect thing to work forever without worrying for other fs
    activities ? Or filesystem to pin block forever :) I am not gonna
    take side here but i don't think we can please both side, one will
    have to be mean to the user ie either the RDMA user or the file-
    system which also percolate to being mean to end user.

    > I think your approach with ODP only is the only one that meets your
    > requirements, the only other data-integrity-preserving approach is to
    > block/fail ftruncate/etc.

    >
    > > From my point of view driver should listen to ftruncate before the
    > > mmu notifier kicks in and send event to userspace and maybe wait
    > > and block ftruncate (or move it to a worker thread).
    >
    > We can do this, but we can't guarantee forward progress in userspace
    > and the best way we have to cancel that is portable to all RDMA
    > hardware is to kill the process(es)..
    >
    > So if that is acceptable then we could use user notifiers and allow
    > non-ODP users...

    Yes ODP with listening to _all_ mmu notifier event is the only
    sane way. But for hardware not capable of doing that (GPU are
    capable, so are mlx5, i won't do a list of the bad ones). We
    either keep the status quo that is today behavior or we do
    something either mean to the RDMA user or mean to the file-
    system. And previous discussion on failing ftruncate where a
    no no, can't remember why. In any case i am personnaly fine with
    what ever which is:
    S1: keep block pin until RDMA goes away, even if it means
    that RDMA user is no longer really accessing anything
    that make sense (ie the page is no longer part of the
    file or the original vma so as this point it fully
    disconnected from the original intent ie today status
    quo we pin block and annoy filesystem while we pretend
    that everything is fine.
    S2: notify userspace program through device/sub-system
    specific API and delay ftruncate. After a while if there
    is no answer just be mean and force hardware to use
    crappy page as anyway this is what happens today (note
    we can fully mirror today behavior by allocating pages
    and copying existing content their and then swaping
    out to point the hardware to those pages.
    S3: be mean to filesystem a keep block pin for as long as
    they are active GUP, this means failing ftruncate and
    or possibly munmap().

    S3 can be split in sub-choices. Do we want to take vote ? Or
    is there a way that can please everyone ?

    Cheers,
    Jérôme

    \
     
     \ /
      Last update: 2018-12-13 01:18    [W:3.722 / U:0.016 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site