lkml.org 
[lkml]   [2008]   [Feb]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [ofa-general] Re: Demand paging for memory regions
    On Wed, Feb 13, 2008 at 06:23:08PM -0500, Pete Wyckoff wrote:
    > christian.bell@qlogic.com wrote on Tue, 12 Feb 2008 20:09 -0800:
    > > One other area that has not been brought up yet (I think) is the
    > > applicability of notifiers in letting users know when pinned memory
    > > is reclaimed by the kernel. This is useful when a lower-level
    > > library employs lazy deregistration strategies on memory regions that
    > > are subsequently released to the kernel via the application's use of
    > > munmap or sbrk. Ohio Supercomputing Center has work in this area but
    > > a generalized approach in the kernel would certainly be welcome.
    >
    > The whole need for memory registration is a giant pain. There is no
    > motivating application need for it---it is simply a hack around
    > virtual memory and the lack of full VM support in current hardware.
    > There are real hardware issues that interact poorly with virtual
    > memory, as discussed previously in this thread.

    Well, the registrations also exist to provide protection against
    rouge/faulty remotes, but for the purposes of MPI that is probably not
    important.

    Here is a thought.. Some RDMA hardware can change the page tables on
    the fly. What if the kernel had a mechanism to dynamically maintain a
    full registration of the processes entire address space ('mlocked' but
    able to be migrated)? MPI would never need to register a buffer, and
    all the messy cases with munmap/sbrk/etc go away - the risk is that
    other MPI nodes can randomly scribble all over the process :)

    Christoph: It seemed to me you were first talking about
    freeing/swapping/faulting RDMA'able pages - but would pure migration
    as a special hardware supported case be useful like Catilan suggested?

    Regards,
    Jason


    \
     
     \ /
      Last update: 2008-02-14 01:03    [W:0.023 / U:121.968 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site