lkml.org 
[lkml]   [2005]   [Apr]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation
    Andy Isaacson wrote:

    > I'm familiar with MPI 1.0 and 2.0, but I haven't been following the
    > development of modern messaging APIs, so I might not make sense here...
    >
    > Assuming that the app calls into the library on a fairly regular basis,

    Not really. The whole point is to have the adapter DMA the data directly from memory to
    the network. That's why it's called RDMA - remote DMA.

    > Therefore, cluster admins are going to do their
    > darndest to avoid this behavior, so we might as well just kill the job
    > and make it explicit.

    Yes, and if it turns out that the same MPI application dies on Linux but not on Solaris
    because Linux doesn't really care if the memory stays pinned, then we're going to see a
    lot of MPI customers transitioning away from Linux.

    > *You* need to come up with a solution that looks good to *the community*
    > if you want it merged.

    True, but I'm not going to waste my time adding this support if the consensus I get from
    the kernel developers that they don't want Linux to behave this way.

    > Do you guys simply raise RLIMIT_MEMLOCK to allow apps to lock their
    > pages? Or are you doing something more nasty?

    A little more nasty. I raise RLIMIT_MEMLOCK in the driver to "unlimited" and also set
    cap_raise(IPC_LOCK). I do this because I need to support all 2.4 and 2.6 kernel versions
    with the same driver, but only 2.6.10 and later have any support for non-root mlock().

    If and when our driver is submitted to the official kernel, that nastiness will be removed
    of course.

    --
    Timur Tabi
    Staff Software Engineer
    timur.tabi@ammasso.com

    One thing a Southern boy will never say is,
    "I don't think duct tape will fix it."
    -- Ed Smylie, NASA engineer for Apollo 13
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-04-21 22:12    [W:4.129 / U:0.652 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site