[lkml]   [2006]   [Mar]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [openib-general] Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
Matthew Frost wrote:
> To the point. I, insightful betimes, but a non-user of the technology,
> can grep TFM's and find out what the names could mean, but we're left
> guessing at what some of these *do*. Translating names falls into the
> "any idiot can" category of data mining, but if you code for them, we can
> see context. If you named them more transparently, we might even use
> them right. Maybe just comment well?

Documentation in the form of comments are provided in the header files. I can
clarify the calls if needed. To help understand the structure of this patch,
three modules were submitted:

ib_addr - maps IP addresses to an RDMA device.
rdma_cm - Adds connection management over Infiniband using IP addressing. This
exports most of the symbols below.
rdma_ucm - Exports the rdma_cm functionality to userspace. This uses most of
the exported symbols (starting at rdma_create_id and below).

> +EXPORT_SYMBOL(rdma_wq); Work Queue (do what to it?)

This is used by ib_addr and rdma_cm modules to invoke user callbacks.
Additional code not yet ready for merging will also make use of this work queue.
The intent is to re-use this work queue, rather than each module creating
their own.

> +EXPORT_SYMBOL(rdma_translate_ip); Translate IP Address
> +EXPORT_SYMBOL(rdma_resolve_ip); Resolve IP Address
> +EXPORT_SYMBOL(rdma_addr_cancel); Address Cancel (memory?)

These exports are from the ib_addr module. The routines are called by the
rdma_cm module. The first two map an IP address to a local Infiniband device
address. Rdma_resolve_ip is an asynchronous call, so rdma_addr_cancel is used
to cancel its operation. 'rdma_cancel_resolve_ip' might have been a clearer name.

> +EXPORT_SYMBOL(rdma_create_id); Create (?) ID

The rdma_cm_id created by this call is required for the calls below.
Conceptually, it may help to think of an rdma_cm_id as somewhat like a socket.

> +EXPORT_SYMBOL(rdma_create_qp); Create Queue Pair (WQ,CQ)
> +EXPORT_SYMBOL(rdma_destroy_qp); Destroy Queue Pair (WQ,CQ)

'rdma_create_qp' associates a QP with an rdma_cm_id, so that the rdma_cm can
perform the QP transitions for the user during connection establishment.

> +EXPORT_SYMBOL(rdma_init_qp_attr); Set Initial Queue Pair Attributes (?)

This initializes the QP attributes for a user that wants to manually perform QP
transitions. It is provided mainly for userspace support.

> +EXPORT_SYMBOL(rdma_destroy_id); Destroy (?) ID
> +EXPORT_SYMBOL(rdma_listen); Listen (to ... socket, port, pipe, what?)

Listens across RDMA devices for connection requests. The listen is on an

> +EXPORT_SYMBOL(rdma_resolve_route); Resolve Route (datagram path?)

In Infiniband terms, this obtains a path record from the subnet manager. The
path record specifies the route through the subnet that packets between two
connected queue pairs will take.

> +EXPORT_SYMBOL(rdma_resolve_addr); Resolve Address (memory?)

This converts struct sockaddr to RDMA addresses. It ends up calling
rdma_translate_ip and rdma_resolve_ip, but performs some additional work to
handle device hotplug events.

> +EXPORT_SYMBOL(rdma_bind_addr); Bind Address (memory?)

Associates an rdma_cm_id to a specific struct sockaddr.

> +EXPORT_SYMBOL(rdma_connect); Connect
> +EXPORT_SYMBOL(rdma_accept); Accept
> +EXPORT_SYMBOL(rdma_reject); Reject
> +EXPORT_SYMBOL(rdma_disconnect); Disconnect
> Address vs. IP - I know we're talking about a net/dma kluge here, but the
> twin usage is bugging me. I'm intuiting the _addr as memory addresses,
> rather than IP addresses, which seem to be _ip, but my poor gray goo
> suffers pointer overload.

Maybe the naming is off here. I used _ip when referring specifically to an IP
address, and _addr when using a struct sockaddr. In some cases, such as
rdma_bind_addr, the 'address' may be nothing more than a port number.

> +EXPORT_SYMBOL(ib_get_rmpp_segment); Reliable MultiPacket Protocol

This is from a separate patch. It is exported by the ib_mad module, and used by
the ib_umad module. There is at least one out of tree module (not yet ready for
merging) that will make use of it.

> +EXPORT_SYMBOL(ib_copy_qp_attr_to_user); Push Queue Pair Attribute
> +EXPORT_SYMBOL(ib_copy_path_rec_to_user); Push Path Record
> +EXPORT_SYMBOL(ib_copy_path_rec_from_user); Retrieve Path Record

These are used by ib_uverbs, ib_ucm, and rdma_ucm modules.

> +EXPORT_SYMBOL(ib_modify_qp_is_ok); Yes, Modify Queue Pair, or "QP is
> OK", or "QP was Modified OK"?

This is from a separate patch. It should be exported by ib_verbs, and used by
ib_mthca. The call verifies that the settings to modify a QP from one state to
the next are valid. I believe that the check used to be part of the ib_mthca
driver itself, but additional kernel drivers, such as the ipath driver recently
submitted for inclusion, now make use of this routine.

> +EXPORT_SYMBOL(ip_dev_find); Find IP device (sub(/ip/, "ib")? find the
> network interface device?)

This is the network call in fib_frontend.c. It is being re-exported (the export
was removed a couple of versions ago) for use by ib_addr.

>>Please explain the thinking behind the choice of a non-GPL export.
>>(Yes, we discussed this when inifiniband was first merged, but it
>>doesn't hurt to reiterate).

The agreement made within the OpenIB community, from where this code originates,
is that all source code be licensed under a dual license of BSD/GPL. I am not a
lawyer, so I don't know the implications of changing the exports to be GPL only,
given the OpenIB license. But my understanding is that makes using those
functions less attractive.

- Sean
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2006-03-20 19:37    [W:0.086 / U:5.648 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site