Messages in this thread | | | Subject | Re: IB: Add DMA mapping functions to allow device drivers to interpose | From | Ralph Campbell <> | Date | Wed, 13 Dec 2006 13:22:40 -0800 |
| |
On Wed, 2006-12-13 at 12:30 -0800, Andrew Morton wrote: > On Wed, 13 Dec 2006 12:11:27 -0800 > Ralph Campbell <ralph.campbell@qlogic.com> wrote: snip. > > My preference would be to change the offending uses of dma_addr_t > > to u64. Do you have a better solution? > > We should be able to use dma_addr_t for this? Is it not the case that the > values we're dealing with here _are_ DMA addresses? I think a more complete > description of the problem we're trying to solve here would help. > > I'm not sure what the problem is with sparc64 - perhaps its dma_addr_t > really is a "cookie" and isn't a physical bus address? But you want a value > which is really a physical bus address? Dunno. > > Perhaps dma64_addr_t can be used here.
The fundamental problem is that ib_get_dma_mr() returns a pointer representing a memory region for all of physical memory and the IB interface consumer is expected to call dma_map_single() to get a dma_addr_t to pass to the HCA driver with the memory region pointer.
For an HCA like Mellanox, the dma_addr_t really is a DMA address which the hardware uses to DMA data from the chip to memory.
The ib_ipath HCA driver does not generally use DMA addresses. The hardware does DMA the receive packet to a ring of buffers but the receive interrupt handler then has to copy the data from the receive buffer to the address specified by the ib_post_recv() function. This means the driver needs a kernel virtual address instead of the memory region + DMA address that is passed by ib_post_recv().
The ib_dma_*() functions were added to allow the ib_ipath driver to interpose on the dma_*() functions so that a kernel virtual address can be returned instead of allocating an IOMMU DMA address. Without the interposing functions, there is no way for the ib_ipath driver to convert the IOMMU address back into a kernel virtual address since it never sees the original inputs used to allocate the IOMMU address.
I don't want to make ib_dma_map_single() return a dma_addr_t cookie and use it to lookup the kernel virtual address since this is a performance critical code path in the receive interrupt handler and the "cookie" needs to be a byte address which can be offset within a page.
The secondary problem is that on sparc64, dma_addr_t is a u32 so if ib_dma_map_single() returns a void * cast to dma_addr_t, it is truncated. We might be able to convince the sparc64 maintainers to make dma_addr_t u64 but dma64_addr_t won't work because that is specific to sparc64. The type has to be architecture neutral.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |