Messages in this thread | | | Date | Fri, 17 Mar 2000 08:27:09 -0800 | From | Brian Pomerantz <> | Subject | Re: DMA to user space |
| |
On Thu, Mar 16, 2000 at 12:12:11PM -0500, Richard B. Johnson wrote: > On Thu, 16 Mar 2000, John Wilson wrote: > > > On Thu, Mar 16, 2000 at 08:22:38AM -0500, Richard B. Johnson wrote: > > > No. Not a "user-buffer". It may be possible to memory-map something that's > > > DMA..able to user space, but even that's not guaranteed. To have something > > > DMA..able in the ix86 machines, it needs to be: > > > > > > (1) Physical, below 16 Megabytes > > > (2) DATA alignment must be such that a 64k block is not crossed > > > during a DMA transfer. The ix86 machines use a page register > > > to access 64k pages. > > > (3) You need access to the DMA controller which the kernel isn't > > > going to give you, without a fight iopl(3), etc., from user- > > > space. > > > > No no, these requirements are for ISA DMA, not DMA in general. I'm using > > a PCI device, so there are no address restrictions (on x86 anyway, I dunno > > how PCI works on CPUs with >32 addr bits), and it has its own DMA paging > > registers so it doesn't require a contiguous buffer either. The buffer > > just has to be locked in place, which sounds like mlock() would be my friend. > > > > Also, my driver is a real driver, it doesn't run in user space. I just want > > it to be able to set the device up so that the DMA buffer (*only*) is in > > user space, and let 'er rip. > > > > If your device is doing it's own DMA, basically you are telling it to > write its stuff to some physical address that you give it. If this > is correct, then your driver needs to lock user pages in place if > you are not going to use kernel memory. >
We are dealing with the same sort of issue at LLNL for a high speed interconnect card that has its own MMU. We wrote a generic co-processor interface into the kernel to handle the issues involved. What happens is the card's MMU has tables on it to map pages of memory on the host cpu to a contiguous block of memory that it can use to DMA. In essence, it duplicates the memory tables for a particular process in the card's memory. When a page in that process is swapped out or is no longer valid, the hooks we added allows a device driver to register a callback that gets called when this happens. It seems to work really well. This makes it so no pages need to be locked down, which makes GANG scheduling of large jobs MUCH easier to deal with.
We are in the process of getting permission to release all of the kernel modifications we've been making. As you can imagine, sometimes it is hard to get things out of a DOE controlled lab. :)
BAPper
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |