[lkml]   [2021]   [Jun]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH v8 00/10] Introduce VDUSE - vDPA Device in Userspace

在 2021/6/28 下午1:54, Liu, Xiaodong 写道:
>> Several issues:
>> - VDUSE needs to limit the total size of the bounce buffers (64M if I was not
>> wrong). Does it work for SPDK?
> Yes, Jason. It is enough and works for SPDK.
> Since it's a kind of bounce buffer mainly for in-flight IO, so limited size like
> 64MB is enough.


>> - VDUSE can use hugepages but I'm not sure we can mandate hugepages (or we
>> need introduce new flags for supporting this)
> Same with your worry, I'm afraid too that it is a hard for a kernel module
> to directly preallocate hugepage internal.
> What I tried is that:
> 1. A simple agent daemon (represents for one device) `preallocates` and maps
> dozens of 2MB hugepages (like 64MB) for one device.
> 2. The daemon passes its mapping addr&len and hugepage fd to kernel
> module through created IOCTL.
> 3. Kernel module remaps the hugepages inside kernel.

Such model should work, but the main "issue" is that it introduce
overheads in the case of vhost-vDPA.

Note that in the case of vhost-vDPA, we don't use bounce buffer, the
userspace pages were shared directly.

And since DMA is not done per page, it prevents us from using tricks
like vm_insert_page() in those cases.

> 4. Vhost user target gets and maps hugepage fd from kernel module
> in vhost-user msg through Unix Domain Socket cmsg.
> Then kernel module and target map on the same hugepage based
> bounce buffer for in-flight IO.
> If there is one option in VDUSE to map userspace preallocated memory, then
> VDUSE should be able to mandate it even it is hugepage based.

As above, this requires some kind of re-design since VDUSE depends on
the model of mmap(MAP_SHARED) instead of umem registering.


 \ /
  Last update: 2021-06-29 06:19    [W:0.071 / U:0.168 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site