Messages in this thread | | | Subject | Re: virtio-blk: should num_vqs be limited by num_possible_cpus()? | From | Jason Wang <> | Date | Wed, 20 Mar 2019 20:53:33 +0800 |
| |
On 2019/3/19 上午10:22, Dongli Zhang wrote: > Hi Jason, > > On 3/18/19 3:47 PM, Jason Wang wrote: >> On 2019/3/15 下午8:41, Cornelia Huck wrote: >>> On Fri, 15 Mar 2019 12:50:11 +0800 >>> Jason Wang <jasowang@redhat.com> wrote: >>> >>>> Or something like I proposed several years ago? >>>> https://do-db2.lkml.org/lkml/2014/12/25/169 >>>> >>>> Btw, for virtio-net, I think we actually want to go for having a maximum >>>> number of supported queues like what hardware did. This would be useful >>>> for e.g cpu hotplug or XDP (requires per cpu TX queue). But the current >>>> vector allocation doesn't support this which will results all virtqueues >>>> to share a single vector. We may indeed need more flexible policy here. >>> I think it should be possible for the driver to give the transport >>> hints how to set up their queues/interrupt structures. (The driver >>> probably knows best about its requirements.) Perhaps whether a queue is >>> high or low frequency, or whether it should be low latency, or even >>> whether two queues could share a notification mechanism without >>> drawbacks. It's up to the transport to make use of that information, if >>> possible. >> >> Exactly and it was what the above series tried to do by providing hints of e.g >> which queues want to share a notification. >> > I read about your patch set on providing more flexibility of queue-to-vector > mapping. > > One use case of the patch set is we would be able to enable more queues when > there is limited number of vectors. > > Another use case we may classify queues as hight priority or low priority as > mentioned by Cornelia. > > For virtio-blk, we may extend virtio-blk based on this patch set to enable > something similar to write_queues/poll_queues in nvme, when (set->nr_maps != 1). > > > Yet, the question I am asking in this email thread is for a difference scenario. > > The issue is not we are not having enough vectors (although this is why only 1 > vector is allocated for all virtio-blk queues). As so far virtio-blk has > (set->nr_maps == 1), block layer would limit the number of hw queues by > nr_cpu_ids, we indeed do not need more than nr_cpu_ids hw queues in virtio-blk. > > That's why I ask why not change the flow as below options when the number of > supported hw queues is more than nr_cpu_ids (and set->nr_maps == 1. virtio-blk > does not set nr_maps and block layer would set it to 1 when the driver does not > specify with a value): > > option 1: > As what nvme and xen-netfront do, limit the hw queue number by nr_cpu_ids.
How do they limit the hw queue number? A command?
> > option 2: > If the vectors is not enough, use the max number vector (indeed nr_cpu_ids) as > number of hw queues.
We can share vectors in this case.
> > option 3: > We should allow more vectors even the block layer would support at most > nr_cpu_ids queues. > > > I understand a new policy for queue-vector mapping is very helpful. I am just > asking the question from block layer's point of view. > > Thank you very much! > > Dongli Zhang
Don't know much for block, cc Stefan for more idea.
Thanks
| |