Messages in this thread | | | Date | Wed, 29 Mar 2023 18:52:06 +0800 | From | Ming Lei <> | Subject | Re: [PATCH V5 16/16] block: ublk_drv: apply io_uring FUSED_CMD for supporting zero copy |
| |
On Wed, Mar 29, 2023 at 06:01:16PM +0800, Ziyang Zhang wrote: > On 2023/3/29 17:00, Ming Lei wrote: > > On Wed, Mar 29, 2023 at 10:57:53AM +0800, Ziyang Zhang wrote: > >> On 2023/3/28 23:09, Ming Lei wrote: > >>> Apply io_uring fused command for supporting zero copy: > >>> > >> > >> [...] > >> > >>> > >>> @@ -1374,7 +1533,12 @@ static int ublk_ch_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags) > >>> if (!ubq || ub_cmd->q_id != ubq->q_id) > >>> goto out; > >>> > >>> - if (ubq->ubq_daemon && ubq->ubq_daemon != current) > >>> + /* > >>> + * The fused command reads the io buffer data structure only, so it > >>> + * is fine to be issued from other context. > >>> + */ > >>> + if ((ubq->ubq_daemon && ubq->ubq_daemon != current) && > >>> + (cmd_op != UBLK_IO_FUSED_SUBMIT_IO)) > >>> goto out; > >>> > >> > >> Hi Ming, > >> > >> What is your use case that fused io_uring cmd is issued from another thread? > >> I think it is good practice to operate one io_uring instance in one thread > >> only. > > > > So far we limit io command has to be issued from the queue context, > > which is still not friendly from userspace viewpoint, the reason is > > that we can't get io_uring exit notification and ublk's use case is > > very special since the queued io command may not be completed forever, > > OK, so UBLK_IO_FUSED_SUBMIT_IO is guaranteed to be completed because it is > not queued. FETCH_REQ and COMMIT_AMD_FETCH are queued io commands and could > not be completed forever so they have to be issued from ubq_daemon. Right?
Yeah, any io command should be issued from ubq daemon context.
> > BTW, maybe NEED_GET_DATA can be issued from other context...
So far it won't be supported.
As I mentioned in the link, if io_uring can provide io_uring exit callback, we may relax this limit.
> > > see: > > > > https://lore.kernel.org/linux-fsdevel/ZBxTdCj60+s1aZqA@ovpn-8-16.pek2.redhat.com/ > > > > I remember that people raised concern about this implementation. > > > > But for normal IO, it could be issued from io wq simply because of > > link(dependency) or whatever, and userspace is still allowed to submit > > io from another pthread via same io_uring ctx. > > Yes, we can submit to the same ctx from different pthread but lock may be required.
Right.
> IMO, users may only choose ubq_daemon as the only submitter.
At least any io command should be issued from ubq daemon now, but normal io can be issued from any context.
Thanks, Ming
| |