Messages in this thread | | | Date | Tue, 1 Mar 2022 16:39:38 -0400 | From | Jason Gunthorpe <> | Subject | Re: [PATCH v6 09/10] hisi_acc_vfio_pci: Add support for VFIO live migration |
| |
On Tue, Mar 01, 2022 at 12:30:47PM -0700, Alex Williamson wrote: > Wouldn't it make more sense if initial-bytes started at QM_MATCH_SIZE > and dirty-bytes was always sizeof(vf_data) - QM_MATCH_SIZE? ie. QEMU > would know that it has sizeof(vf_data) - QM_MATCH_SIZE remaining even > while it's getting ENOMSG after reading QM_MATCH_SIZE bytes of data.
The purpose of this ioctl is to help userspace guess when moving on to STOP_COPY is a good idea ie when the device has done almost all the work it is going to be able to do in PRE_COPY. ENOMSG is a similar indicator.
I expect all devices to have some additional STOP_COPY trailer_data in addition to their PRE_COPY initial_data and dirty_data
There is a choice to make if we report the trailer_data during PRE_COPY or not. As this is all estimates, it doesn't matter unless the trailer_data is very big.
Having all devices trend toward a 0 dirty_bytes to say they are are done all the pre-copy they can do makes sense from an API perspective. If one device trends toward 10MB due to a big trailer_data and one trends toward 0 bytes, how will qemu consistently decide when best to trigger STOP_COPY? It makes the API less useful.
So, I would not include trailer_data in the dirty_bytes.
Estimating when to move on to STOP_COPY and trying to enforce a SLA on STOP_COPY are different tasks and will probably end up with different interfaces.
I still think the right way to approach the SLA is to inform the driver what the permitted time and data size target is for STOP_COPY and the driver can proceed or not based on its own internal calculation.
> useful yet and you don't want to add dead kernel code, then let's > define that this ioctl is only available in the PRE_COPY* states and > returns -errno in the STOP_COPY state.
I'm OK with that, in acc it is done by checking migf->total_bytes > QM_MATCH_SIZE during the read fop
> devices in STOP_COPY and let's also define if there's actually anything > userspace can infer about remaining STOP_COPY data size while in > PRE_COPY* via this ioctl. For example, is dirty-bytes zero or the > remaining data structure size?
If we keep it then I would say it doesn't matter, userspace has to sum the two values to get the total remaining length estimate, it is just a bit quirky.
Jason
| |