lkml.org 
[lkml]   [2023]   [Oct]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: [RFC] vdpa/mlx5: preserve CVQ vringh index
    From
    On 10/27/2023 2:31 AM, Jason Wang wrote:
    > On Fri, Oct 27, 2023 at 4:07 AM Steve Sistare <steven.sistare@oracle.com> wrote:
    >>
    >> mlx5_vdpa does not preserve userland's view of vring base for the control
    >> queue in the following sequence:
    >>
    >> ioctl VHOST_SET_VRING_BASE
    >> ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
    >> mlx5_vdpa_set_status()
    >> setup_cvq_vring()
    >> vringh_init_iotlb()
    >> vringh_init_kern()
    >> vrh->last_avail_idx = 0;
    >> ioctl VHOST_GET_VRING_BASE
    >>
    >> To fix, restore the value of cvq->vring.last_avail_idx after calling
    >> vringh_init_iotlb.
    >>
    >> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
    >> ---
    >> drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 ++++++-
    >> drivers/vhost/vringh.c | 30 ++++++++++++++++++++++++++++++
    >> include/linux/vringh.h | 2 ++
    >> 3 files changed, 38 insertions(+), 1 deletion(-)
    >>
    >> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
    >> index 946488b8989f..f64758143115 100644
    >> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
    >> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
    >> @@ -2795,13 +2795,18 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
    >> struct mlx5_control_vq *cvq = &mvdev->cvq;
    >> int err = 0;
    >>
    >> - if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ))
    >> + if (mvdev->actual_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) {
    >> + u16 last_avail_idx = cvq->vring.last_avail_idx;
    >> +
    >> err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
    >> MLX5_CVQ_MAX_ENT, false,
    >> (struct vring_desc *)(uintptr_t)cvq->desc_addr,
    >> (struct vring_avail *)(uintptr_t)cvq->driver_addr,
    >> (struct vring_used *)(uintptr_t)cvq->device_addr);
    >>
    >> + if (!err)
    >> + vringh_set_base_iotlb(&cvq->vring, last_avail_idx);
    >
    > Btw, vringh_set_base_iotlb() deserves an independent patch and it
    > seems it is not specific to IOTLB,

    Agreed on both. I initially submitted the smallest change needed to show the problem.

    so we probably need an indirection
    > to have vringh_set_base() first.

    Not sure what you mean. I have defined:

    static inline int __vringh_set_base() ...

    int vringh_set_base_iotlb()
    return __vringh_set_base(vrh, idx, putu16_iotlb);

    to which I would add:

    int vringh_set_base_user()
    return __vringh_set_base(vrh, idx, putu16_user);

    int vringh_set_base_kern()
    return __vringh_set_base(vrh, idx, putu16_kern;

    all in the same patch.

    The call site in mlx5_vnet.c would be a 2nd patch.

    - Steve

    > Or I wonder if it's better to just introduce a new parameter to
    > vringh_init_iotlb()...

    I considered that, but IMO the parameter list there is already large, and it would
    be strange to add a parameter for the initial value of avail, but not for used, and
    no one needs the latter.

    - Steve

    >> + }
    >> return err;
    >> }
    >>
    >> diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c
    >> index 7b8fd977f71c..799762c83007 100644
    >> --- a/drivers/vhost/vringh.c
    >> +++ b/drivers/vhost/vringh.c
    >> @@ -595,6 +595,24 @@ static inline void __vringh_notify_disable(struct vringh *vrh,
    >> }
    >> }
    >>
    >> +static inline int __vringh_set_base(struct vringh *vrh, u16 idx,
    >> + int (*putu16)(const struct vringh *vrh,
    >> + __virtio16 *p, u16 val))
    >> +{
    >> + int ret;
    >> +
    >> + ret = putu16(vrh, &vrh->vring.avail->idx, idx);
    >> + if (ret)
    >> + return ret;
    >> +
    >> + ret = putu16(vrh, &vrh->vring.used->idx, idx);
    >> + if (ret)
    >> + return ret;
    >> +
    >> + vrh->last_avail_idx = vrh->last_used_idx = idx;
    >> + return 0;
    >> +}
    >> +
    >> /* Userspace access helpers: in this case, addresses are really userspace. */
    >> static inline int getu16_user(const struct vringh *vrh, u16 *val, const __virtio16 *p)
    >> {
    >> @@ -1456,6 +1474,18 @@ void vringh_set_iotlb(struct vringh *vrh, struct vhost_iotlb *iotlb,
    >> }
    >> EXPORT_SYMBOL(vringh_set_iotlb);
    >>
    >> +/**
    >> + * vringh_set_base_iotlb - set avail_idx and used_idx
    >> + * @vrh: the vring
    >> + * @idx: the value to set
    >> + */
    >> +int vringh_set_base_iotlb(struct vringh *vrh, u16 idx)
    >> +{
    >> + return __vringh_set_base(vrh, idx, putu16_iotlb);
    >> +}
    >> +EXPORT_SYMBOL(vringh_set_base_iotlb);
    >> +
    >> +
    >> /**
    >> * vringh_getdesc_iotlb - get next available descriptor from ring with
    >> * IOTLB.
    >> diff --git a/include/linux/vringh.h b/include/linux/vringh.h
    >> index c3a8117dabe8..e9b8af4e6a5e 100644
    >> --- a/include/linux/vringh.h
    >> +++ b/include/linux/vringh.h
    >> @@ -306,6 +306,8 @@ int vringh_init_iotlb_va(struct vringh *vrh, u64 features,
    >> struct vring_avail *avail,
    >> struct vring_used *used);
    >>
    >> +int vringh_set_base_iotlb(struct vringh *vrh, u16 idx);
    >> +
    >> int vringh_getdesc_iotlb(struct vringh *vrh,
    >> struct vringh_kiov *riov,
    >> struct vringh_kiov *wiov,
    >> --
    >> 2.39.3
    >>
    >

    \
     
     \ /
      Last update: 2023-10-30 15:06    [W:7.214 / U:0.216 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site