lkml.org 
[lkml]   [2022]   [Sep]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: [for-next PATCH v5 03/11] RDMA: Extend RDMA kernel verbs ABI to support flush
    From
    Leon, Jason


    On 27/09/2022 13:53, Li Zhijian wrote:
    > /*
    > @@ -4321,6 +4330,8 @@ int ib_dealloc_xrcd_user(struct ib_xrcd *xrcd, struct ib_udata *udata);
    > static inline int ib_check_mr_access(struct ib_device *ib_dev,
    > unsigned int flags)
    > {
    > + u64 device_cap = ib_dev->attrs.device_cap_flags;
    > +
    > /*
    > * Local write permission is required if remote write or
    > * remote atomic permission is also requested.
    > @@ -4335,6 +4346,13 @@ static inline int ib_check_mr_access(struct ib_device *ib_dev,
    > if (flags & IB_ACCESS_ON_DEMAND &&
    > !(ib_dev->attrs.kernel_cap_flags & IBK_ON_DEMAND_PAGING))
    > return -EINVAL;
    > +
    > + if ((flags & IB_ACCESS_FLUSH_GLOBAL &&
    > + !(device_cap & IB_DEVICE_FLUSH_GLOBAL)) ||
    > + (flags & IB_ACCESS_FLUSH_PERSISTENT &&
    > + !(device_cap & IB_DEVICE_FLUSH_PERSISTENT)))
    > + return -EINVAL;
    > +
    Regarding of the return value of ib_check_mr_access. While updating the man page of ibv_reg_mr(3) of rdma-core,
    ```
           IBV_ACCESS_REMOTE_READ Enable Remote Read Access
           IBV_ACCESS_REMOTE_ATOMIC Enable Remote Atomic Operation Access (if supported)
           IBV_ACCESS_MW_BIND Enable Memory Window Binding
           IBV_ACCESS_ZERO_BASED  Use  byte offset from beginning of MR to access this MR, instead of a pointer address
           IBV_ACCESS_ON_DEMAND Create an on-demand paging MR (if supported)
    ...
    RETURN VALUE
           ibv_reg_mr() / ibv_reg_mr_iova() / ibv_reg_dmabuf_mr() returns a pointer to the registered MR, or NULL if the request fails.  The local key (L_Key) field lkey is used as the lkey field of struct  ibv_sge  when  posting
           buffers  with  ibv_post_* verbs, and the the remote key (R_Key) field rkey is used by remote processes to perform Atomic and RDMA operations.  The remote process places this rkey as the rkey field of struct ibv_send_wr
           passed to the ibv_post_send function.
    ```
    we can see, IBV_ACCESS_REMOTE_ATOMIC and IBV_ACCESS_ON_DEMAND are tagged "if supported" . but currently kernel
    just returns EINVAL when user registers a MR with IB_ACCESS_ON_DEMAND to RXE.

    I wonder we should return -EOPNOTSUPP if the device doesn't support requested capabilities

    Thanks
    Li


    > return 0;
    > }
    >

    \
     
     \ /
      Last update: 2022-09-29 08:23    [W:4.152 / U:0.780 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site