Messages in this thread | | | Date | Tue, 07 Apr 2015 11:46:57 -0400 | From | Tom Talpey <> | Subject | Re: [PATCH v2 09/17] IB/Verbs: Use helper cap_read_multi_sge() and reform svc_rdma_accept() |
| |
On 4/7/2015 8:34 AM, Michael Wang wrote: > /** > + * cap_read_multi_sge - Check if the port of device has the capability > + * RDMA Read Multiple Scatter-Gather Entries. > + * > + * @device: Device to be checked > + * @port_num: Port number of the device > + * > + * Return 0 when port of the device don't support > + * RDMA Read Multiple Scatter-Gather Entries. > + */ > +static inline int cap_read_multi_sge(struct ib_device *device, u8 port_num) > +{ > + return !rdma_transport_iwarp(device, port_num); > +}
This just papers over the issue we discussed earlier. How *many* entries does the device support? If a device supports one, or two, is that enough? How does the upper layer know the limit?
This needs an explicit device attribute, to be fixed properly.
> + > +/** > * cap_ipoib - Check if the port of device has the capability > * IP over Infiniband. > * > diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c > index e011027..604d035 100644 > --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c > +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c > @@ -118,8 +118,8 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp, > > static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count) > { > - if (rdma_node_get_transport(xprt->sc_cm_id->device->node_type) == > - RDMA_TRANSPORT_IWARP) > + if (!cap_read_multi_sge(xprt->sc_cm_id->device, > + xprt->sc_cm_id->port_num)) > return 1; > else > return min_t(int, sge_count, xprt->sc_max_sge);
This is incorrect. The RDMA Read max is not at all the same as the max_sge. It is a different operation, with a different set of work request parameters.
In other words, the above same comment applies.
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c > index 4e61880..e75175d 100644 > --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c > +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c > @@ -979,8 +979,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) > /* > * Determine if a DMA MR is required and if so, what privs are required > */ > - switch (rdma_node_get_transport(newxprt->sc_cm_id->device->node_type)) { > - case RDMA_TRANSPORT_IWARP: > + if (rdma_transport_iwarp(newxprt->sc_cm_id->device, > + newxprt->sc_cm_id->port_num)) { > newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
Do I read this correctly that it is forcing the "read with invalidate" capability to "on" for all iWARP devices? I don't think that is correct, for the legacy devices you're also supporting.
> @@ -992,8 +992,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) > dma_mr_acc = IB_ACCESS_LOCAL_WRITE; > } else > need_dma_mr = 0; > - break; > - case RDMA_TRANSPORT_IB: > + } else if (rdma_ib_mgmt(newxprt->sc_cm_id->device, > + newxprt->sc_cm_id->port_num)) { > if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) { > need_dma_mr = 1; > dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
Now I'm even more confused. How is the presence of IB management related to needing a privileged lmr?
| |