Multi-packet RQ with RC/SRQ

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hi, I have been investigating the multi-packet/striding RQ feature that’s present in ConnectX-4+. The documentation on this feature is scant but I was curious if it was possible to setup a SRQ with multi-packet support that can be connected to the usual RC QPs, and if there’s an example code I could look at. I’ve seen several examples that use a raw packet QPs with multi-packet support, but that does not solve my problem. Thank you!

2

Hi,

Thanks for your question. It may require a support case in Nvidia portal, which will be handled according to the support entitlement.

Best Regards,
Anatoly

4

I am also researching this issue. According to https://docs.nvidia.com/networking/display/rdmacore50/multi-packet+rq, multi-packet RQ seems to be only applicable to WQ. In MLNX-OFED 4.x, the ibv_exp_wq contains the srq field. However, in the latest version, ibv_wq does not have this field. Perhaps only by using MLNX-OFED 4.x can Multi-packet RQ be used together with SRQ. I wonder if DOCA-OFED has solved this problem. The relevant code is as follows:

// verbs_exp.h in mlnx-ofed-kernel-dkms 4.9-OFED.4.9.5.1.0.1
struct ibv_exp_wq {      
	struct ibv_context *context;
	void *wq_context; /* Associated Context of the WQ */
	uint32_t handle;
	/* Protection domain WQ should be associated with */
	struct ibv_pd *pd;
	/* CQ to be associated with the WQ */
	struct ibv_cq *cq;
	/* SRQ handle if WQ is to be associated with an SRQ, otherwise NULL */
	struct ibv_srq *srq;
	uint32_t wq_num;
	enum ibv_exp_wq_state state;
	enum ibv_exp_wq_type wq_type;
	uint32_t comp_mask;
};

struct ibv_exp_wq_init_attr {
	// ...
	struct ibv_exp_wq_mp_rq mp_rq;
	// ...
}

// verbs.h in mlnx-ofed-kernel-dkms 24.07.OFED.24.07.0.6.1.1-1
struct ibv_wq { 
	struct ibv_context *context;
	void *wq_context;
	struct ibv_pd *pd;
	struct ibv_cq *cq;
	uint32_t wq_num;
	uint32_t handle;
	enum ibv_wq_state state;
	enum ibv_wq_type wq_type;
	int (*post_recv)(struct ibv_wq *current,
	struct ibv_recv_wr *recv_wr,
	struct ibv_recv_wr **bad_recv_wr);
	pthread_mutex_t mutex;
	pthread_cond_t cond;
	uint32_t events_completed;
	uint32_t comp_mask;
};