How can I add a timestamp in Roce?

1

follow the manual. I used ibv_exp_query_device to query device capability, but I got an error as follows.

ibv_exp_query_device: invalid comp_mask !!! (comp_mask = 0x30373839 valid_mask = 0x7fffff)

my adapter is ConnectX-4.

ethtools -T eth1

Time stamping parameters for eth1:

Capabilities:

hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)

software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE)

hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE)

software-receive (SOF_TIMESTAMPING_RX_SOFTWARE)

software-system-clock (SOF_TIMESTAMPING_SOFTWARE)

hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE)

PTP Hardware Clock: 0

Hardware Transmit Timestamp Modes:

off (HWTSTAMP_TX_OFF)

on (HWTSTAMP_TX_ON)

Hardware Receive Filter Modes:

none (HWTSTAMP_FILTER_NONE)

all (HWTSTAMP_FILTER_ALL)

2

Hi Chris,

For this issue please open a case with Mellanox support providing the configuration that was done in order to use time stamping.

support@mellanox.com mailto:support@mellanox.com .

Thank you,

3

Hi Chris,

have you received a solution for your problem?

I’m quite interested because we just have installed a new cluster in our compagny and, unfortunately, we have the same error messsage than you when we launch some MPI jobs.

It is a very strong problem for us since, up to now, we are not able to start our studies.

Thank you.

4

If using HPC-X or OpenMPI from the Mellanox OFED, you might add ‘-mca btl self -mca pml ucx’