Hi NVIDIA Community,
I’m experiencing issues running DOCA SDK example applications on a BlueField-3 ARM host and would appreciate any help or guidance.
Issue Description
I’m trying to run the eth_l2_fwd example application on the BlueField-3 ARM host, but it fails during initialization with errors related to port polling and pipe creation.
The command and logs are following:
ubuntu@localhost:/tmp/build/eth_l2_fwd$ sudo ./doca_eth_l2_fwd -d mlx5_0,mlx5_1
[10:16:19:885728][3513391][DOCA][WRN][doca_mmap.cpp:2026][mmap_set_memrange] Mmap 0xaaab0051af00: Memory range isn't aligned to 64B - addr=0xffff900fb010. For best performance using CPU memory
, align address to 64B (cache-line size). For best performance using GPU memory, align address to 64KB (page size)
[10:16:19:890912][3513391][DOCA][WRN][engine_model.c:94][adapt_queue_depth] adapting queue depth to 128.
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/3513391/mp_socket
EAL: Selected IOVA mode 'VA'
TELEMETRY: No legacy callbacks, legacy socket not created
EAL: Probe PCI driver: mlx5_pci (15b3:a2dc) device: 0000:03:00.0 (socket -1)
mlx5_net: port 0: TX queue 0 not yet set up
mlx5_net: port 0: TX queue 1 not yet set up
mlx5_net: port 0: TX queue 2 not yet set up
mlx5_net: port 0: TX queue 3 not yet set up
mlx5_net: port 0: TX queue 4 not yet set up
mlx5_net: port 0: TX queue 5 not yet set up
mlx5_net: port 0: TX queue 6 not yet set up
mlx5_net: port 0: TX queue 7 not yet set up
mlx5_net: port 0: TX queue 8 not yet set up
mlx5_net: port 0: TX queue 9 not yet set up
mlx5_net: port 0: TX queue 10 not yet set up
mlx5_net: port 0: TX queue 11 not yet set up
mlx5_net: port 0: TX queue 12 not yet set up
mlx5_net: port 0: TX queue 13 not yet set up
mlx5_net: port 0: TX queue 14 not yet set up
mlx5_net: port 0: TX queue 15 not yet set up
mlx5_net: port 0: TX queue 0 not yet set up
mlx5_net: port 0: TX queue 1 not yet set up
mlx5_net: port 0: TX queue 2 not yet set up
mlx5_net: port 0: TX queue 3 not yet set up
mlx5_net: port 0: TX queue 4 not yet set up
mlx5_net: port 0: TX queue 5 not yet set up
mlx5_net: port 0: TX queue 6 not yet set up
mlx5_net: port 0: TX queue 7 not yet set up
mlx5_net: port 0: TX queue 8 not yet set up
mlx5_net: port 0: TX queue 9 not yet set up
mlx5_net: port 0: TX queue 10 not yet set up
mlx5_net: port 0: TX queue 11 not yet set up
mlx5_net: port 0: TX queue 12 not yet set up
mlx5_net: port 0: TX queue 13 not yet set up
mlx5_net: port 0: TX queue 14 not yet set up
mlx5_net: port 0: TX queue 15 not yet set up
mlx5_net: Port 0 Rx queue 0 was not set up
[10:16:20:792383][3513391][DOCA][ERR][hws_port.c:3597][hws_port_set_poll_mark] failed allowing poll for mark on port 0
[10:16:20:792402][3513391][DOCA][ERR][engine_pipe.c:678][engine_pipe_create] failed creating pipe - build failed rc=(-19)
[10:16:20:792778][3513391][DOCA][ERR][doca_flow.c:2095][doca_flow_pipe_create] engine pipe creation failed, rc = -19
[10:16:20:792811][3513391][DOCA][ERR][eth_rxq_common.c:204][create_root_pipe] Failed to create doca flow pipe, err: DOCA_ERROR_DRIVER
[10:16:20:792827][3513391][DOCA][ERR][eth_rxq_common.c:276][allocate_eth_rxq_flow_resources] Failed to allocate eth_rxq_flow_resources: failed to create root pipe, err: DOCA_ERROR_DRIVER
[10:16:20:801777][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801806][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801815][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801821][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801831][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801839][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801845][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801852][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801859][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801864][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801876][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801898][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801907][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801913][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:20:801922][3513391][DOCA][CRT][hws_port.c:3822][vnf_def_fdb_pop_completion_cb] default rule pop completion status failed
[10:16:22:071425][3513391][DOCA][ERR][eth_l2_fwd_core.c:820][eth_l2_fwd_execute] Failed to allocate ETH RXQ flow resources for device 1: DOCA Driver call failure
[10:16:22:071458][3513391][DOCA][ERR][eth_l2_fwd.c:468][main] Failed to execute Ethernet L2 Forwarding Application: DOCA Driver call failure
[10:16:22:071951][3513391][DOCA][ERR][doca_dpdk.cpp:476][priv_doca_dpdk_dev_as_port] DPDK bridge failed to find port id for device=0xaaab005122f0
[10:16:22:071969][3513391][DOCA][ERR][doca_dpdk.cpp:493][priv_doca_dpdk_dev_get_port_metadata] Failed to get port metadata: port_id was not found
[10:16:22:071983][3513391][DOCA][ERR][doca_eth_rxq.c:1506][eth_rxq_stop_cpu_ctx] ETH_RXQ 0xaaab00b802f0: Failed to stop eth_rxq: failed to unmap external RQ from a dpdk queue. err=DOCA_ERROR_NOT_FOUND
[10:16:22:071991][3513391][DOCA][ERR][doca_pe.cpp:1096][stop_context] Progress engine 0xaaab0051ad40: Failed to stop ctx=0xaaab00b802f0
[10:16:22:072003][3513391][DOCA][ERR][doca_ctx.cpp:652][priv_doca_ctx_set_state_to_idle] CTX 0xaaab00b802f0 failed to stop with status DOCA_ERROR_NOT_FOUND
[10:16:22:072010][3513391][DOCA][ERR][eth_l2_fwd_core.c:906][eth_l2_fwd_cleanup] Failed to stop device 1 ETH RXQ DOCA CTX: Requested Resource Not Found
[10:16:22:072017][3513391][DOCA][ERR][eth_l2_fwd.c:475][main] Failed to clean up Ethernet L2 Forwarding Application resources: Requested Resource Not Found
This is the likely starting point of the error.
[10:16:20:792383][3513391][DOCA][ERR][hws_port.c:3597][hws_port_set_poll_mark] failed allowing poll for mark on port 0
The same error occurs with other receive-based implementations that use the eth_rxq_common.c in the samples/doca_eth/ directory.
Device Information
- mlxfwmanager
ubuntu@localhost:~$ sudo mlxfwmanager -d /dev/mst/mt41692_pciconf0
Querying Mellanox devices firmware ...
Device #1:
----------
Device Type: BlueField3
Part Number: 900-9D3B6-00CV-A_Ax
Description: NVIDIA BlueField-3 B3220 P-Series FHHL DPU; 200GbE (default mode) / NDR200 IB;
Dual-port QSFP112; PCIe Gen5.0 x16 with x16 PCIe extension option;
16 Arm cores; 32GB on-board DDR; integrated BMC; Crypto Enabled
PSID: MT_0000000884
PCI Device Name: /dev/mst/mt41692_pciconf0
Base MAC: a088c281aad4
Versions: Current Available
FW 32.44.1036 N/A
PXE 3.7.0500 N/A
UEFI 14.37.0014 N/A
UEFI Virtio blk 22.4.0014 N/A
UEFI Virtio net 21.4.0013 N/A
Status: No matching image found
- mlxconfig
Device #1:
----------
Device type: BlueField3
Name: 900-9D3B6-00CV-A_Ax
Description: NVIDIA BlueField-3 B3220 P-Series FHHL DPU; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe Gen5.0 x16 with x16 PCIe extension option; 16 Arm cores; 32GB on-board DDR; integrated BMC; Crypto Enabled
Device: /dev/mst/mt41692_pciconf0
Configurations: Next Boot
MODULE_SPLIT_M0 Array[0..15]
MODULE_SPLIT_M1 Array[0..15]
LARGE_MTU_TWEAK_64 False(0)
AES_XTS_TWEAK_INC_64 False(0)
CRYPTO_POLICY UNRESTRICTED(1)
RO CUSTOMIZATION_NUMBER Array[0..15]
DPA_AUTHENTICATION False(0)
NVME_EMULATION_ENABLE False(0)
NVME_EMULATION_NUM_VF 0
NVME_EMULATION_NUM_PF 1
NVME_EMULATION_VENDOR_ID 5555
NVME_EMULATION_DEVICE_ID 24577
NVME_EMULATION_CLASS_CODE 67586
NVME_EMULATION_REVISION_ID 0
NVME_EMULATION_SUBSYSTEM_VENDOR_ID 0
NVME_EMULATION_SUBSYSTEM_ID 0
NVME_EMULATION_NUM_MSIX 0
NVME_EMULATION_NUM_VF_MSIX 0
NVME_EMULATION_MAX_QUEUE_DEPTH 0
PCI_SWITCH_EMULATION_NUM_PORT 16
PCI_SWITCH_EMULATION_ENABLE False(0)
UPT_EMULATION_ENABLE False(0)
VIRTIO_BLK_EMULATION_ENABLE False(0)
VIRTIO_BLK_EMULATION_NUM_VF 0
VIRTIO_BLK_EMULATION_NUM_PF 0
VIRTIO_BLK_EMU_SUBS_VENDOR_ID 6900
VIRTIO_BLK_EMULATION_SUBSYSTEM_ID 4162
VIRTIO_BLK_EMULATION_NUM_MSIX 2
VIRTIO_BLK_EMULATION_NUM_VF_MSIX 0
VIRTIO_FS_EMULATION_ENABLE False(0)
VIRTIO_FS_EMULATION_NUM_VF 0
VIRTIO_FS_EMULATION_NUM_PF 0
VIRTIO_FS_EMU_SUBSYSTEM_VENDOR_ID 6900
VIRTIO_FS_EMULATION_SUBSYSTEM_ID 4186
VIRTIO_FS_EMULATION_NUM_MSIX 2
VIRTIO_FS_EMULATION_NUM_VF_MSIX 0
VIRTIO_NET_EMULATION_ENABLE False(0)
VIRTIO_NET_EMULATION_NUM_VF 0
VIRTIO_NET_EMULATION_NUM_PF 0
VIRTIO_NET_EMU_SUBSYSTEM_VENDOR_ID 6900
VIRTIO_NET_EMULATION_SUBSYSTEM_ID 4161
VIRTIO_NET_EMULATION_NUM_MSIX 2
VIRTIO_NET_EMULATION_NUM_VF_MSIX 0
HOST_PRIV_FLASH_ACCESS DEVICE_DEFAULT(0)
HOST_PRIV_NV_HOST DEVICE_DEFAULT(0)
HOST_PRIV_NV_PORT DEVICE_DEFAULT(0)
HOST_PRIV_NV_GLOBAL DEVICE_DEFAULT(0)
HOST_PRIV_NV_INTERNAL_CPU DEVICE_DEFAULT(0)
HOST_PRIV_PCC_UPDATE DEVICE_DEFAULT(0)
HOST_PRIV_FW_UPDATE DEVICE_DEFAULT(0)
HOST_PRIV_NIC_RESET DEVICE_DEFAULT(0)
FLEX_PARSER_PROFILE_ENABLE 0
PROG_PARSE_GRAPH False(0)
FLEX_IPV4_OVER_VXLAN_PORT 0
SWITCH_COMPT_FEATURE_MASK 0x0(0)
PF_BAR2_SIZE 3
INTERNAL_CPU_RSHIM ENABLED(0)
PF_NUM_OF_VF_VALID False(0)
NON_PREFETCHABLE_PF_BAR False(0)
VF_VPD_ENABLE False(0)
PF_NUM_PF_MSIX_VALID False(0)
PER_PF_NUM_SF False(0)
STRICT_VF_MSIX_NUM False(0)
VF_NODNIC_ENABLE False(0)
NUM_PF_MSIX_VALID True(1)
NUM_OF_VFS 16
NUM_OF_PF 2
PF_BAR2_ENABLE True(1)
HIDE_PORT2_PF False(0)
SRIOV_EN True(1)
PF_LOG_BAR_SIZE 5
VF_LOG_BAR_SIZE 1
NUM_PF_MSIX 63
NUM_VF_MSIX 11
PCIE_CREDIT_TOKEN_TIMEOUT 0
PCI_DOWNSTREAM_PORT_OWNER Array[0..15]
STRAP_SD_OR_MH False(0)
OFF_BOARD_SERIALIZER False(0)
PCORE2_REVERSAL False(0)
PCORE1_REVERSAL False(0)
PCORE0_REVERSAL False(0)
PCI_BUS01_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS01_SWITCH_INDEX 0
PCI_BUS01_SPEED PCI_GEN_1(0)
PCI_BUS01_ASPM False(0)
PCI_BUS01_WIDTH PCI_INACTIVE(0)
PCI_BUS00_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS00_SWITCH_INDEX 0
PCI_BUS00_SPEED PCI_GEN_1(0)
PCI_BUS00_ASPM False(0)
PCI_BUS00_WIDTH PCI_INACTIVE(0)
PCI_BUS03_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS03_SWITCH_INDEX 0
PCI_BUS03_SPEED PCI_GEN_1(0)
PCI_BUS03_ASPM False(0)
PCI_BUS03_WIDTH PCI_INACTIVE(0)
PCI_BUS02_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS02_SWITCH_INDEX 0
PCI_BUS02_SPEED PCI_GEN_1(0)
PCI_BUS02_ASPM False(0)
PCI_BUS02_WIDTH PCI_INACTIVE(0)
PCI_BUS05_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS05_SWITCH_INDEX 0
PCI_BUS05_SPEED PCI_GEN_1(0)
PCI_BUS05_ASPM False(0)
PCI_BUS05_WIDTH PCI_INACTIVE(0)
PCI_BUS04_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS04_SWITCH_INDEX 0
PCI_BUS04_SPEED PCI_GEN_1(0)
PCI_BUS04_ASPM False(0)
PCI_BUS04_WIDTH PCI_INACTIVE(0)
PCI_BUS07_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS07_SWITCH_INDEX 0
PCI_BUS07_SPEED PCI_GEN_1(0)
PCI_BUS07_ASPM False(0)
PCI_BUS07_WIDTH PCI_INACTIVE(0)
PCI_BUS06_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS06_SWITCH_INDEX 0
PCI_BUS06_SPEED PCI_GEN_1(0)
PCI_BUS06_ASPM False(0)
PCI_BUS06_WIDTH PCI_INACTIVE(0)
PCI_BUS11_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS11_SWITCH_INDEX 0
PCI_BUS11_SPEED PCI_GEN_1(0)
PCI_BUS11_ASPM False(0)
PCI_BUS11_WIDTH PCI_INACTIVE(0)
PCI_BUS10_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS10_SWITCH_INDEX 0
PCI_BUS10_SPEED PCI_GEN_1(0)
PCI_BUS10_ASPM False(0)
PCI_BUS10_WIDTH PCI_INACTIVE(0)
PCI_BUS13_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS13_SWITCH_INDEX 0
PCI_BUS13_SPEED PCI_GEN_1(0)
PCI_BUS13_ASPM False(0)
PCI_BUS13_WIDTH PCI_INACTIVE(0)
PCI_BUS12_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS12_SWITCH_INDEX 0
PCI_BUS12_SPEED PCI_GEN_1(0)
PCI_BUS12_ASPM False(0)
PCI_BUS12_WIDTH PCI_INACTIVE(0)
PCI_BUS15_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS15_SWITCH_INDEX 0
PCI_BUS15_SPEED PCI_GEN_1(0)
PCI_BUS15_ASPM False(0)
PCI_BUS15_WIDTH PCI_INACTIVE(0)
PCI_BUS14_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS14_SWITCH_INDEX 0
PCI_BUS14_SPEED PCI_GEN_1(0)
PCI_BUS14_ASPM False(0)
PCI_BUS14_WIDTH PCI_INACTIVE(0)
PCI_BUS17_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS17_SWITCH_INDEX 0
PCI_BUS17_SPEED PCI_GEN_1(0)
PCI_BUS17_ASPM False(0)
PCI_BUS17_WIDTH PCI_INACTIVE(0)
PCI_BUS16_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS16_SWITCH_INDEX 0
PCI_BUS16_SPEED PCI_GEN_1(0)
PCI_BUS16_ASPM False(0)
PCI_BUS16_WIDTH PCI_INACTIVE(0)
PCI_BUS21_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS21_SWITCH_INDEX 0
PCI_BUS21_SPEED PCI_GEN_1(0)
PCI_BUS21_ASPM False(0)
PCI_BUS21_WIDTH PCI_INACTIVE(0)
PCI_BUS20_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS20_SWITCH_INDEX 0
PCI_BUS20_SPEED PCI_GEN_1(0)
PCI_BUS20_ASPM False(0)
PCI_BUS20_WIDTH PCI_INACTIVE(0)
PCI_BUS23_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS23_SWITCH_INDEX 0
PCI_BUS23_SPEED PCI_GEN_1(0)
PCI_BUS23_ASPM False(0)
PCI_BUS23_WIDTH PCI_INACTIVE(0)
PCI_BUS22_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS22_SWITCH_INDEX 0
PCI_BUS22_SPEED PCI_GEN_1(0)
PCI_BUS22_ASPM False(0)
PCI_BUS22_WIDTH PCI_INACTIVE(0)
PCI_BUS25_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS25_SWITCH_INDEX 0
PCI_BUS25_SPEED PCI_GEN_1(0)
PCI_BUS25_ASPM False(0)
PCI_BUS25_WIDTH PCI_INACTIVE(0)
PCI_BUS24_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS24_SWITCH_INDEX 0
PCI_BUS24_SPEED PCI_GEN_1(0)
PCI_BUS24_ASPM False(0)
PCI_BUS24_WIDTH PCI_INACTIVE(0)
PCI_BUS27_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS27_SWITCH_INDEX 0
PCI_BUS27_SPEED PCI_GEN_1(0)
PCI_BUS27_ASPM False(0)
PCI_BUS27_WIDTH PCI_INACTIVE(0)
PCI_BUS26_HIERARCHY_TYPE PCIE_ENDPOINT(0)
PCI_BUS26_SWITCH_INDEX 0
PCI_BUS26_SPEED PCI_GEN_1(0)
PCI_BUS26_ASPM False(0)
PCI_BUS26_WIDTH PCI_INACTIVE(0)
PCI_SWITCH1_UPSTREAM_PORT_PEX 0
PCI_SWITCH1_UPSTREAM_PORT_BUS 0
PCI_SWITCH0_UPSTREAM_PORT_PEX 0
PCI_SWITCH0_UPSTREAM_PORT_BUS 0
PCI_SWITCH2_UPSTREAM_PORT_PEX 0
PCI_SWITCH2_UPSTREAM_PORT_BUS 0
ROCE_CC_DCQCN_COMPATIBILITY_MODE DEVICE_DEFAULT(0)
ROCE_CC_STEERING_EXT DISABLED(1)
ROCE_CC_LEGACY_DCQCN_SW False(0)
HOST_CHAINING_MODE DISABLED(0)
HOST_CHAINING_CACHE_DISABLE False(0)
HOST_CHAINING_DESCRIPTORS Array[0..7]
HOST_CHAINING_TOTAL_BUFFER_SIZE Array[0..7]
RDE_DISABLE False(0)
PLDM_FW_UPDATE_DISABLE False(0)
RBT_DISABLE False(0)
PCIE_SMBUS_DISABLE False(0)
PCIE_IN_BAND_VDM_DISABLE False(0)
SM_DISABLE False(0)
LOG_DCR_HASH_TABLE_SIZE 11
MAX_PACKET_LIFETIME 0
DCR_LIFO_SIZE 16384
INTERNAL_CPU_MODEL EMBEDDED_CPU(1)
INTERNAL_CPU_PAGE_SUPPLIER ECPF(0)
INTERNAL_CPU_ESWITCH_MANAGER ECPF(0)
INTERNAL_CPU_IB_VPORT0 ECPF(0)
INTERNAL_CPU_OFFLOAD_ENGINE ENABLED(0)
ESWITCH_HAIRPIN_DESCRIPTORS Array[0..7]
ESWITCH_HAIRPIN_TOT_BUFFER_SIZE Array[0..7]
MEMIC_BAR_SIZE 0
MEMIC_SIZE_LIMIT _256KB(1)
MEMIC_ATOMIC MEMIC_ATOMIC_ENABLE(2)
PCC_NP_HANDLE_CORE_UTIL DEVICE_DEFAULT(0)
PCC_HANDLE_CORE_UTIL DEVICE_DEFAULT(0)
PCC_INT_NP_RTT_DSCP 26
PCC_INT_NP_RTT_DSCP_EN False(0)
PCC_INT_NP_RTT_DATA_MODE RTT_V0(64)
PCC_INT_EN False(0)
PCC_INT_SYSTEM_RTT 0
STEERING_CACHE_REFRESH 0
TX_SCHEDULER_LOCALITY_MODE DEVICE_DEFAULT(0)
ICM_CACHE_MODE DEVICE_DEFAULT(0)
TX_SCHEDULER_FWS_REACTIVITY DIRECT(1)
HAIRPIN_DATA_BUFFER_LOCK False(0)
TX_SCHEDULER_BURST 0
ZERO_TOUCH_TUNING_ENABLE False(0)
TX_PCI_DATA_FETCH_LATENCY 40
RT_PPS_ENABLED_ON_POWERUP False(0)
LAG_RESOURCE_ALLOCATION DEVICE_DEFAULT(0)
ACCURATE_TX_SCHEDULER False(0)
PARTIAL_RESET_EN False(0)
RESET_WITH_HOST_ON_ERRORS False(0)
USER_PROGRAMMABLE_CC False(0)
ROCE_NEXT_PROTOCOL 254
LOG_MAX_QUEUE 17
LOG_MAX_OUTSTANDING_READ_ATOMIC 0
MULTIPATH_DSCP DEVICE_DEFAULT(0)
SWP_L4_CHECKSUM_MODE DEVICE_DEFAULT(0)
CQE_COMPRESSION BALANCED(0)
IP_OVER_VXLAN_EN False(0)
MKEY_BY_NAME False(0)
PRIO_TAG_REQUIRED_EN False(0)
UCTX_EN True(1)
REAL_TIME_CLOCK_ENABLE False(0)
RDMA_SELECTIVE_REPEAT_EN False(0)
PCI_ATOMIC_MODE PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0)
TUNNEL_ECN_COPY_DISABLE False(0)
LRO_LOG_TIMEOUT0 6
LRO_LOG_TIMEOUT1 7
LRO_LOG_TIMEOUT2 8
LRO_LOG_TIMEOUT3 13
LOG_TX_PSN_WINDOW 9
VF_MIGRATION_MODE DEVICE_DEFAULT(0)
LOG_MAX_OUTSTANDING_WQE 7
ROCE_ADAPTIVE_ROUTING_EN False(0)
TUNNEL_IP_PROTO_ENTROPY_DISABLE False(0)
INT_LOG_MAX_PAYLOAD_SIZE AUTOMATIC(0)
KEEP_ETH_LINK_UP_P1 True(1)
KEEP_IB_LINK_UP_P1 False(0)
KEEP_LINK_UP_ON_BOOT_P1 False(0)
KEEP_LINK_UP_ON_STANDBY_P1 False(0)
DO_NOT_CLEAR_PORT_STATS_P1 False(0)
AUTO_POWER_SAVE_LINK_DOWN_P1 False(0)
KEEP_ETH_LINK_UP_P2 True(1)
KEEP_IB_LINK_UP_P2 False(0)
KEEP_LINK_UP_ON_BOOT_P2 False(0)
KEEP_LINK_UP_ON_STANDBY_P2 False(0)
DO_NOT_CLEAR_PORT_STATS_P2 False(0)
AUTO_POWER_SAVE_LINK_DOWN_P2 False(0)
PHY_AUTO_NEG_P1 DEVICE_DEFAULT(0)
PHY_RATE_MASK_OVERRIDE_P1 False(0)
PHY_FEC_OVERRIDE_P1 DEVICE_DEFAULT(0)
PHY_AUTO_NEG_P2 DEVICE_DEFAULT(0)
PHY_RATE_MASK_OVERRIDE_P2 False(0)
PHY_FEC_OVERRIDE_P2 DEVICE_DEFAULT(0)
LLDP_NB_DCBX_P1 False(0)
LLDP_NB_RX_MODE_P1 OFF(0)
LLDP_NB_TX_MODE_P1 OFF(0)
LLDP_NB_DCBX_P2 False(0)
LLDP_NB_RX_MODE_P2 OFF(0)
LLDP_NB_TX_MODE_P2 OFF(0)
DCBX_IEEE_P1 True(1)
DCBX_CEE_P1 True(1)
DCBX_WILLING_P1 True(1)
DCBX_IEEE_P2 True(1)
DCBX_CEE_P2 True(1)
DCBX_WILLING_P2 True(1)
DUP_MAC_ACTION_P1 LAST_CFG(0)
MPFS_MC_LOOPBACK_DISABLE_P1 False(0)
MPFS_UC_LOOPBACK_DISABLE_P1 False(0)
UNKNOWN_UPLINK_MAC_FLOOD_P1 False(0)
SRIOV_IB_ROUTING_MODE_P1 LID(1)
IB_ROUTING_MODE_P1 LID(1)
DUP_MAC_ACTION_P2 LAST_CFG(0)
MPFS_MC_LOOPBACK_DISABLE_P2 False(0)
MPFS_UC_LOOPBACK_DISABLE_P2 False(0)
UNKNOWN_UPLINK_MAC_FLOOD_P2 False(0)
SRIOV_IB_ROUTING_MODE_P2 LID(1)
IB_ROUTING_MODE_P2 LID(1)
NUM_OF_VL_P1 _4_VLs(3)
NUM_OF_TC_P1 _8_TCs(0)
NUM_OF_PFC_P1 8
VL15_BUFFER_SIZE_P1 0
QOS_TRUST_STATE_P1 TRUST_PCP(1)
ETS_SCHED_MODE_P1 device_default(0)
VL_BUFFER_ALLOCATION_P1 FULL_DYNAMIC(1)
NUM_OF_VL_P2 _4_VLs(3)
NUM_OF_TC_P2 _8_TCs(0)
NUM_OF_PFC_P2 8
VL15_BUFFER_SIZE_P2 0
QOS_TRUST_STATE_P2 TRUST_PCP(1)
ETS_SCHED_MODE_P2 device_default(0)
VL_BUFFER_ALLOCATION_P2 FULL_DYNAMIC(1)
ROCE_CC_PRIO_MASK_P1 255
ROCE_CC_CNP_MODERATION_P1 DEVICE_DEFAULT(0)
ROCE_CC_SHAPER_COALESCE_P1 DEVICE_DEFAULT(0)
IB_CC_SHAPER_COALESCE_P1 DEVICE_DEFAULT(0)
ROCE_CC_PRIO_MASK_P2 255
ROCE_CC_CNP_MODERATION_P2 DEVICE_DEFAULT(0)
ROCE_CC_SHAPER_COALESCE_P2 DEVICE_DEFAULT(0)
IB_CC_SHAPER_COALESCE_P2 DEVICE_DEFAULT(0)
CLAMP_TGT_RATE_AFTER_TIME_INC_P1 True(1)
CLAMP_TGT_RATE_P1 False(0)
RPG_TIME_RESET_P1 300
RPG_BYTE_RESET_P1 32767
RPG_THRESHOLD_P1 1
RPG_MAX_RATE_P1 0
RPG_AI_RATE_P1 5
RPG_HAI_RATE_P1 50
RPG_GD_P1 11
RPG_MIN_DEC_FAC_P1 50
RPG_MIN_RATE_P1 1
RATE_TO_SET_ON_FIRST_CNP_P1 0
DCE_TCP_G_P1 1019
DCE_TCP_RTT_P1 1
RATE_REDUCE_MONITOR_PERIOD_P1 4
INITIAL_ALPHA_VALUE_P1 1023
MIN_TIME_BETWEEN_CNPS_P1 4
CNP_802P_PRIO_P1 6
CNP_DSCP_P1 48
CLAMP_TGT_RATE_AFTER_TIME_INC_P2 True(1)
CLAMP_TGT_RATE_P2 False(0)
RPG_TIME_RESET_P2 300
RPG_BYTE_RESET_P2 32767
RPG_THRESHOLD_P2 1
RPG_MAX_RATE_P2 0
RPG_AI_RATE_P2 5
RPG_HAI_RATE_P2 50
RPG_GD_P2 11
RPG_MIN_DEC_FAC_P2 50
RPG_MIN_RATE_P2 1
RATE_TO_SET_ON_FIRST_CNP_P2 0
DCE_TCP_G_P2 1019
DCE_TCP_RTT_P2 1
RATE_REDUCE_MONITOR_PERIOD_P2 4
INITIAL_ALPHA_VALUE_P2 1023
MIN_TIME_BETWEEN_CNPS_P2 4
CNP_802P_PRIO_P2 6
CNP_DSCP_P2 48
ROCE_RTT_RESP_DSCP_P1 0
ROCE_RTT_RESP_DSCP_MODE_P1 DEVICE_DEFAULT(0)
ROCE_RTT_RESP_DSCP_P2 0
ROCE_RTT_RESP_DSCP_MODE_P2 DEVICE_DEFAULT(0)
LINK_TYPE_P1 ETH(2)
LINK_TYPE_P2 ETH(2)
ECPF_ESWITCH_MANAGER ECPF(1)
ECPF_PAGE_SUPPLIER ECPF(1)
PORT_OWNER True(1)
ALLOW_RD_COUNTERS True(1)
RENEG_ON_CHANGE True(1)
TRACER_ENABLE True(1)
SILENT_MODE False(0)
MKEY_BY_NAME_RANGE DEVICE_DEFAULT(0)
ROCE_CONTROL ROCE_ENABLE(2)
PCI_WR_ORDERING per_mkey(0)
MULTI_PORT_VHCA_EN False(0)
PF_TOTAL_SF 0
PF_SD_GROUP 0
PF_DEVICE_ID_ENABLE False(0)
PF_SF_BAR_SIZE 0
PF_NUM_PF_MSIX 63
PF_DEVICE_ID 41692
BOOT_VLAN 1
LEGACY_BOOT_PROTOCOL PXE(1)
BOOT_INTERRUPT_DIS False(0)
BOOT_LACP_DIS True(1)
BOOT_VLAN_EN False(0)
IP_VER IPv4(0)
BOOT_UNDI_NETWORK_WAIT 0
BOOT_DBG_LOG False(0)
BOOT_PKEY 0
UEFI_HII_EN True(1)
UEFI_LOGS DISABLED(0)
BAR_PAGE_ALIGNMENT DEVICE_DEFAULT(0)
MANAGEMENT_PF_MODE DEVICE_DEFAULT(0)
P2P_ORDERING_MODE DEVICE_DEFAULT(0)
EXP_ROM_VIRTIO_NET_PXE_ENABLE False(0)
EXP_ROM_VIRTIO_NET_UEFI_ARM_ENABLE False(0)
EXP_ROM_VIRTIO_NET_UEFI_x86_ENABLE False(0)
EXP_ROM_VIRTIO_BLK_UEFI_ARM_ENABLE False(0)
EXP_ROM_VIRTIO_BLK_UEFI_x86_ENABLE False(0)
EXP_ROM_NVME_UEFI_x86_ENABLE True(1)
ATS_ENABLED False(0)
DYNAMIC_VF_MSIX_TABLE False(0)
EXP_ROM_UEFI_ARM_ENABLE True(1)
EXP_ROM_UEFI_x86_ENABLE True(1)
EXP_ROM_PXE_ENABLE True(1)
ADVANCED_PCI_SETTINGS False(0)
SAFE_MODE_THRESHOLD 10
SAFE_MODE_ENABLE True(1)
The 'RO' shows parameters which are for read only and cannot be changed
- devlink
ubuntu@localhost:~$ devlink port
pci/0000:03:00.0/262143: type eth netdev p0 flavour physical port 0 splittable false
function:
hw_addr 00:00:00:00:00:00 state inactive opstate detached
pci/0000:03:00.0/196608: type eth netdev pf0hpf flavour pcipf controller 1 pfnum 0 external true splittable false
function:
hw_addr a0:88:c2:81:aa:d4 state inactive opstate detached
pci/0000:03:00.0/229408: type eth netdev en3f0pf0sf0 flavour pcisf controller 0 pfnum 0 sfnum 0 splittable false
function:
hw_addr 02:d4:65:37:a1:19 state active opstate attached
pci/0000:03:00.1/327679: type eth netdev p1 flavour physical port 1 splittable false
function:
hw_addr 00:00:00:00:00:00 state inactive opstate detached
pci/0000:03:00.1/262144: type eth netdev pf1hpf flavour pcipf controller 1 pfnum 1 external true splittable false
function:
hw_addr a0:88:c2:81:aa:d5 state inactive opstate detached
pci/0000:03:00.1/294944: type eth netdev en3f1pf1sf0 flavour pcisf controller 0 pfnum 1 sfnum 0 splittable false
function:
hw_addr 02:c8:1d:8a:9d:ca state active opstate attached
auxiliary/mlx5_core.sf.2/6619136: type eth netdev enp3s0f0s0 flavour virtual splittable false
auxiliary/mlx5_core.sf.3/8716288: type eth netdev enp3s0f1s0 flavour virtual splittable false
I’m happy to provide any additional information, logs, or configuration details that might help diagnose this issue.
Thank you.