Device and environment details: Issue:
sudo mlxconfig -d /dev/mst/mt41692_pciconf0 s PCI_BUS00_HIERARCHY_TYPE=0
But the tool responds with:
-E- Unknown Parameter: PCI_BUS00_HIERARCHY_TYPE
Questions:
Is PCI_BUS00_HIERARCHY_TYPE still valid for my specific device (B3220) and firmware version, or has it been renamed/removed?
If the parameter has changed, what is the correct parameter name or sequence for configuring PCIe bifurcation for the downstream port on this device?
Are there prerequisites or mode settings (DPU mode vs NIC mode) that must be set before bifurcation can be configured, which might hide or disable the parameter?
Are there known firmware/tool version requirements for enabling that parameter?
If I mis-configure bifurcation and the DPU/ARM system fails to boot, what is the recommended recovery path through mlxconfig or firmware fallback?
Hi,
Before applying set parameter, please run first
sudo mlxconfig -d /dev/mst/mt41692_pciconf0 q | grep PCI_BUS00_HIERARCHY_TYPE
If you get that parameter you can set
If not appear maybe need search the right parameter
Thanks
Thank you for your help. The parameter does not appear in the output ofsudo mlxconfig -d /dev/mst/mt41692_pciconf0 qrenamed , removed , or deprecated ?
Yes, I followed that documentation and applied the configuration steps described in the “DPU ARMs as Root Port on Peripherals” section of the PCIe Bifurcation Configuration Options chapter.
Here are the commands I used:
mlxconfig -d <device> s PCI_BUS00_HIERARCHY_TYPE=0
mlxconfig -d <device> s PCI_BUS00_WIDTH=5
mlxconfig -d <device> s PCI_BUS00_SPEED=4
mlxconfig -d <device> s PCI_BUS10_HIERARCHY_TYPE=2
mlxconfig -d <device> s PCI_BUS10_WIDTH=3
mlxconfig -d <device> s PCI_BUS10_SPEED=4
mlxconfig -d <device> s PCI_BUS12_HIERARCHY_TYPE=2
mlxconfig -d <device> s PCI_BUS12_WIDTH=3
mlxconfig -d <device> s PCI_BUS12_SPEED=4
mlxconfig -d <device> s PCI_BUS14_HIERARCHY_TYPE=2
mlxconfig -d <device> s PCI_BUS14_WIDTH=3
mlxconfig -d <device> s PCI_BUS14_SPEED=4
mlxconfig -d <device> s PCI_BUS16_HIERARCHY_TYPE=2
mlxconfig -d <device> s PCI_BUS16_WIDTH=3
mlxconfig -d <device> s PCI_BUS16_SPEED=4
However, the very first command failed with the following error:
ubuntu@localhost:~$ sudo mlxconfig -d /dev/mst/mt41692_pciconf0 s PCI_BUS00_HIERARCHY_TYPE=0
Device #1:
----------
Device type: BlueField3
Name: 900-9D3B6-00CV-A_Ax
Description: NVIDIA BlueField-3 B3220 P-Series FHHL DPU; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe Gen5.0 x16 with x16 PCIe extension option; 16 Arm cores; 32GB on-board DDR; integrated BMC; Crypto Enabled
Device: /dev/mst/mt41692_pciconf0
Configurations: Next Boot New
-E- Unknown Parameter: PCI_BUS00_HIERARCHY_TYPE
Share
sudo mlxconfig -d /dev/mst/mt41692_pciconf0 q
ubuntu@localhost:~$ sudo mlxconfig -d /dev/mst/mt41692_pciconf0 q
Device #1:
----------
Device type: BlueField3
Name: 900-9D3B6-00CV-A_Ax
Description: NVIDIA BlueField-3 B3220 P-Series FHHL DPU; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe Gen5.0 x16 with x16 PCIe extension option; 16 Arm cores; 32GB on-board DDR; integrated BMC; Crypto Enabled
Device: /dev/mst/mt41692_pciconf0
Configurations: Next Boot
MODULE_SPLIT_M0 Array[0..15]
MODULE_SPLIT_M1 Array[0..15]
MEMIC_BAR_SIZE 0
MEMIC_SIZE_LIMIT _256KB(1)
MEMIC_ATOMIC MEMIC_ATOMIC_ENABLE(2)
HOST_CHAINING_MODE DISABLED(0)
HOST_CHAINING_CACHE_DISABLE False(0)
HOST_CHAINING_DESCRIPTORS Array[0..7]
HOST_CHAINING_TOTAL_BUFFER_SIZE Array[0..7]
INTERNAL_CPU_MODEL EMBEDDED_CPU(1)
INTERNAL_CPU_PAGE_SUPPLIER ECPF(0)
INTERNAL_CPU_ESWITCH_MANAGER ECPF(0)
INTERNAL_CPU_IB_VPORT0 ECPF(0)
INTERNAL_CPU_OFFLOAD_ENGINE ENABLED(0)
FLEX_PARSER_PROFILE_ENABLE 0
PROG_PARSE_GRAPH False(0)
FLEX_IPV4_OVER_VXLAN_PORT 0
ROCE_NEXT_PROTOCOL 254
ESWITCH_HAIRPIN_DESCRIPTORS Array[0..7]
ESWITCH_HAIRPIN_TOT_BUFFER_SIZE Array[0..7]
PF_BAR2_SIZE 3
INTERNAL_CPU_RSHIM ENABLED(0)
PF_NUM_OF_VF_VALID False(0)
NON_PREFETCHABLE_PF_BAR False(0)
VF_VPD_ENABLE False(0)
PF_NUM_PF_MSIX_VALID False(0)
PER_PF_NUM_SF False(0)
STRICT_VF_MSIX_NUM False(0)
VF_NODNIC_ENABLE False(0)
NUM_PF_MSIX_VALID True(1)
NUM_OF_VFS 16
NUM_OF_PF 2
PF_BAR2_ENABLE True(1)
HIDE_PORT2_PF False(0)
SRIOV_EN True(1)
PF_LOG_BAR_SIZE 5
VF_LOG_BAR_SIZE 1
NUM_PF_MSIX 63
NUM_VF_MSIX 11
INT_LOG_MAX_PAYLOAD_SIZE AUTOMATIC(0)
PCIE_CREDIT_TOKEN_TIMEOUT 0
RT_PPS_ENABLED_ON_POWERUP False(0)
LAG_RESOURCE_ALLOCATION DEVICE_DEFAULT(0)
ACCURATE_TX_SCHEDULER False(0)
PARTIAL_RESET_EN False(0)
RESET_WITH_HOST_ON_ERRORS False(0)
NVME_EMULATION_ENABLE False(0)
NVME_EMULATION_NUM_VF 0
NVME_EMULATION_NUM_PF 1
NVME_EMULATION_VENDOR_ID 5555
NVME_EMULATION_DEVICE_ID 24577
NVME_EMULATION_CLASS_CODE 67586
NVME_EMULATION_REVISION_ID 0
NVME_EMULATION_SUBSYSTEM_VENDOR_ID 0
NVME_EMULATION_SUBSYSTEM_ID 0
NVME_EMULATION_NUM_MSIX 0
NVME_EMULATION_MAX_QUEUE_DEPTH 0
PCI_SWITCH_EMULATION_NUM_PORT 16
PCI_SWITCH_EMULATION_ENABLE False(0)
VIRTIO_NET_EMULATION_ENABLE False(0)
VIRTIO_NET_EMULATION_NUM_VF 0
VIRTIO_NET_EMULATION_NUM_PF 0
VIRTIO_NET_EMU_SUBSYSTEM_VENDOR_ID 6900
VIRTIO_NET_EMULATION_SUBSYSTEM_ID 4161
VIRTIO_NET_EMULATION_NUM_MSIX 2
VIRTIO_BLK_EMULATION_ENABLE False(0)
VIRTIO_BLK_EMULATION_NUM_VF 0
VIRTIO_BLK_EMULATION_NUM_PF 0
VIRTIO_BLK_EMU_SUBSYSTEM_VENDOR_ID 6900
VIRTIO_BLK_EMULATION_SUBSYSTEM_ID 4162
VIRTIO_BLK_EMULATION_NUM_MSIX 2
PCI_DOWNSTREAM_PORT_OWNER Array[0..15]
VIRTIO_FS_EMULATION_ENABLE False(0)
VIRTIO_FS_EMULATION_NUM_VF 0
VIRTIO_FS_EMULATION_NUM_PF 0
VIRTIO_FS_EMU_SUBSYSTEM_VENDOR_ID 6900
VIRTIO_FS_EMULATION_SUBSYSTEM_ID 4186
VIRTIO_FS_EMULATION_NUM_MSIX 2
CQE_COMPRESSION BALANCED(0)
IP_OVER_VXLAN_EN False(0)
MKEY_BY_NAME False(0)
PRIO_TAG_REQUIRED_EN False(0)
UCTX_EN True(1)
REAL_TIME_CLOCK_ENABLE False(0)
RDMA_SELECTIVE_REPEAT_EN False(0)
PCI_ATOMIC_MODE PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0)
TUNNEL_ECN_COPY_DISABLE False(0)
LRO_LOG_TIMEOUT0 6
LRO_LOG_TIMEOUT1 7
LRO_LOG_TIMEOUT2 8
LRO_LOG_TIMEOUT3 13
LOG_TX_PSN_WINDOW 9
VF_MIGRATION_MODE DEVICE_DEFAULT(0)
LOG_MAX_OUTSTANDING_WQE 7
ROCE_ADAPTIVE_ROUTING_EN False(0)
TUNNEL_IP_PROTO_ENTROPY_DISABLE False(0)
USER_PROGRAMMABLE_CC False(0)
PCC_HANDLE_CORE_UTIL DEVICE_DEFAULT(0)
PCC_INT_NP_RTT_DSCP 26
PCC_INT_NP_RTT_DSCP_EN False(0)
PCC_INT_NP_RTT_DATA_MODE RTT_V0(64)
PCC_INT_EN False(0)
PCC_INT_SYSTEM_RTT 0
MULTI_PCI_RESOURCE_SHARING DEVICE_DEFAULT(0)
ICM_CACHE_MODE DEVICE_DEFAULT(0)
HAIRPIN_DATA_BUFFER_LOCK False(0)
TX_SCHEDULER_BURST 0
ZERO_TOUCH_TUNING_ENABLE False(0)
LOG_MAX_QUEUE 17
UPT_EMULATION_ENABLE False(0)
AES_XTS_TWEAK_INC_64 False(0)
CRYPTO_POLICY UNRESTRICTED(1)
LOG_DCR_HASH_TABLE_SIZE 11
MAX_PACKET_LIFETIME 0
DCR_LIFO_SIZE 16384
LINK_TYPE_P1 ETH(2)
LINK_TYPE_P2 ETH(2)
ROCE_CC_PRIO_MASK_P1 255
ROCE_CC_SHAPER_COALESCE_P1 DEVICE_DEFAULT(0)
IB_CC_SHAPER_COALESCE_P1 DEVICE_DEFAULT(0)
ROCE_CC_PRIO_MASK_P2 255
ROCE_CC_SHAPER_COALESCE_P2 DEVICE_DEFAULT(0)
IB_CC_SHAPER_COALESCE_P2 DEVICE_DEFAULT(0)
CLAMP_TGT_RATE_AFTER_TIME_INC_P1 True(1)
CLAMP_TGT_RATE_P1 False(0)
RPG_TIME_RESET_P1 300
RPG_BYTE_RESET_P1 32767
RPG_THRESHOLD_P1 1
RPG_MAX_RATE_P1 0
RPG_AI_RATE_P1 5
RPG_HAI_RATE_P1 50
RPG_GD_P1 11
RPG_MIN_DEC_FAC_P1 50
RPG_MIN_RATE_P1 1
RATE_TO_SET_ON_FIRST_CNP_P1 0
DCE_TCP_G_P1 1019
DCE_TCP_RTT_P1 1
RATE_REDUCE_MONITOR_PERIOD_P1 4
INITIAL_ALPHA_VALUE_P1 1023
MIN_TIME_BETWEEN_CNPS_P1 4
CNP_802P_PRIO_P1 6
CNP_DSCP_P1 48
CLAMP_TGT_RATE_AFTER_TIME_INC_P2 True(1)
CLAMP_TGT_RATE_P2 False(0)
RPG_TIME_RESET_P2 300
RPG_BYTE_RESET_P2 32767
RPG_THRESHOLD_P2 1
RPG_MAX_RATE_P2 0
RPG_AI_RATE_P2 5
RPG_HAI_RATE_P2 50
RPG_GD_P2 11
RPG_MIN_DEC_FAC_P2 50
RPG_MIN_RATE_P2 1
RATE_TO_SET_ON_FIRST_CNP_P2 0
DCE_TCP_G_P2 1019
DCE_TCP_RTT_P2 1
RATE_REDUCE_MONITOR_PERIOD_P2 4
INITIAL_ALPHA_VALUE_P2 1023
MIN_TIME_BETWEEN_CNPS_P2 4
CNP_802P_PRIO_P2 6
CNP_DSCP_P2 48
LLDP_NB_DCBX_P1 False(0)
LLDP_NB_RX_MODE_P1 OFF(0)
LLDP_NB_TX_MODE_P1 OFF(0)
LLDP_NB_DCBX_P2 False(0)
LLDP_NB_RX_MODE_P2 OFF(0)
LLDP_NB_TX_MODE_P2 OFF(0)
ROCE_RTT_RESP_DSCP_P1 0
ROCE_RTT_RESP_DSCP_MODE_P1 DEVICE_DEFAULT(0)
ROCE_RTT_RESP_DSCP_P2 0
ROCE_RTT_RESP_DSCP_MODE_P2 DEVICE_DEFAULT(0)
DCBX_IEEE_P1 True(1)
DCBX_CEE_P1 True(1)
DCBX_WILLING_P1 True(1)
DCBX_IEEE_P2 True(1)
DCBX_CEE_P2 True(1)
DCBX_WILLING_P2 True(1)
KEEP_ETH_LINK_UP_P1 True(1)
KEEP_IB_LINK_UP_P1 False(0)
KEEP_LINK_UP_ON_BOOT_P1 False(0)
KEEP_LINK_UP_ON_STANDBY_P1 False(0)
DO_NOT_CLEAR_PORT_STATS_P1 False(0)
AUTO_POWER_SAVE_LINK_DOWN_P1 False(0)
KEEP_ETH_LINK_UP_P2 True(1)
KEEP_IB_LINK_UP_P2 False(0)
KEEP_LINK_UP_ON_BOOT_P2 False(0)
KEEP_LINK_UP_ON_STANDBY_P2 False(0)
DO_NOT_CLEAR_PORT_STATS_P2 False(0)
AUTO_POWER_SAVE_LINK_DOWN_P2 False(0)
NUM_OF_VL_P1 _4_VLs(3)
NUM_OF_TC_P1 _8_TCs(0)
NUM_OF_PFC_P1 8
VL15_BUFFER_SIZE_P1 0
QOS_TRUST_STATE_P1 TRUST_PCP(1)
NUM_OF_VL_P2 _4_VLs(3)
NUM_OF_TC_P2 _8_TCs(0)
NUM_OF_PFC_P2 8
VL15_BUFFER_SIZE_P2 0
QOS_TRUST_STATE_P2 TRUST_PCP(1)
DUP_MAC_ACTION_P1 LAST_CFG(0)
MPFS_MC_LOOPBACK_DISABLE_P1 False(0)
MPFS_UC_LOOPBACK_DISABLE_P1 False(0)
UNKNOWN_UPLINK_MAC_FLOOD_P1 False(0)
SRIOV_IB_ROUTING_MODE_P1 LID(1)
IB_ROUTING_MODE_P1 LID(1)
DUP_MAC_ACTION_P2 LAST_CFG(0)
MPFS_MC_LOOPBACK_DISABLE_P2 False(0)
MPFS_UC_LOOPBACK_DISABLE_P2 False(0)
UNKNOWN_UPLINK_MAC_FLOOD_P2 False(0)
SRIOV_IB_ROUTING_MODE_P2 LID(1)
IB_ROUTING_MODE_P2 LID(1)
PHY_AUTO_NEG_P1 DEVICE_DEFAULT(0)
PHY_RATE_MASK_OVERRIDE_P1 False(0)
PHY_FEC_OVERRIDE_P1 DEVICE_DEFAULT(0)
PHY_AUTO_NEG_P2 DEVICE_DEFAULT(0)
PHY_RATE_MASK_OVERRIDE_P2 False(0)
PHY_FEC_OVERRIDE_P2 DEVICE_DEFAULT(0)
PF_TOTAL_SF 0
PF_SF_BAR_SIZE 0
PF_NUM_PF_MSIX 63
ROCE_CONTROL ROCE_ENABLE(2)
PCI_WR_ORDERING per_mkey(0)
MULTI_PORT_VHCA_EN False(0)
ECPF_ESWITCH_MANAGER ECPF(1)
ECPF_PAGE_SUPPLIER ECPF(1)
PORT_OWNER True(1)
ALLOW_RD_COUNTERS True(1)
RENEG_ON_CHANGE True(1)
TRACER_ENABLE True(1)
IP_VER IPv4(0)
BOOT_UNDI_NETWORK_WAIT 0
UEFI_HII_EN True(1)
BOOT_DBG_LOG False(0)
UEFI_LOGS DISABLED(0)
BOOT_VLAN 1
LEGACY_BOOT_PROTOCOL PXE(1)
BOOT_INTERRUPT_DIS False(0)
BOOT_LACP_DIS True(1)
BOOT_VLAN_EN False(0)
BOOT_PKEY 0
P2P_ORDERING_MODE DEVICE_DEFAULT(0)
EXP_ROM_VIRTIO_NET_PXE_ENABLE False(0)
EXP_ROM_VIRTIO_NET_UEFI_ARM_ENABLE False(0)
EXP_ROM_VIRTIO_NET_UEFI_x86_ENABLE False(0)
EXP_ROM_VIRTIO_BLK_UEFI_ARM_ENABLE False(0)
EXP_ROM_VIRTIO_BLK_UEFI_x86_ENABLE False(0)
EXP_ROM_NVME_UEFI_x86_ENABLE True(1)
ATS_ENABLED False(0)
DYNAMIC_VF_MSIX_TABLE False(0)
EXP_ROM_UEFI_ARM_ENABLE True(1)
EXP_ROM_UEFI_x86_ENABLE True(1)
EXP_ROM_PXE_ENABLE True(1)
ADVANCED_PCI_SETTINGS False(0)
SAFE_MODE_THRESHOLD 10
SAFE_MODE_ENABLE True(1)
Hi,
Please run this command:
root@localhost:/home/ubuntu# bf-info
share the output
Hi@tuanama00
# bf-info command not found
root@localhost:/home/ubuntu# bf-info
bf-info: command not found
# bfb-info
root@localhost:/home/ubuntu# bfb-info
Versions:
ATF: v2.2(release):4.0.3-25-g4e9af27
UEFI: 4.0.3-10-gb8abccc
BSP: mlxbf-bootimages 4.0.3-12773
DOCA Base (OFED): 23.04-0.5.3.0
MFT: 4.24.0-72
DOCA: 2.0.2027-1
Firmware:
BF2 FW: 24.37.1300
BF3 FW: 32.37.1306
mlnx-dpdk: ‘MLNX_DPDK 22.11.1.4.2’
mlx-regex 1.2-ubuntu1
virtio-net-controller 1.5.18-1
collectx-clxapi 1.13.2
libvma 9.8.20-1
libxlio 3.0.2-1.2304053
dpcp 1.1.39-1.2304053
dpkg-query: no packages found matching mlnx-snap
SNAP3:
mlnx-libsnap 1.5.2-5
spdk 23.01-6
DOCA:
doca-apps 2.0.2027-1
doca-apps-dev 2.0.2027-1
doca-grpc 2.0.2027-1
doca-grpc-dev 2.0.2027-1
doca-libs 2.0.2027-1
doca-prime-runtime 2.0.2027-1
doca-prime-sdk 2.0.2027-1
doca-prime-tools 2.0.2027-1
doca-runtime 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
doca-runtime-kernel 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
doca-runtime-user 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
doca-samples 2.0.2027-1
doca-sdk 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
doca-sdk-kernel 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
doca-sdk-user 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
doca-services 2.0.2027-1
doca-tools 2.0.2027-1.23.04.0.5.3.0.bf.4.0.2.12679.10.23
dpa-compiler 1.4.0
dpacc 1.4.0
flexio 23.4.1494
libdoca-libs-dev 2.0.2027-1
librxpcompiler-dev 23.04.1
rxp-compiler 23.04.1
rxpbench 23.04.0
OFED:
dpcp 1.1.39-1.2304053
ibacm 2304mlnx44-1.2304053
ibutils2 2.1.1-0.162.MLNX20230417.g738750f2.2304053
ibverbs-providers:arm64 2304mlnx44-1.2304053
ibverbs-utils 2304mlnx44-1.2304053
infiniband-diags 2304mlnx44-1.2304053
libibmad-dev:arm64 2304mlnx44-1.2304053
libibmad5:arm64 2304mlnx44-1.2304053
libibnetdisc5:arm64 2304mlnx44-1.2304053
libibumad-dev:arm64 2304mlnx44-1.2304053
libibumad3:arm64 2304mlnx44-1.2304053
libibverbs-dev:arm64 2304mlnx44-1.2304053
libibverbs1:arm64 2304mlnx44-1.2304053
libopensm 5.15.0.MLNX20230417.d84ecf64-0.1.2304053
libopensm-devel 5.15.0.MLNX20230417.d84ecf64-0.1.2304053
libopenvswitch:arm64 2.17.7-1.2304053
librdmacm-dev:arm64 2304mlnx44-1.2304053
librdmacm1:arm64 2304mlnx44-1.2304053
libvma 9.8.20-1
libvma-dev 9.8.20-1
libvma-utils 9.8.20-1
libxlio 3.0.2-1.2304053
libxlio-dev 3.0.2-1.2304053
libxlio-utils 3.0.2-1.2304053
mlnx-dpdk 22.11.0-1.4.2.23040530.1.4.2
mlnx-dpdk-dev:arm64 22.11.0-1.4.2.23040530.1.4.2
mlnx-ethtool 6.0-1.2304053
mlnx-iproute2 6.2.0-1.2304053
mlnx-ofed-kernel-utils 23.04-OFED.23.04.0.5.3.1.bf.kver.5.15.0-1015-bluefield
mlnx-tools 23.04-0.2304053
mstflint 4.16.1-2.2304053
opensm 5.15.0.MLNX20230417.d84ecf64-0.1.2304053
openvswitch-common 2.17.7-1.2304053
openvswitch-ipsec 2.17.7-1.2304053
openvswitch-switch 2.17.7-1.2304053
perftest 23.04.0-0.23.g63e250f.2304053
python3-openvswitch 2.17.7-1.2304053
python3-pyverbs:arm64 2304mlnx44-1.2304053
rdma-core 2304mlnx44-1.2304053
rdmacm-utils 2304mlnx44-1.2304053
srptools 2304mlnx44-1.2304053
ucx 1.15.0-1.2304053
tuanama
October 31, 2025, 1:52am
10
ShellonLe:
DOCA: 2.0.2027-1
This is the issue: bifurcation is only available from version 2.5 onwards. check documentation.
You need update firmware
1 Like