BlueField3 DPU TLP Emulation

Infrastructure & Networking DOCA Infrastructure Management
1

Hello NVIDIA team,

I am trying to run the DOCA 3.5 vnet_pci_dev reference application on a BlueField-3 B3220 P-Series DPU. TLP emulation settings are accepted by mlxconfig and retained in “Next Boot”, but remain disabled in “Current” after reset.

My environment:

  • Device name reported by mlxconfig: 900-9D3B6-00CV-A_Ax
  • PSID: MT_0000000884
  • NIC firmware: 32.50.1002
  • DOCA runtime: 3.5.0098, verified from the loaded system libraries on both DPU and host
  • Running mlx5_core driver: 2607.0.100; MFT: 4.37.0-154
  • DPU: Ubuntu 24.04, kernel 6.8.0-1030-bluefield-64k, DPU mode
  • ATF: 4.16.0-8-gda930bca9; UEFI: 4.16.0-48-g3e1cb60aa0
  • DPU BMC firmware remains on the 25.10 release
  • Commands and capability probes run as root. mlxprivhost q --full reports PRIVILEGED, with all listed restrictions disabled.

I followed the official TLP guide and Virtio-net application guide. For the latest baseline, I backed up the configuration, executed mlxconfig reset, and applied the documented TLP/switch settings plus PF_BAR2_ENABLE=0 PER_PF_NUM_SF=1 PF_SF_BAR_SIZE=8 PF_TOTAL_SF=320.

I then gracefully shut down the DPU Arm OS, confirmed a fresh INFO[BL31]: System Off message through host RShim, and rebooted the host. Both host and DPU boot IDs changed. Physical power removal and restoration had also been tried during earlier troubleshooting.

After the latest reset:

Parameter Current Next Boot
TLP_EMULATION_ENABLE 0 1
TLP_EMULATION_NUM_PF 0 32
PCI_SWITCH_EMULATION_NUM_TLP_PORT 0 1
PCI_SWITCH_EMULATION_ENABLE 1 1
PCI_SWITCH_EMULATION_NUM_PORT 2 2

Other configuration changes did take effect: PF0 SF count/BAR size are now 320/8, and the previous PF1 SF configuration was cleared to 0/0.

Additional read-only diagnostics:

  • Native DOCA TLP capability queries return DOCA_ERROR_NOT_SUPPORTED.
  • General QUERY_HCA_CAP succeeds, but tlp_device_emulation_manager is 0.
  • A direct DevX query of TLP capability group 0x2a returns firmware status 0x3 (BAD_PARAM), syndrome 0x1687ea, on both DPU PFs. This is a capability-query error; I have not identified a firmware startup error explaining the activation failure.

Could you please clarify:

  1. Is TLP emulation supported on this specific PSID with firmware 32.50.1002?
  2. What conditions determine whether tlp_device_emulation_manager becomes enabled, and are there additional configuration or platform requirements?
  3. What does syndrome 0x1687ea mean in this query context, and which diagnostic commands or firmware logs would explain why the saved TLP settings do not become active?

Please advise the appropriate corrective procedure or any applicable known issue/workaround. Thank you.

2

Hello,

Thank you for posting your query on NVIDIA Community.

Based on the information shared, I recommend reviewing the official DOCA device-emulation documentation and validating that the setup is using a documented supported firmware / software combination for BlueField-3.

From the symptoms you described, this looks less like a simple mlxconfig persistence issue and more like the active firmware image not exposing TLP emulation manager capability. In that situation, Next Boot values may be present while the feature still remains disabled in Current after reboot, and the DOCA capability query can continue to report DOCA_ERROR_NOT_SUPPORTED.

For reference, please review:

A few guidance points based on the currently documented setup flow:

  • MT_0000000884 corresponds to the BlueField-3 B3220 P-Series platform, so the base card itself is a valid BlueField-3 platform.

  • For TLP emulation, the relevant prerequisite is not only the NVCONFIG values, but also whether the active firmware image exposes the TLP emulation capability required by the DOCA application.

  • A full cold power cycle is typically required for these settings to take effect; a warm reboot may not be sufficient.

  • The public guides do not document a way to interpret low-level firmware syndrome values such as 0x1687ea, so that level of analysis is generally beyond the scope of forum guidance.

If you continue to see tlp_device_emulation_manager = 0 / DOCA_ERROR_NOT_SUPPORTED after validating the documented configuration and reboot requirements, additional troubleshooting would require an active support entitlement. If you have active support coverage, please open a support ticket through the NVIDIA Enterprise Support Portal.

If support coverage is not currently in place, please reach out to Networking-Contracts@nvidia.com for contract information.

Thanks,
Alena