ConnectX-4 LX: mlxconfig nvconfig parameters not applied after FW 14.32.1908/1912 --only mlxfwreset works

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Hi everyone,

We’re experiencing a systematic issue where nvconfig parameters set via mlxconfig are written to NVRAM (visible in “Next Boot” column) but are not applied to “Current” values after a reboot or a full power cycle on ConnectX-4 LX cards flashed to FW 14.32.1908 / 14.32.1912. This reproduces on every host we’ve upgraded – it’s not specific to a single server, platform, or card. The issue is not present on older firmware versions – parameters applied correctly before the upgrade on all the same hosts
In our infrastructure, the parameters we primarily depend on are SRIOV_EN and NUM_OF_VFS

Running mlxfwreset after mlxconfig set does apply the parameters – “Current” matches “Next Boot” afterward. This is consistent across all hosts. This suggests the firmware CAN read the this parameters from NVRAM but only does so on a PCI-level reset (PERST#), not on a normal reboot or power cycle

Questions:

  1. Is this expected behavior for FW 14.32.1908/1912 – i.e., are these parameters now only applied on PCI-level reset rather than on reboot/power cycle? If so, this is a regression from all previous FW versions
  2. Was there a change in the NVRAM-to-runtime config initialization flow between older FW and 14.32.1908 that now requires mlxfwreset instead of reboot/power cycle?
  3. If mlxfwreset is now the only way to apply nvconfig, should mlxconfig emit a warning recommending mlxfwreset instead of “Please power cycle machine to load new configurations”?
  4. If this is not expected behavior, is there a known workaround to restore the old behavior where nvconfig parameters are applied on normal reboot/power cycle? For example:
    • A specific mlxconfig flag or sequence we’re missing?
    • A firmware config option that controls the NVRAM-to-runtime initialization path?
    • A way to force the firmware to re-read NVRAM during normal PCIe enumeration without mlxfwreset?
2

Update: the controlling variable is the MFT version, not the firmware

We bisected this on a single host with the firmware left completely untouched, and the behaviour tracks the MFT version exactly.

Test host: ConnectX-4 Lx, MCX4421A-ACA_Bx, FW 14.32.1908, unchanged across all runs. Ubuntu 20.04, kernel 6.6.142. One version per run, same procedure each time:

mlxconfig -y -d set NUM_OF_VFS=
reboot
mlxconfig -e -d q | grep NUM_OF_VFS

│ MFT │ Branch │ Build / release │ Hint printed │ Applied after reboot
│ 4.30.1-113 │ 2024 LTS │ built 2025-02-18 │ Please reboot machine │ yes
│ 4.30.1-1219 │ 2024 LTS │ built 2026-05-18 │ Please reboot machine │ yes
│ 4.32.0-120 │ mainline │ built 2025-04-30 │ Please power cycle machine │ no
│ 4.33.0-169 │ mainline │ built 2025-07-24 │ Please power cycle machine │ no
│ 4.34.1-10 │ 2025 LTS │ released 2025-12-15 │ — │ no
│ 4.34.1-18 │ 2025 LTS │ released 2026-03-23 │ Please power cycle machine │ no (full power cycle does not help either) │

Points we think matter:

  1. The firmware was identical in all six runs. Only MFT changed. In our original rollout the FW upgrade and the MFT upgrade (4.26.1-6 → 4.34.1-10, pulled in by DOCA 3.2.1) happened in the same operation, which is why we initially attributed this to the firmware. Please disregard the firmware framing in the first post. We have not tested new MFT against older firmware, so we cannot rule out firmware as a co-factor — but it is not the variable that controls the outcome.
  2. This is not an “old versus new” problem. 4.30.1-1219 is built 2026-05-18, newer than every failing build including 4.34.1-18. The 2024 LTS branch works at every patch level we tested; mainline fails from 4.32.0 onward and the 2025 LTS branch inherits the failure. Whatever changed in mainline was evidently never backported to 4.30.1.
  3. The failing versions change the hint printed after Applying… Done! from Please reboot machine to load new configurations to Please power cycle machine to load new configurations. This predicted the outcome in all six runs. The hint is also incorrect on its own terms: a full chassis power cycle does not apply the configuration on this device either.
  4. 4.34.1-18 does not help, although it carries the fix for “MFT was missing the Adabe changes for the new nvconfig” (internal ref 4874697/4857974, discovered in 4.34.0-145, shipped in DOCA 3.2.2).
  5. mlxfwreset still applies the pending value, and only that one – every subsequent change needs it again, so it is not a repair.

Revised questions:

  1. What changed in the nvconfig write path in mainline MFT between 4.30.1 and 4.32.0 that ConnectX-4 Lx firmware no longer picks up during normal boot?
  2. Is this a known regression, and is a fix planned for the 2025 LTS branch (4.34.1-x)?