Instead of getting 100Gb/s between the two I get just over 13Gb/s… but the latency test looks fine. It’s suspiciously 1/8 of what it should be. It’s most likely some silly configuration errors.
This is on two freshly configured Sparks with the latest firmware upgrades.
Found the root cause: it’s the latest 6.17 kernel that’s causing the regression. Reverting back to 6.11 kernel shows 100Gb/s as expected. Some configurations are broken with 6.17