Feasibility of 2-3x Speedup via Speculative Decoding on High-Compute (1000 TFLOPS) / Low-BW Hardware

[raphael.amorim]

Lab to support speculative decoding on the spark

Speculative Decoding | DGX Spark

Thread with lots of performance numbers for inference in single and cluster mode:

ConnectX-7 NIC in DGX Spark

I’d like to ask about the port configuration of the ConnectX-7 NIC in DGX Spark. On our DGX Spark, the onboard ConnectX-7 NIC appears as four ports when checking with ip -br a and lspci: roceP2p1s0f0 port 1 ==> enP2p1s0f0np0 (Down) roceP2p1s0f1 port 1 ==> enP2p1s0f1np1 (Up) rocep1s0f0 port 1 ==> enp1s0f0np0 (Down) rocep1s0f1 port 1 ==> enp1s0f1np1 (Up) However, the chassis only has two physical ports. If anyone is familiar with this behavior, could you help explain how the NIC …