Jetson Thor Official Container for vLLM 0.16 fails to load nemotron-3-super -- says mixed-precision quant config is unsupported in vLLM 0.16 container

Robotics & Edge Computing Jetson Systems Jetson Thor
,
1

The official Thor vLLM container ( Package vllm · GitHub ) fails to load nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 due to the mixed-precision quant config format requiring vLLM 0.17.1+. Nano 4B and Nano 30B work fine. Full details and reproduction steps here: https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4/discussions/20

5

Hi,

This seems to be the same issue as you reported in the comment below:

Have you tried the steps shared in that topic?
Thanks.

6

Don’t know that it would absolutely work, but the NGC docker image nvcr.io/nvidia/vllm:26.03-py3 has vLLM Version 0.17.1+a03ca76a