Jetson Agx thor

[AastaLLL]

Hi,

Please find our vLLM container for Thor below:

GPU-optimized AI, Machine Learning, & HPC Software | NVIDIA NGC

Below is an example to run gpt-oss-20b for your reference:

Issue with run gpt-oss-120b in vLLM

Hi, Thanks for your patience. Below are the steps to run gpt-oss-20b with our new vLLM container. (Test with 20b model, but 120b is expected to work as well) For gpt-oss model, you will need the WAR mentioned in this link on Harmony encoding. $ sudo docker run -it --rm nvcr.io/nvidia/vllm:25.09-py3 # mkdir /etc/encodings # wget https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken -O /etc/encodings/cl100k_base.tiktoken # wget https://openaipublic.blob.core.windows.net/e…

Thanks.