Suitable VLLM Container for Jetson Xavier NX with JetPack 5.1.4

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi, I’m new to Jetson.
I have a Jetson Xavier NX device with JetPack 5.1.4 installed via an SD card.
I’m trying to run a VLLM server on it, but I’m not sure which container is compatible with my setup.

Here is the output of cat /etc/nv_tegra_release:

R35 (release), REVISION: 6.1, GCID: 39721438, BOARD: t186ref, EABI: aarch64, DATE: Tue Mar  4 10:13:09 UTC 2025

nvcc --version

  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2022 NVIDIA Corporation
  Built on Sun_Oct_23_22:16:07_PDT_2022
  Cuda compilation tools, release 11.4, V11.4.315
  Build cuda_11.4.r11.4/compiler.31964100_0

Can anyone recommend a compatible VLLM container for this configuration?

Thanks in advance!

3

Hi,

We don’t have a vLLM container for the r35 branch.
But you can follow our Dockerfile to see if you can build on for your environment:

Thanks.

1 Like
4

Thank you for your response.
I’m just trying to understand — the documentation states that it’s possible to upgrade to a recent CUDA version.
If I update CUDA to the latest version,
will I be able to run a standard container with the latest VLLM?
Or are there other components on the host machine that I need to take into account?

5

Hi,

We don’t have too much experience with vLLM on r35.

The latest CUDA for the r35 environment is 12.2.
You can give it a try to see if it can work.

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.