Jetson Orin Nano development kit with Triton Inference Server GPU problem

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi all,

I am trying to run a YOLOv8 ONNX model on my Jetson Orin Nano development kit using a Triton inference server based on the code provided in [https://github.com/omarabid59/yolov8-triton/](https://forums.developer.nvidia.com/[https://github.com/omarabid59/yolov8-triton/](https://github.com/omarabid59/yolov8-triton/))

However, I cannot run the model only on CPU and not on the GPU since I get following error message:

ERROR: This container was built for NVIDIA Driver Release 560.28 or later, but

version 540.4.0 was detected and compatibility mode is UNAVAILABLE.

[[]]

Can somebody help me to solve this error? I started the Triton inference server by first creating a docker file with:

FROM nvcr.io/nvidia/tritonserver:24.08-py3-igpu

RUN pip install opencv-python && \

apt update && \

apt install -y libgl1 && \

rm -rf /var/lib/apt/lists/*

CMD [“tritonserver”, “–model-repository=/models” ]

Next, I build the container with

DOCKER_NAME=“yolov8-triton”

docker build -t $DOCKER_NAME .

And finally I started the Triton inference container with (which raises the error above):

DOCKER_NAME=“yolov8-triton”

docker run -it --rm \

-p 8000:8000 \

-p 8001:8001 \

-p 8002:8002 \

-v ./models:/models \

$DOCKER_NAME

I have already tried to downgrade to an earlier triton server package (FROM nvcr.io/nvidia/tritonserver:23.02-py3-igpu) which should be compatible with driver version 540.4.0 but unfortunately without any succes.

Many thanks for the help!

Lode

7

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

8

You can try this:

9

Sorry for late reply! Is this still an DeepStream issue to support? Thanks! what is the jetpack’s version? It seems the tritonsever version conflicts with jetpack. You can try other tritonserver versions, like tritonserver:25.01 mentioned above.