How to run latest Supported Triton Inference Server 23.07 on docker

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hi, i have jetson xavier agx 16gb. I have flashed my device (JP 5.1.4, latest as of today).

When i looked support matrix(matrix) for xavier agx latest supported Triton version is

Release 2.36.0 corresponding to NGC container 23.07

image

I have updated my cuda version to latest supported cuda 12.2 successfully.
but when i run docker compose with
image: nvcr.io/nvidia/tritonserver:23.07-py3

i get

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'csv'
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime instead.: unknown
3

Hi,

Please find more info in the comment below:

But please use the default CUDA 11.4 version instead of 12.2.
Thanks.

4

i was failing all the time using desktop version dockers. I used dusty jetson containers version 35.4.1
now it is working but will upgrading cuda make any benefit to me i was wondering. Also thanks for your reply.

5

Hi,

For JetPack 5, only CUDA is upgradable and no corresponding cuDNN/TensorRT/PyTorch/TensorFlow packages are available.

Since the Triton server depends on these backends, it’s recommended to stay on the default CUDA 11.

Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.