Error when trying to launch the nvidia:jetson-voice container on Xavier NX

Hi

After downloading the container and tried to launch the Triton Container, I got the below error.

Appreciate any help

sudo docker run --runtime nvidia -it --rm --network host nvcr.io/nvidia/jetson-voice:r32.4.2 trtserver --model-control-mode=none --model-repository=models/repository/jasper-asr-streaming-vad/

I0426 20:21:28.171917 1 server.cc:115] Initializing TensorRT Inference Server
I0426 20:21:28.305807 1 server_status.cc:55] New status tracking for model ‘ctc-decoder-cpu-trt-vad-streaming’
I0426 20:21:28.305923 1 server_status.cc:55] New status tracking for model ‘feature-extractor-trt-vad-streaming’
I0426 20:21:28.306043 1 server_status.cc:55] New status tracking for model ‘jasper-asr-trt-ensemble-vad-streaming’
I0426 20:21:28.306162 1 server_status.cc:55] New status tracking for model ‘jasper-trt-decoder-streaming’
I0426 20:21:28.306209 1 server_status.cc:55] New status tracking for model ‘jasper-trt-encoder-streaming’
I0426 20:21:28.306243 1 server_status.cc:55] New status tracking for model ‘voice-activity-detector-trt-ctc-streaming’
I0426 20:21:28.306540 1 model_repository_manager.cc:675] loading: ctc-decoder-cpu-trt-vad-streaming:1
I0426 20:21:28.307001 1 model_repository_manager.cc:675] loading: feature-extractor-trt-vad-streaming:1
I0426 20:21:28.307397 1 model_repository_manager.cc:675] loading: jasper-trt-decoder-streaming:1
I0426 20:21:28.307635 1 custom_backend.cc:202] Creating instance ctc-decoder-cpu-trt-vad-streaming_0_0_cpu on CPU using libctcdecoder-cpu.so
I0426 20:21:28.307793 1 model_repository_manager.cc:675] loading: jasper-trt-encoder-streaming:1
I0426 20:21:28.307919 1 custom_backend.cc:205] Creating instance feature-extractor-trt-vad-streaming_0_gpu0 on GPU 0 (7.2) using libfeature-extractor.so
I0426 20:21:28.308569 1 model_repository_manager.cc:675] loading: voice-activity-detector-trt-ctc-streaming:1
I0426 20:21:28.310062 1 custom_backend.cc:205] Creating instance voice-activity-detector-trt-ctc-streaming_0_gpu0 on GPU 0 (7.2) using libvoice-activity-detector.so
I0426 20:21:28.324682 1 model_repository_manager.cc:829] successfully loaded ‘voice-activity-detector-trt-ctc-streaming’ version 1
E0426 20:21:31.030114 1 logging.cc:43] coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
E0426 20:21:31.030664 1 logging.cc:43] INVALID_STATE: std::exception
E0426 20:21:31.030722 1 logging.cc:43] INVALID_CONFIG: Deserialize the cuda engine failed.
E0426 20:21:31.030885 1 model_repository_manager.cc:832] failed to load ‘jasper-trt-decoder-streaming’ version 1: Internal: unable to create TensorRT engine
E0426 20:21:31.031197 1 logging.cc:43] coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
E0426 20:21:31.031350 1 logging.cc:43] INVALID_STATE: std::exception
E0426 20:21:31.031389 1 logging.cc:43] INVALID_CONFIG: Deserialize the cuda engine failed.
E0426 20:21:31.035361 1 model_repository_manager.cc:832] failed to load ‘jasper-trt-encoder-streaming’ version 1: Internal: unable to create TensorRT engine
I0426 20:21:33.874237 1 model_repository_manager.cc:829] successfully loaded ‘feature-extractor-trt-vad-streaming’ version 1
I0426 20:21:39.610524 1 model_repository_manager.cc:829] successfully loaded ‘ctc-decoder-cpu-trt-vad-streaming’ version 1
E0426 20:21:39.610858 1 model_repository_manager.cc:1087] Invalid argument: ensemble ‘jasper-asr-trt-ensemble-vad-streaming’ depends on ‘jasper-trt-decoder-streaming’ which has no loaded version
I0426 20:21:39.612572 1 model_repository_manager.cc:808] successfully unloaded ‘voice-activity-detector-trt-ctc-streaming’ version 1
I0426 20:21:39.779017 1 model_repository_manager.cc:808] successfully unloaded ‘feature-extractor-trt-vad-streaming’ version 1
I0426 20:21:39.861104 1 model_repository_manager.cc:808] successfully unloaded ‘ctc-decoder-cpu-trt-vad-streaming’ version 1
error: creating server: INTERNAL - failed to load all models

It seems to me that there is an issue with the TensorRT version,

I launched the below command:

dpkg -l | grep TensorRT

and got:

ii graphsurgeon-tf 7.1.3-1+cuda10.2 arm64 GraphSurgeon for TensorRT package
ii libnvinfer-bin 7.1.3-1+cuda10.2 arm64 TensorRT binaries
ii libnvinfer-dev 7.1.3-1+cuda10.2 arm64 TensorRT development libraries and headers
ii libnvinfer-doc 7.1.3-1+cuda10.2 all TensorRT documentation
ii libnvinfer-plugin-dev 7.1.3-1+cuda10.2 arm64 TensorRT plugin libraries
ii libnvinfer-plugin7 7.1.3-1+cuda10.2 arm64 TensorRT plugin libraries
ii libnvinfer-samples 7.1.3-1+cuda10.2 all TensorRT samples
ii libnvinfer7 7.1.3-1+cuda10.2 arm64 TensorRT runtime libraries
ii libnvonnxparsers-dev 7.1.3-1+cuda10.2 arm64 TensorRT ONNX libraries
ii libnvonnxparsers7 7.1.3-1+cuda10.2 arm64 TensorRT ONNX libraries
ii libnvparsers-dev 7.1.3-1+cuda10.2 arm64 TensorRT parsers libraries
ii libnvparsers7 7.1.3-1+cuda10.2 arm64 TensorRT parsers libraries
ii nvidia-container-csv-tensorrt 7.1.3.0-1+cuda10.2 arm64 Jetpack TensorRT CSV file
ii python-libnvinfer 7.1.3-1+cuda10.2 arm64 Python bindings for TensorRT
ii python-libnvinfer-dev 7.1.3-1+cuda10.2 arm64 Python development package for TensorRT
ii python3-libnvinfer 7.1.3-1+cuda10.2 arm64 Python 3 bindings for TensorRT
ii python3-libnvinfer-dev 7.1.3-1+cuda10.2 arm64 Python 3 development package for TensorRT
ii tensorrt 7.1.3.0-1+cuda10.2 arm64 Meta package of TensorRT
ii uff-converter-tf 7.1.3-1+cuda10.2 arm64 UFF converter for TensorRT package

Hi @amehrez, this container was built for JetPack 4.4 DP. Are you using this SD card image to run it from?

https://developer.nvidia.com/jetson-nx-developer-kit-sd-card-image-44-dp

Hi @dusty_nv

Thanks for your reply.

I’m using Jetpack 4.4.1 [L4T 32.4.4]

Regards,