Triton server died before reaching ready state. Terminating Riva startup

[user5360]

Hardware - GPU:A100
Operating System: Ubuntu 20.04.2 LTS
Riva Version:1.9
I want to run Riva Quick Start Guide, there isnt any error when I run riva_init.sh but when I run ash riva_start.sh appers “Waiting for Riva server to load all models…retrying in 10 seconds” and then it failed.
I already tried the suggestions on this forums:

Jarvis_start.sh times out

I’ve been receiving some similar output. Here is the output of docker logs jarvis-speech: Jarvis waiting for Triton server to load all models…retrying in 1 second I0421 16:34:06.496129 51 onnxruntime.cc:1728] TRITONBACKEND_Initialize: onnxruntime I0421 16:34:06.496415 51 onnxruntime.cc:1738] Triton TRITONBACKEND API version: 1.0 I0421 16:34:06.496424 51 onnxruntime.cc:1744] ‘onnxruntime’ TRITONBACKEND API version: 1.0 Jarvis waiting for Triton server to load all models…retrying in 1 second …

Triton server died before reaching ready state. Terminating Riva startup

Please provide the following information when requesting support. Hardware - GPU (A100) Hardware - CPU Operating System: Ubuntu 20.04 Riva Version: 1.6.0-beta TLT Version Hi I would like to test Riva on the workstation, since there is no internet access on this workstation, here is what I’ve done: Download the docker images to my Linux device via running bash riva_init.sh Save these 3 docker images into tar.gz file Transfer those 3 tar.gz files to the workstation. Load docker images wi…

Triton server died before reaching ready state. Terminating Jarvis startup

Hi, I want to set up the Jarvis server with jarvis_init.sh, but is facing a problem of: Triton server died before reaching ready state. Terminating Jarvis startup. I have tried ignoring this issue and run jarvis_start.sh, but it just loops Waiting for Jarvis server to load all models...retrying in 10 seconds, and ultimately printed out Health ready check failed. Check Jarvis logs with: docker logs jarvis-speech I am not sure what I am doing wrong. Below is the output of docker logs jarvis-s…

In my logs there is this:
I0208 22:43:07.789749 73 server.cc:586]
±------------------------------------------------------------------------------±--------±-------+
| Model | Version | Status |
±------------------------------------------------------------------------------±--------±-------+
| citrinet-1024-es-US-asr-offline-ctc-decoder-cpu-streaming-offline | 1 | READY |
| citrinet-1024-es-US-asr-offline-feature-extractor-streaming-offline | 1 | READY |
| citrinet-1024-es-US-asr-offline-voice-activity-detector-ctc-streaming-offline | 1 | READY |
| citrinet-1024-es-US-asr-streaming | 1 | READY |
| citrinet-1024-es-US-asr-streaming-ctc-decoder-cpu-streaming | 1 | READY |
| citrinet-1024-es-US-asr-streaming-feature-extractor-streaming | 1 | READY |
| citrinet-1024-es-US-asr-streaming-voice-activity-detector-ctc-streaming | 1 | READY |
| riva-trt-citrinet-1024-es-US-asr-streaming-am-streaming | 1 | READY |
±------------------------------------------------------------------------------±--------±-------+
but then appears this:
0208 22:43:07.790076 73 server.cc:234] Waiting for in-flight requests to complete.
I0208 22:43:07.790084 73 model_repository_manager.cc:1078] unloading: riva-trt-citrinet-1024-es-US-asr-streaming-am-streaming:1
I0208 22:43:07.790202 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming-voice-activity-detector-ctc-streaming:1
I0208 22:43:07.790379 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming-feature-extractor-streaming:1
I0208 22:43:07.790497 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming:1
I0208 22:43:07.790684 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-offline-voice-activity-detector-ctc-streaming-offline:1
I0208 22:43:07.790870 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-offline-feature-extractor-streaming-offline:1
I0208 22:43:07.791054 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-offline-ctc-decoder-cpu-streaming-offline:1
I0208 22:43:07.791145 73 vad_library.cc:24] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.791893 73 feature-extractor.cc:406] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.792228 73 feature-extractor.cc:406] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.792306 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming-ctc-decoder-cpu-streaming:1
I0208 22:43:07.792532 73 server.cc:249] Timeout 30: Found 8 live models and 0 in-flight non-inference requests
I0208 22:43:07.792683 73 vad_library.cc:24] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.792757 73 ctc-decoder-library.cc:25] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.793274 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming’ version 1
I0208 22:43:07.794656 73 ctc-decoder-library.cc:25] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.826363 73 vad_library.cc:20] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.826498 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming-voice-activity-detector-ctc-streaming’ version 1
I0208 22:43:07.828030 73 vad_library.cc:20] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.832055 73 feature-extractor.cc:403] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.835083 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-offline-voice-activity-detector-ctc-streaming-offline’ version 1
I0208 22:43:07.839222 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-offline-feature-extractor-streaming-offline’ version 1
I0208 22:43:07.841670 73 feature-extractor.cc:403] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.844036 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming-feature-extractor-streaming’ version 1
I0208 22:43:07.914260 73 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU -17, GPU +0, now: CPU 1849, GPU 5359 (MiB)
I0208 22:43:08.058973 73 model_repository_manager.cc:1195] successfully unloaded ‘riva-trt-citrinet-1024-es-US-asr-streaming-am-streaming’ version 1
I0208 22:43:08.328622 73 ctc-decoder-library.cc:22] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:08.328709 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming-ctc-decoder-cpu-streaming’ version 1
I0208 22:43:08.410028 73 ctc-decoder-library.cc:22] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:08.446365 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-offline-ctc-decoder-cpu-streaming-offline’ version 1
I0208 22:43:08.792667 73 server.cc:249] Timeout 29: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models

Riva waiting for Triton server to load all models…retrying in 1 second
Riva waiting for Triton server to load all models…retrying in 1 second
Triton server died before reaching ready state. Terminating Riva startup.
Check Triton logs with: docker logs
/opt/riva/bin/start-riva: line 1: kill: (73) - No such process

[sjunkin]

Please share the full logs, there is usually a line beginning with EXXXX that will outline why the models are unloading and the server is shutting down.

[claudia.garmendia]

Hi! Happens the same here. I am trying to start RIVA server and once Triton have successfully loaded the models and the model status is Ready I find the following logs:

‘’

I1018 10:55:24.311255 102 server.cc:264] Waiting for in-flight requests to complete.
I1018 10:55:24.311268 102 server.cc:280] Timeout 30: Found 0 model versions that have in-flight inferences
I1018 10:55:24.311548 102 server.cc:295] All models are stopped, unloading models
I1018 10:55:24.311563 102 server.cc:302] Timeout 30: Found 3 live models and 0 in-flight non-inference requests
I1018 10:55:24.312150 102 feature-extractor.cc:415] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I1018 10:55:24.312257 102 endpointing_library.cc:28] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I1018 10:55:24.312331 102 ctc-decoder-library.cc:27] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I1018 10:55:24.321672 102 endpointing_library.cc:23] TRITONBACKEND_ModelFinalize: delete model state
I1018 10:55:24.322528 102 model_lifecycle.cc:578] successfully unloaded ‘conformer-en-US-asr-offline-endpointing-streaming-offline’ version 1
I1018 10:55:24.335075 102 feature-extractor.cc:411] TRITONBACKEND_ModelFinalize: delete model state
I1018 10:55:24.335562 102 model_lifecycle.cc:578] successfully unloaded ‘conformer-en-US-asr-offline-feature-extractor-streaming-offline’ version 1
I1018 10:55:24.892730 102 ctc-decoder-library.cc:24] TRITONBACKEND_ModelFinalize: delete model state
I1018 10:55:24.893293 102 model_lifecycle.cc:578] successfully unloaded ‘conformer-en-US-asr-offline-ctc-decoder-cpu-streaming-offline’ version 1
Riva waiting for Triton server to load all models…retrying in 1 second
I1018 10:55:25.311761 102 server.cc:302] Timeout 29: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models
Riva waiting for Triton server to load all models…retrying in 1 second
Riva waiting for Triton server to load all models…retrying in 1 second
Triton server died before reaching ready state. Terminating Riva startup.
Check Triton logs with: docker logs
/opt/riva/bin/start-riva: line 1: kill: (102) - No such process
‘’

Someone help please?