Triton server died before reaching ready state. Terminating Riva startup

Hardware - GPU:A100
Operating System: Ubuntu 20.04.2 LTS
Riva Version:1.9
I want to run Riva Quick Start Guide, there isnt any error when I run riva_init.sh but when I run ash riva_start.sh appers “Waiting for Riva server to load all models…retrying in 10 seconds” and then it failed.
I already tried the suggestions on this forums:

In my logs there is this:
I0208 22:43:07.789749 73 server.cc:586]
±------------------------------------------------------------------------------±--------±-------+
| Model | Version | Status |
±------------------------------------------------------------------------------±--------±-------+
| citrinet-1024-es-US-asr-offline-ctc-decoder-cpu-streaming-offline | 1 | READY |
| citrinet-1024-es-US-asr-offline-feature-extractor-streaming-offline | 1 | READY |
| citrinet-1024-es-US-asr-offline-voice-activity-detector-ctc-streaming-offline | 1 | READY |
| citrinet-1024-es-US-asr-streaming | 1 | READY |
| citrinet-1024-es-US-asr-streaming-ctc-decoder-cpu-streaming | 1 | READY |
| citrinet-1024-es-US-asr-streaming-feature-extractor-streaming | 1 | READY |
| citrinet-1024-es-US-asr-streaming-voice-activity-detector-ctc-streaming | 1 | READY |
| riva-trt-citrinet-1024-es-US-asr-streaming-am-streaming | 1 | READY |
±------------------------------------------------------------------------------±--------±-------+
but then appears this:
0208 22:43:07.790076 73 server.cc:234] Waiting for in-flight requests to complete.
I0208 22:43:07.790084 73 model_repository_manager.cc:1078] unloading: riva-trt-citrinet-1024-es-US-asr-streaming-am-streaming:1
I0208 22:43:07.790202 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming-voice-activity-detector-ctc-streaming:1
I0208 22:43:07.790379 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming-feature-extractor-streaming:1
I0208 22:43:07.790497 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming:1
I0208 22:43:07.790684 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-offline-voice-activity-detector-ctc-streaming-offline:1
I0208 22:43:07.790870 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-offline-feature-extractor-streaming-offline:1
I0208 22:43:07.791054 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-offline-ctc-decoder-cpu-streaming-offline:1
I0208 22:43:07.791145 73 vad_library.cc:24] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.791893 73 feature-extractor.cc:406] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.792228 73 feature-extractor.cc:406] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.792306 73 model_repository_manager.cc:1078] unloading: citrinet-1024-es-US-asr-streaming-ctc-decoder-cpu-streaming:1
I0208 22:43:07.792532 73 server.cc:249] Timeout 30: Found 8 live models and 0 in-flight non-inference requests
I0208 22:43:07.792683 73 vad_library.cc:24] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.792757 73 ctc-decoder-library.cc:25] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.793274 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming’ version 1
I0208 22:43:07.794656 73 ctc-decoder-library.cc:25] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0208 22:43:07.826363 73 vad_library.cc:20] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.826498 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming-voice-activity-detector-ctc-streaming’ version 1
I0208 22:43:07.828030 73 vad_library.cc:20] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.832055 73 feature-extractor.cc:403] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.835083 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-offline-voice-activity-detector-ctc-streaming-offline’ version 1
I0208 22:43:07.839222 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-offline-feature-extractor-streaming-offline’ version 1
I0208 22:43:07.841670 73 feature-extractor.cc:403] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:07.844036 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming-feature-extractor-streaming’ version 1
I0208 22:43:07.914260 73 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU -17, GPU +0, now: CPU 1849, GPU 5359 (MiB)
I0208 22:43:08.058973 73 model_repository_manager.cc:1195] successfully unloaded ‘riva-trt-citrinet-1024-es-US-asr-streaming-am-streaming’ version 1
I0208 22:43:08.328622 73 ctc-decoder-library.cc:22] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:08.328709 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-streaming-ctc-decoder-cpu-streaming’ version 1
I0208 22:43:08.410028 73 ctc-decoder-library.cc:22] TRITONBACKEND_ModelFinalize: delete model state
I0208 22:43:08.446365 73 model_repository_manager.cc:1195] successfully unloaded ‘citrinet-1024-es-US-asr-offline-ctc-decoder-cpu-streaming-offline’ version 1
I0208 22:43:08.792667 73 server.cc:249] Timeout 29: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models

Riva waiting for Triton server to load all models…retrying in 1 second
Riva waiting for Triton server to load all models…retrying in 1 second
Triton server died before reaching ready state. Terminating Riva startup.
Check Triton logs with: docker logs
/opt/riva/bin/start-riva: line 1: kill: (73) - No such process

1 Like

Please share the full logs, there is usually a line beginning with EXXXX that will outline why the models are unloading and the server is shutting down.