“Waiting for Riva server to load all Models” NVIDIA SPARK DGX

AI & Data Science Deep Learning (Training & Inference) Riva
1

Trying to RIVA TTS on a new NVIDIA DGX SPARK and it fails.

Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Health ready check failed.

In the logs i can see the following:

Riva waiting for Triton server to load all models…retrying in 1 second
Triton server died before reaching ready state. Terminating Riva startup.
Check Triton logs with: docker logs
/opt/riva/bin/start-riva: line 1: kill: (140) - No such process

Any help would be fantastic

4

please share which RIVA NGC container image you are using? docs commands attempted.

or if you are following a playbook, please share which one.

5

I’m using the offline / on-prem NVIDIA Riva 2.19.0 quickstart.

Container image:

nvcr.io/nvidia/riva/riva-speech:2.19.0
6

Yes, I’m getting the same error on my Spark server

7

Any ideas on this?
Its pretty annoying that it doesnt work on your own hardware

8

engineering working on this. expect an update on RIVA workload soon.