I bought a Jetson AGX Orin Development Kit recently and wanted to run some speech AI on it.
I followed the official Quick Start Guide [cannot put the link due to being a new user].
All steps went OK, until the “bas riva_start.sh” step. I got the following output on screen after issuing the command.
~/riva_quickstart_arm64_v2.9.0$ bash riva_start.sh
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Health ready check failed.
Check Riva logs with: docker logs riva-speech
The output of the “docker logs riva-speech” command is attached here (logs.txt).
logs.txt (82.2 KB)
Hope to get help here. Many thanks!
1 Like
Hi @yanchaoliu
Thanks for your interest in Riva
I will check regarding this error with the Riva Team, in meantime one quick suggestion
Can you try the setup again with a different port (i.e default port 50051), from logs i find you are using 8001 port,
Kindly also share the config.sh used
Thanks
HI @yanchaoliu
I have updates from Riva Team
The Team analyzed the logs, The errors are due to unrecognized symbols from the TRT library, which happens due to TRT version mismatch b/w container and host, e.g.
unable to load shared library: /usr/lib/aarch64-linux-gnu/libnvinfer.so.8: undefined symbol: _ZN5nvdla8IProfile17setUseSoftMaxOptzEb
They told me to verify
- which Jetpack version has been installed on the Jetson Orin ?
- whether all Jetpack components installed successfully?
we strongly beleive it may be jetson related
For Riva 2.9.0 (which is your version I believe), Jetpack 5.0.2 should be used.
If you are using latest Jetpack 5.1, Please switch to Riva 2.10.0 release.
Please check and provide us the feedback
Thanks
I’m using nvidia agx orin
bash ./riva_init.sh
Logging into NGC docker registry if necessary…
Pulling required docker images if necessary…
Note: This may take some time, depending on the speed of your Internet connection.
Pulling Riva Speech Server images.
Pulling nvcr.io/nvidia/riva/riva-speech:2.11.0. This may take some time…
Downloading models (RMIRs) from NGC…
Note: this may take some time, depending on the speed of your Internet connection.
To skip this process and use existing RMIRs set the location and corresponding flag in config.sh.
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as ‘csv’
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime instead.: unknown.
Error in downloading models.
bash ./riva_start.sh
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Health ready check failed.
Check Riva logs with: docker logs riva-speech
config.sh file
GPU family of target platform. Supported values: tegra, non-tegra
riva_target_gpu_family=“non-tegra”
Name of tegra platform that is being used. Supported tegra platforms: orin, xavier
riva_tegra_platform=“orin”
Enable or Disable Riva Services
service_enabled_asr=true
service_enabled_nlp=true
service_enabled_tts=true
service_enabled_nmt=true
language_code=(“en-US”)
asr_acoustic_model=(“conformer”)
gpus_to_use=“device=0”
MODEL_DEPLOY_KEY=“tlt_encode”
riva_model_loc=“riva-model-repo”
if [[ $riva_target_gpu_family == “tegra” ]]; then
riva_model_loc=“pwd
/model_repository”
fi
use_existing_rmirs=false
Ports to expose for Riva services
riva_speech_api_port=“50051”
NGC orgs
riva_ngc_org=“nvidia”
riva_ngc_team=“riva”
riva_ngc_image_version=“2.11.0”
riva_ngc_model_version=“2.11.0”
help me sort out my problem
Also it’s taking a lot more time to pull models while executing riva_init.sh file
Hi @ritthyk5050
Thanks for your interest in Riva
Apologies for the delay
-
Can you kindly share the Jetpack version used
Riva 2.11 required JetPack 5.1 or JetPack 5.1.1
-
kindly set the default runtime to nvidia
on the Jetson platform if no already done
You have set the default runtime to nvidia
on the Jetson platform by adding the following line in the /etc/docker/daemon.json
file. Restart the Docker service using sudo systemctl restart docker
after editing the file.
"default-runtime": "nvidia"
Please check the above and let us know if it helps
Thanks
Jetson Model: AGX Orin
Jetpack Version: 5.1 (L4T 35.2.1)
Riva Version: 2.11.0
I am having the same problem as OP. Nearly identical config. I have also done the "default-runtime": "nvidia"
sugeestion. Running bash riva_start.sh
produces same result. Runs until finally quitting and saying:
...
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Health ready check failed.
Check Riva logs with: docker logs riva-speech
I have included my config.sh and the output of docker logs riva-speech
in the log.txt file.
I am only trying to run the nlp models to test a simple sample script but can not get the Riva Service to work. Is Riva 2.10.0 more stable?
config.sh (13.3 KB)
log.txt (13.0 KB)
Hi @hpetty_c2i
Thanks for your interest in Riva
Can you run the following command and share with us the output
docker info | grep -i runtime
Also can you check if you are able to run any basic container with nvidia-runtime
2. docker run -it --rm --runtime nvidia ubuntu:20.04
Thanks
Runtimes: io.containerd.runtime.v1.linux nvidia runc io.containerd.runc.v2
Default Runtime: nvidia
And yes I am able to run the container