Hardware
GPU (Orin NX Developer Kit)
Module: Nvidia Jetson Orin NX (16GB ram)
Jetpack: 5.1
L4T: 35.2.1
Operating System: Linux
Distribution: Ubuntu 20.04 Focal Fossa
Release: 5.10.104-tegra
Python: 3.8.10
Libraries
CUDA: 11.4.315
cuDNN: 8.6.0.166
TensorRT: 5.1
VPI: 2.2.4
Vulkan: 1.3.204
OpenCV: 4.5.4 with CUDA: NO
Riva Version:
CLI: 3.35.0
Quickstart: “nvidia/riva/riva_quickstart_arm64:2.14.0”
Model registry: “nvidia/riva/speechtotext_en_us_lm:deployable_v6.0”
Models downloaded from NGC using riva_init.sh
- Folder Name: model_repository (in locked mode)
- models_asr_conformer_en_us_str_v2.14.0-tegra-orin
- models_nlp_punctuation_bert_base_en_us_v2.14.0-tegra-orin
When i try to start riva using the command
sudo bash /home/…/RIVA/riva_quickstart_arm64_v2.3.0/riva_start.sh
I get the following error message and fails to start the Riva server
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Health ready check failed.
Check Riva logs with: docker logs riva-speech
Need help on how to resolve the above error message and successfully have the Riva server started.
Docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nvcr.io/nvidia/riva/riva-speech 2.14.0-servicemaker a99cca57c4c2 9 days ago 10.2GB
nvcr.io/nvidia/riva/riva-speech 2.14.0-l4t-aarch64 0f23cffd4704 9 days ago 12.6GB
nvcr.io/nvidia/riva/riva-speech 2.14.0 eb3ba05fc193 9 days ago 9.65GB
arm64v8/python slim ac234f0af83b 2 weeks ago 156MB
nvcr.io/nvidia/riva/riva-speech 2.3.0-arm64-server a609bc9fdc59 18 months ago 11.9GB
List all Containers
docker container ls --all
provides the following output
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7ddf0d978a61 nvcr.io/nvidia/riva/riva-speech:2.14.0-l4t-aarch64 “start-riva --riva-u…” 12 minutes ago Created 0.0.0.0:8000-8002->8000-8002/tcp, :::8000-8002->8000-8002/tcp, 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp, 0.0.0.0:50051->50051/tcp, :::50051->50051/tcp riva-speech
Install steps followed (so far) - could help to recreate this issue
-
Got the NGC api key
-
Downloaded the CLI from: NVIDIA NGC
-
Issued & verified OK on:
find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5
Matched binary hash using: sha256sum ngccli_arm64.zip
- Issued the following
chmod u+x ngc-cli/ngc
echo “export PATH="$PATH:$(pwd)/ngc-cli"” >> ~/.bash_profile && source ~/.bash_profile
ngc config set
Outputs the following
Validating configuration... Successfully validated configuration. Saving configuration... Successfully saved NGC configuration to /home/akiso/.ngc/config
- Downloaded Riva quick start
ngc registry resource download-version “nvidia/riva/riva_quickstart_arm64:2.14.0”
- Make changes to config.sh in the folder riva_quickstart_v2.14.0 as follows
service_enabled_asr=true
service_enabled_nlp=false
service_enabled_tts=false
gpus_to_use=“device=1”
- Downloaded the pretrained model
ngc registry model download-version “nvidia/riva/speechtotext_en_us_lm:deployable_v6.0”
- Changes to /etc/docker/daemon.json
Blanked out
OR
{
“default-runtime”: “nvidia”,
“runtimes”: {
“nvidia”: {
“path”: “/usr/bin/nvidia-container-runtime”,
“runtimeArgs”:
}
}
}
- Clean Riva server
sudo bash /home/…/RIVA/riva_quickstart_arm64_v2.3.0/riva_clean.sh
- Initialize Riva server
sudo bash /home/…/RIVA/riva_quickstart_arm64_v2.3.0/riva_init.sh
(Downloaded the model_repository) and 2 models
- models_asr_conformer_en_us_str_v2.14.0-tegra-orin
- models_nlp_punctuation_bert_base_en_us_v2.14.0-tegra-orin
Completed Riva server initialization
echo ‘Riva initialization complete. Run ./riva_start.sh to launch services.’
Riva initialization complete. Run ./riva_start.sh to launch services.
- Start Riva server
sudo bash /home/…/RIVA/riva_quickstart_arm64_v2.3.0/riva_start.sh
the start fails with the error listed above
Also attempted the recommendation in the link, it did not work for this setup
Docker logs files are attached
riva-speech:2.3.0-arm64-server_docker.log (12.8 KB)
riva-speech:2.14.0_docker.log (14.0 KB)
riva-speech:2.14.0-l4t-aarch64_docker.log (10.6 KB)
riva-speech:2.14.0-servicemaker_docker.log (17.0 KB)