Riva Quickstart 2.4.0 installation fails on AGX Orin

user108910 · August 26, 2022, 3:42pm

Please provide the following information when requesting support.

Hardware - AGX Orin
Hardware - CPU
Operating System Ubuntu 20.04 / JetPack 5.0.2
Riva Version 2.2.4
Docker version 2.10.17
TLT Version (if relevant)
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)

I try to install the riva_quickstart_arm64_v2.4.0 following the instructions from the Quick Start Guide. The script riva_init.sh initially failed:

markus@markus-orin:/mnt/SSD1000/riva_quickstart_arm64_v2.4.0$ bash riva_init.sh
Logging into NGC docker registry if necessary...
Pulling required docker images if necessary...
Note: This may take some time, depending on the speed of your Internet connection.
> Pulling Riva Speech Server images.
  > Pulling nvcr.io/nvidia/riva/riva-speech:2.4.0-l4t-aarch64-server. This may take some time...
  > Attempt 1 out of 3 failed
  > Trying again...
  > Attempt 2 out of 3 failed
  > Trying again...
  > Attempt 3 out of 3 failed
Error occurred pulling 'nvcr.io/nvidia/riva/riva-speech:2.4.0-l4t-aarch64-server'.
Error response from daemon: manifest for nvcr.io/nvidia/riva/riva-speech:2.4.0-l4t-aarch64-server not found: manifest unknown: manifest unknown

After checking the tags on Riva Speech Skills | NVIDIA NGC and changing line 226 in config.sh

Line 226 (original): image_speech_api="nvcr.io/${NGC_TARGET}/riva-speech:${riva_ngc_image_version}-l4t-aarch64-server"

Line 226 (changed): image_speech_api="nvcr.io/${NGC_TARGET}/riva-speech:${riva_ngc_image_version}-server-l4t-aarch64"

the script riva_init.sh completes successfully:

markus@markus-orin:/mnt/SSD1000/riva_quickstart_arm64_v2.4.0$ bash riva_init.sh
Logging into NGC docker registry if necessary...
Pulling required docker images if necessary...
Note: This may take some time, depending on the speed of your Internet connection.
> Pulling Riva Speech Server images.
  > Pulling nvcr.io/nvidia/riva/riva-speech:2.4.0-server-l4t-aarch64. This may take some time...

Downloading models (RMIRs) from NGC...
Note: this may take some time, depending on the speed of your Internet connection.
To skip this process and use existing RMIRs set the location and corresponding flag in config.sh.

Downloaded 1.96 GB in 7m 29s, Download speed: 4.47 MB/s                  
----------------------------------------------------
Transfer id: models_asr_conformer_en_us_str_v2.4.0-arm64 Download status: Completed.
Downloaded local path: /data/artifacts/models_asr_conformer_en_us_str_v2.4.0-arm64
Total files downloaded: 1 
Total downloaded size: 1.96 GB
Started at: 2022-08-26 15:09:03.219577
Completed at: 2022-08-26 15:16:32.964045
Duration taken: 7m 29s
----------------------------------------------------
Downloaded 190.96 MB in 58s, Download speed: 3.29 MB/s                
----------------------------------------------------
Transfer id: models_nlp_punctuation_bert_base_en_us_v2.4.0-arm64 Download status: Completed.
Downloaded local path: /data/artifacts/models_nlp_punctuation_bert_base_en_us_v2.4.0-arm64
Total files downloaded: 1 
Total downloaded size: 190.96 MB
Started at: 2022-08-26 15:16:37.423220
Completed at: 2022-08-26 15:17:35.522715
Duration taken: 58s
----------------------------------------------------
Downloaded 191.19 MB in 54s, Download speed: 3.53 MB/s               
----------------------------------------------------
Transfer id: models_nlp_intent_slot_misty_bert_base_v2.4.0-arm64 Download status: Completed.
Downloaded local path: /data/artifacts/models_nlp_intent_slot_misty_bert_base_v2.4.0-arm64
Total files downloaded: 1 
Total downloaded size: 191.19 MB
Started at: 2022-08-26 15:17:39.613611
Completed at: 2022-08-26 15:18:33.708269
Duration taken: 54s
----------------------------------------------------
Downloaded 105.93 MB in 59s, Download speed: 1.79 MB/s               
----------------------------------------------------
Transfer id: models_tts_fastpitch_hifigan_en_us_female_1_v2.4.0-arm64 Download status: Completed.
Downloaded local path: /data/artifacts/models_tts_fastpitch_hifigan_en_us_female_1_v2.4.0-arm64
Total files downloaded: 1 
Total downloaded size: 105.93 MB
Started at: 2022-08-26 15:18:38.049166
Completed at: 2022-08-26 15:19:37.147515
Duration taken: 59s
----------------------------------------------------

+ [[ tegra != \t\e\g\r\a ]]
+ [[ tegra == \t\e\g\r\a ]]
+ '[' -d /mnt/SSD1000/riva_quickstart_arm64_v2.4.0/model_repository/rmir ']'
+ [[ tegra == \t\e\g\r\a ]]
+ '[' -d /mnt/SSD1000/riva_quickstart_arm64_v2.4.0/model_repository/prebuilt ']'
+ echo 'Converting prebuilts at /mnt/SSD1000/riva_quickstart_arm64_v2.4.0/model_repository/prebuilt to Riva Model repository.'
Converting prebuilts at /mnt/SSD1000/riva_quickstart_arm64_v2.4.0/model_repository/prebuilt to Riva Model repository.
+ docker run -it -d --rm -v /mnt/SSD1000/riva_quickstart_arm64_v2.4.0/model_repository:/data --name riva-models-extract nvcr.io/nvidia/riva/riva-speech:2.4.0-server-l4t-aarch64
+ docker exec riva-models-extract bash -c 'mkdir -p /data/models; \
      for file in /data/prebuilt/*.tar.gz; do tar xf $file -C /data/models/ &> /dev/null; done'
+ docker container stop riva-models-extract
+ echo

+ echo 'Riva initialization complete. Run ./riva_start.sh to launch services.'
Riva initialization complete. Run ./riva_start.sh to launch services.

Running riva_start.sh results in the following error. Any support welcome !

markus@markus-orin:/mnt/SSD1000/riva_quickstart_arm64_v2.4.0$ bash riva_start.sh
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
99a50006e14767b7888eed5cd70d31545c8744685aab712a31070a786810b0e0
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'csv'
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime instead.: unknown.
Error response from daemon: Container 99a50006e14767b7888eed5cd70d31545c8744685aab712a31070a786810b0e0 is not running
Waiting for Riva server to load all models...retrying in 10 seconds

NGC setting format is json as confirmed by cat $HOME/.ngc/config

richg · August 27, 2022, 10:24am

I was having the same issue… It looks like there is a mismatch in the file naming convention.

It looks like maybe the file naming convention is wrong in the config.sh file… try editing the file to read like this at the very end, just change the order of the tags at the end… maybe the issue is the repo. but this gets them to download. I would be interested to know if this resolves the issue… I am still trying to get RIVA working here on both an AGX Xavier, and an Orin dev kit.

# define docker images required to run Riva
image_client=""
image_speech_api="nvcr.io/${NGC_TARGET}/riva-speech:${riva_ngc_image_version}-server-l4t-aarch64"

# define docker images required to setup Riva
image_init_speech="nvcr.io/${NGC_TARGET}/riva-speech:${riva_ngc_image_version}-servicemaker-l4t-aarch64"

I found the file names here… Riva Speech Skills | NVIDIA NGC

I am interested to know if you run into more issue… I’m still working through this as well.

richg · August 27, 2022, 10:43am

Ah, yes, I knew I had another problem… I was getting this error next…

rich@ubuntu:~/riva_quickstart_arm64_v2.4.0$ bash ./riva_start.sh
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
2bb1838f5eac64ac6cb7759286f47648c462d0ce041d7de28acf523be9af1472
docker: Error response from daemon: failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'csv'
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime instead.: unknown.
Error response from daemon: Container 2bb1838f5eac64ac6cb7759286f47648c462d0ce041d7de28acf523be9af1472 is not running

This I was able to resolve by editing the riva-start.sh mode and replacing the GPU related switch with
–runtime=nvidia

I’ll include both of my edited files. I think nvidia needs to add some more logic so that this script works properly.

config.sh (10.0 KB)
riva_start.sh (4.9 KB)

rvinobha · August 29, 2022, 4:23pm

Hi @user108910 and @richg

Thanks for your interest in Riva

I will check regarding the issues with the team and get back

Thanks

rvinobha · August 29, 2022, 5:40pm

Hi @user108910 and @richg

I have an update from team

With JP5.0.2, please check the below:

You have set the default runtime to nvidia on the Jetson platform by adding the following line in the /etc/docker/daemon.json file. Restart the Docker service using sudo systemctl restart docker after editing the file.

"default-runtime": "nvidia"

{
      "default-runtime": "nvidia",
      "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

Request to please check the above and let us know if it helps

Thanks

richg · August 30, 2022, 1:03am

This link isn’t loading … possibly internal?

rvinobha · August 30, 2022, 5:34am

HI @richg

Apologies, I have updated my previous comment, it will soon be updated in our quick start document

Thanks

745580913 · August 30, 2022, 6:39am

it doesn’t works.

rvinobha · September 6, 2022, 3:26pm

HI @745580913 @user108910 and @richg

Can we try with Riva 2.5.0

Thanks

richg · September 11, 2022, 7:34pm

In the config.sh it is still not downloading the docker image for the embedded version.

In the riva_start.sh it has the -gpus= line hard coded this needs to change to --runtime=nvidia for it to work on the embedded platforms.

If this is not the case please direct to a set of concise step by step instructions to follow and I will report back with my findings.

Regards,

Richard Graver

richg · September 11, 2022, 7:47pm

I apologize, I seem to have not downloaded the embedded version. I will test that one shortly…
This is according to Riva Skills Embedded Quick Start | NVIDIA NGC for anyone reading this who wants to test Jetson Orin or Xavier on Jetpack 5.0.2

One thing I do notice is that previously embedded was being labeled as l4t-aarch or aarch-l4t and arm64 was for the arm server platforms, have these been unified? I am correct in assuming that embedded is referring to Jetson because I notice that it is labeled as arm64.

ngc registry resource download-version “nvidia/riva/riva_quickstart_arm64:2.5.0”

richg · September 11, 2022, 7:55pm

(base) (venv) rich@ubuntu:~/riva_quickstart_arm64_v2.5.0$ bash riva_init.sh
Logging into NGC docker registry if necessary...
Pulling required docker images if necessary...
Note: This may take some time, depending on the speed of your Internet connection.
> Pulling Riva Speech Server images.
  > Image nvcr.io/nvidia/riva/riva-speech:2.5.0-l4t-aarch64 exists. Skipping.

Downloading models (RMIRs) from NGC...
Note: this may take some time, depending on the speed of your Internet connection.
To skip this process and use existing RMIRs set the location and corresponding flag in config.sh.

Downloaded 1.94 GB in 1m 10s, Download speed: 28.32 MB/s
--------------------------------------------------------------------------------
   Transfer id: models_asr_conformer_en_us_str_v2.5.0-arm64
   Download status: Completed
   Downloaded local path: /data/artifacts/models_asr_conformer_en_us_str_v2.5.0-arm64
   Total files downloaded: 1
   Total downloaded size: 1.94 GB
   Started at: 2022-09-11 19:50:57.156997
   Completed at: 2022-09-11 19:52:07.268315
   Duration taken: 1m 10s
--------------------------------------------------------------------------------
Downloaded 190.9 MB in 11s, Download speed: 17.32 MB/s
--------------------------------------------------------------------------------
   Transfer id: models_nlp_punctuation_bert_base_en_us_v2.5.0-arm64
   Download status: Completed
   Downloaded local path: /data/artifacts/models_nlp_punctuation_bert_base_en_us_v2.5.0-arm64
   Total files downloaded: 1
   Total downloaded size: 190.9 MB
   Started at: 2022-09-11 19:52:10.276755
   Completed at: 2022-09-11 19:52:21.296123
   Duration taken: 11s
--------------------------------------------------------------------------------
Downloaded 181.59 MB in 10s, Download speed: 18.12 MB/s
--------------------------------------------------------------------------------
   Transfer id: models_nlp_intent_slot_misty_bert_base_v2.5.0-arm64
   Download status: Completed
   Downloaded local path: /data/artifacts/models_nlp_intent_slot_misty_bert_base_v2.5.0-arm64
   Total files downloaded: 1
   Total downloaded size: 181.59 MB
   Started at: 2022-09-11 19:52:23.830194
   Completed at: 2022-09-11 19:52:33.851699
   Duration taken: 10s
--------------------------------------------------------------------------------
Downloaded 105.43 MB in 7s, Download speed: 15.04 MB/s
--------------------------------------------------------------------------------
   Transfer id: models_tts_fastpitch_hifigan_en_us_v2.5.0-arm64
   Download status: Completed
   Downloaded local path: /data/artifacts/models_tts_fastpitch_hifigan_en_us_v2.5.0-arm64
   Total files downloaded: 1
   Total downloaded size: 105.43 MB
   Started at: 2022-09-11 19:52:36.379629
   Completed at: 2022-09-11 19:52:43.394207
   Duration taken: 7s
--------------------------------------------------------------------------------

+ [[ tegra != \t\e\g\r\a ]]
+ [[ tegra == \t\e\g\r\a ]]
+ '[' -d /home/rich/riva_quickstart_arm64_v2.5.0/model_repository/rmir ']'
+ [[ tegra == \t\e\g\r\a ]]
+ '[' -d /home/rich/riva_quickstart_arm64_v2.5.0/model_repository/prebuilt ']'
+ echo 'Converting prebuilts at /home/rich/riva_quickstart_arm64_v2.5.0/model_repository/prebuilt to Riva Model repository.'
Converting prebuilts at /home/rich/riva_quickstart_arm64_v2.5.0/model_repository/prebuilt to Riva Model repository.
+ docker run -it -d --rm -v /home/rich/riva_quickstart_arm64_v2.5.0/model_repository:/data --name riva-models-extract nvcr.io/nvidia/riva/riva-speech:2.5.0-l4t-aarch64
+ docker exec riva-models-extract bash -c 'mkdir -p /data/models; \
      for file in /data/prebuilt/*.tar.gz; do tar xf $file -C /data/models/ &> /dev/null; done'
+ docker container stop riva-models-extract
+ '[' 0 -ne 0 ']'
+ echo

+ echo 'Riva initialization complete. Run ./riva_start.sh to launch services.'
Riva initialization complete. Run ./riva_start.sh to launch services.
(base) (venv) rich@ubuntu:~/riva_quickstart_arm64_v2.5.0$ bash riva_start.sh
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
a61150d1377f22b1b2e64bb2bac097fdddb56cb4bca9f06a847050087d74ae0e
docker: Error response from daemon: failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'csv'
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime instead.: unknown.
Error response from daemon: Container a61150d1377f22b1b2e64bb2bac097fdddb56cb4bca9f06a847050087d74ae0e is not running
Waiting for Riva server to load all models...retrying in 10 seconds
Waiting for Riva server to load all models...retrying in 10 seconds

This resource downloads the correct docker images however the riva_start.sh still contains the --gpus switch at the end which is hard coded, this should be changed to --runtime=nvidia

confirmed that editing the switch on the final launch command from as mentioned above allowed this resource to load properly on Orin.

I0911 19:58:34.891931 36 grpc_server.cc:4544] Started GRPCInferenceService at 0.0.0.0:8001
I0911 19:58:34.892959 36 http_server.cc:3242] Started HTTPService at 0.0.0.0:8000
I0911 19:58:34.934891 36 http_server.cc:180] Started Metrics Service at 0.0.0.0:8002
Waiting for Riva server to load all models...retrying in 10 seconds
Waiting for Riva server to load all models...retrying in 10 seconds
Riva server is ready...
Use this container terminal to run applications:
root@2bffebe28f54:/opt/riva#

745580913 · September 13, 2022, 6:32am

util 2.5.0, the hot word function(language is mandarin) still doesn’t work, how can I get it work? thanks.
grpc code is like below:
let speechContext = new rivaAsr.SpeechContext()
.setPhrasesList(normalSpeechContext)
.setBoost(100);

Topic		Replies	Views
Jetson AGX Orin: riva_init.sh Fails Riva	13	1572	August 3, 2022
Riva Quickstart 2.1.0 installation fails on AGX Orin Riva riva	13	1466	October 17, 2022
Riva quickstart 2.11 fails on xavier nx Riva	3	965	June 29, 2023
Riva_start.sh fails on Jetson Orin Riva	8	1447	April 23, 2024
Bash riva_start.sh never complete on Jetson AGX Orin Riva	9	1029	March 28, 2023
Failed to get riva started Riva riva	7	1792	December 3, 2022
RIVA Quick start for Embedded Platforms Failed to run on Jetson Riva jetson-inference , riva , ngc	2	940	March 21, 2023
Can´t start riva Riva	1	1214	April 5, 2022
On jetpack6.2, cant start riva_start.sh ...retrying in 10 seconds Jetson Orin NX riva	11	170	May 21, 2025
Getting error while instialaizing riva Riva installation , riva	5	1592	June 6, 2022

Riva Quickstart 2.4.0 installation fails on AGX Orin

Related topics