Riva 2.16 quick start error - riva_init.sh - invalid API key

playwithai · August 7, 2024, 9:28pm

Please provide the following information when requesting support.

Hardware - GPU (L40S)
Hardware - CPU(AMD EPYC 9654)
Operating System - Ubuntu 22.04 LTS Server
Riva Version - 2.16
TLT Version (if relevant)
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)

I have successfully installed and tested with sample commands using url of the 3 Riva NIM from NGC - ASR, TTS, NMT.

My $NGC_API_KEY is valid and I can login successfully using either docker or ngc_cli:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

However, I run into api key issue while followig Quick Start Guide — NVIDIA Riva

~/NVIDIA/riva_quickstart_v2.16.0$ bash riva_init.sh
Logging into NGC docker registry if necessary…
Pulling required docker images if necessary…
Note: This may take some time, depending on the speed of your Internet connection.

Pulling Riva Speech Server images.
Image nvcr.io/nvidia/riva/riva-speech:2.16.0 exists. Skipping.
Image nvcr.io/nvidia/riva/riva-speech:2.16.0-servicemaker exists. Skipping.

Downloading models (RMIRs) from NGC…
Note: this may take some time, depending on the speed of your Internet connection.
To skip this process and use existing RMIRs set the location and corresponding flag in config.sh.

==========================
=== Riva Speech Skills ===

NVIDIA Release (build 99025723)
Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:

To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh

To install the open-source samples corresponding to this TensorRT release version
run /opt/tensorrt/install_opensource.sh. To build the open source parsers,
plugins, and samples for current top-of-tree on master or a different branch,
run /opt/tensorrt/install_opensource.sh -b
See GitHub - NVIDIA/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. for more information.

2024-08-07 21:20:06 URL:https://files.ngc.nvidia.com/org/nvidia/team/ngc-apps/recipes/ngc_cli/versions/3.26.0/files/ngccli_linux.zip?versionId=45HwDObJnPZppQsIVrIVZy1ZDl9g1OA0&Expires=1723152006&Signature=CAztWW3PVFyzWI5g9KeChmkA3fHz-lGkCI3rmADVSrZI86fDvq1T0zLKu~8DgTiwKizFqZrj2DxeehQ7pvSYhbcBFAcvSDmMIntqEfM-whDVVTuDFfgN0NOirxCxu2ACT9XYR3XKt00b6JO9vB-87FbMewaCLLMIwY~oVqxna8XLkk3ipKSvD14WfB4dQAEDzHoVIQCS2l6lVVLpkSXhQZ5YtF1AEgcqN10~7hhVzqVIvTQ6aLQ5vK6zXVyO-QQUgrDhNgwxcHOnxYp-E~zvGXjJmfetdM6NWlFDb6jtENhnhQJ-uMl4xIrv9QHlbUszF7IgBNhKny3wZ-wIhTxzgw__&Key-Pair-Id=KCX06E8E9L60W [45878930/45878930] → “ngccli_linux.zip” [1]
/opt/riva
/data/rmir /opt/riva

Downloading nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0…
Error: Invalid apikey
Attempt 1 out of 3 failed
Trying again…
Error: Invalid apikey
Attempt 2 out of 3 failed
Trying again…
Error: Invalid apikey
Attempt 3 out of 3 failed
Error occurred downloading ‘nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0’. Exiting.
Error in downloading models.

playwithai · August 7, 2024, 9:54pm

My NGC_API_KEY is fine and I can use my key to pull Riva NIMs without issue.

$ echo “$NGC_API_KEY” | docker login nvcr.io --username ‘$oauthtoken’ --password-stdin
WARNING! Your password will be stored unencrypted in /home/user/.docker/config.json.
Configure a credential helper to remove this warning. See
docker login | Docker Docs

Login Succeeded

However, seems my key does not work for the following containers as I extracted the docker command from riva_init.sh:

$ docker run
–init -it --rm --gpus “device=0” -v riva-model-repo:/data -e NGC_CLI_API_KEY=$NGC_API_KEY
-e NGC_CLI_ORG=nvidia --name riva-service-maker nvcr.io/nvidia/riva/riva-speech:2.16.0-se
rvicemaker download_ngc_models nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0 nvidia/riva/rmir
_asr_conformer_en_us_ofl:2.16.0 nvidia/riva/rmir_nlp_punctuation_bert_base_en_us:2.16.0 nvidia/r
iva/rmir_nlp_punctuation_bert_base_en_us:2.16.0 nvidia/riva/rmir_tts_fastpitch_hifigan_en_us_ipa
:2.16.0

==========================
=== Riva Speech Skills ===

NVIDIA Release (build 99025723)
Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:

To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh

To install the open-source samples corresponding to this TensorRT release version
run /opt/tensorrt/install_opensource.sh. To build the open source parsers,
plugins, and samples for current top-of-tree on master or a different branch,
run /opt/tensorrt/install_opensource.sh -b
See GitHub - NVIDIA/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. for more information.

2024-08-07 21:48:04 URL:https://files.ngc.nvidia.com/org/nvidia/team/ngc-apps/recipes/ngc_cli/versions/3.26.0/files/ngccli_linux.zip?versionId=45HwDObJnPZppQsIVrIVZy1ZDl9g1OA0&Expires=1723153683&Signature=fmLRROc5Urbr3HDVVgYPOFf1YrgsEyasrfWKpPu08YZm~QKVGqRagVDnOAz3pDSGfiy2CAblyGxxYP5xirKfjZKgvJxlF35XAjPhi8wZqYR2IvxrKOlgvR-NaQwmEZaGzIwumlbx0pTmwt4NvIGAeDb9~qaoub-DDc6xJEHawvXB6A37xHlSTRz911EProOIUPkNZsYt5-L1z-4ESpHjNAjJBwFQGDiNBhLlHn4NFcJhchCcc9m~97gDr~82xH2aeYHd9vxKBhDaQJlQTg5OPa8SSIa4mH2LRpZ-eUOY~ZLPGv8GT3Z7dozgIQBUucCHWexGdBXaght5-G~1UdowLg__&Key-Pair-Id=KCX06E8E9L60W [45878930/45878930] → “ngccli_linux.zip” [1]
/opt/riva
You can now run: /tmp/aws/aws --version
/opt/riva
/data/rmir /opt/riva

Downloading nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0…
Error: Invalid apikey
Attempt 1 out of 3 failed
Trying again…
Error: Invalid apikey
Attempt 2 out of 3 failed
Trying again…
Error: Invalid apikey
Attempt 3 out of 3 failed
Error occurred downloading ‘nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0’. Exiting.

playwithai · August 7, 2024, 9:56pm

Now I narrowed the error to the download_ngc_models command inside the container:

I can pull the container of nvcr.io/nvidia/riva/riva-speech:2.16.0-servicemaker

$docker run --init -it --rm --gpus “device=0”
-v riva-model-repo:/data
-e NGC_CLI_API_KEY=nvapi-aXXXXXX1zg35qFZZ
-e NGC_CLI_ORG=nvidia
–name riva-service-maker
nvcr.io/nvidia/riva/riva-speech:2.16.0-servicemaker

Then I entered the container and run download_ngc_models nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0

Note that my NGC_CLI_API_KEY was passed to the container successfully:

root@a440d114e35f:/opt/riva# echo $NGC_CLI_API_KEY
nvapi-aXXXXXX1zg35qFZZ

root@a440d114e35f:/opt/riva#ngc registry nvidia/riva/rmir_asr_confdownload_ngc_models
/tmp/ngc-cli/ngc
/tmp/aws/aws
/data/rmir /opt/riva
/opt/riva
root@a440d114e35f:/opt/riva# download_ngc_models nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0
/tmp/ngc-cli/ngc
/tmp/aws/aws
/data/rmir /opt/riva

Downloading nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0…
Error: Invalid apikey
Attempt 1 out of 3 failed
Trying again…
Error: Invalid apikey
Attempt 2 out of 3 failed
Trying again…
Error: Invalid apikey
Attempt 3 out of 3 failed
Error occurred downloading ‘nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0’. Exiting.

playwithai · August 7, 2024, 10:28pm

Turns out I can download those models outside the riva-service-maker container, which makes the api error while downloading inside the container interesting…

$ echo “$NGC_API_KEY” | docker login nvcr.io --username ‘$oauthtoken’ --password-stdin
WARNING! Your password will be stored unencrypted in /home/user/.docker/config.json.
Configure a credential helper to remove this warning. See
docker login | Docker Docs

Login Succeeded

$ ngc-cli/ngc registry model download-version nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0
{
“download_end”: “2024-08-07 15:20:00”,
“download_start”: “2024-08-07 15:19:26”,
“download_time”: “33s”,
“files_downloaded”: 1,
“local_path”: “/home/user/NVIDIA/rmir_asr_conformer_en_us_str_v2.16.0”,
“size_downloaded”: “929.76 MB”,
“status”: “COMPLETED”
}

$ ngc-cli/ngc registry model download-version nvidia/riva/rmir_nlp_punctuation_bert_base_en_us:2.16.0
{
“download_end”: “2024-08-07 15:25:42”,
“download_start”: “2024-08-07 15:25:33”,
“download_time”: “8s”,
“files_downloaded”: 1,
“local_path”: “/home/user/NVIDIA/rmir_nlp_punctuation_bert_base_en_us_v2.16.0”,
“size_downloaded”: “384.79 MB”,
“status”: “COMPLETED”
}

playwithai · August 7, 2024, 11:57pm

Okay so I ended up by using ngc to download those 4 models directly then pass them over to riva_init.sh by editing the config.sh:

nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0
nvidia/riva/rmir_asr_conformer_en_us_ofl:2.16.0
nvidia/riva/rmir_nlp_punctuation_bert_base_en_us:2.16.0
nvidia/riva/rmir_tts_fastpitch_hifigan_en_us_ipa:2.16.0

$ ngc-cli/ngc registry model download-version nvidia/riva/rmir_asr_conformer_en_us_str:2.16.0
{
“download_end”: “2024-08-07 15:20:00”,
“download_start”: “2024-08-07 15:19:26”,
“download_time”: “33s”,
“files_downloaded”: 1,
“local_path”: “/home/user/NVIDIA/rmir_asr_conformer_en_us_str_v2.16.0”,
“size_downloaded”: “929.76 MB”,
“status”: “COMPLETED”
}

riva_model_loc=“/home/user/NVIDIA/riva_2.16_models_for_docker”
use_existing_rmirs=true

then run $bash config.sh && bash riva_init.sh

system · August 21, 2024, 11:58pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
NGC RMIRs Error in downloading models Riva riva	17	1019	February 26, 2024
Getting Error on command bash riva_init.sh Riva	10	1030	March 28, 2023
Unable to download RIVA models during riva_init.sh Riva	2	911	October 21, 2022
Jetson AGX Orin: riva_init.sh Fails Riva	13	1434	August 3, 2022
Riva Quickstart 2.4.0 installation fails on AGX Orin Riva	12	1177	September 13, 2022
Riva Speech Skills initialisation - Error 'RivaTokenizer' object has no attribute Riva	3	728	May 5, 2022
Riva quickstart 2.11 fails on xavier nx Riva	3	899	June 29, 2023
Riva_start.sh will not start the server Riva riva	4	1049	August 31, 2023
Error 403 access denied when downloading Hifigan NGC GPU Cloud	0	72	August 1, 2024
Getting error while instialaizing riva Riva installation , riva	5	1522	June 6, 2022

Riva 2.16 quick start error - riva_init.sh - invalid API key

========================== === Riva Speech Skills ===

========================== === Riva Speech Skills ===

Related topics

==========================
=== Riva Speech Skills ===

==========================
=== Riva Speech Skills ===