Please provide the following information when requesting support.
Hardware - GPU (Geforce RTX 3080TI)
Hardware - CPU Intel i9
Operating System Ubuntu 20.04
Riva Version 2.1.0
TLT Version (if relevant)
How to reproduce the issue ? sudo bash riva_init.sh
(This is for errors. Please share the command and the detailed log here)
Here’s the information:
sudo docker run --privileged --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
[sudo] password for paul:
Tue May 3 10:48:16 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | N/A |
| 0% 49C P8 17W / 350W | 324MiB / 12288MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1190 G 35MiB |
| 0 N/A N/A 1889 G 95MiB |
| 0 N/A N/A 2018 G 28MiB |
| 0 N/A N/A 2527 G 0MiB |
| 0 N/A N/A 2572 G 148MiB |
+-----------------------------------------------------------------------------+
I am now encountering an issue with an error extracting 'dict_intents_file'
Below is the output after running sudo bash riva_init.sh
Please enter API key for ngc.nvidia.com:
Logging into NGC docker registry if necessary...
Pulling required docker images if necessary...
Note: This may take some time, depending on the speed of your Internet connection.
> Pulling Riva Speech Server images.
> Image nvcr.io/nvidia/riva/riva-speech:2.1.0-server exists. Skipping.
> Image nvcr.io/nvidia/riva/riva-speech-client:2.1.0 exists. Skipping.
> Image nvcr.io/nvidia/riva/riva-speech:2.1.0-servicemaker exists. Skipping.
Downloading models (RMIRs) from NGC...
Note: this may take some time, depending on the speed of your Internet connection.
To skip this process and use existing RMIRs set the location and corresponding flag in config.sh.
==========================
=== Riva Speech Skills ===
==========================
NVIDIA Release 22.04 (build 37144275)
Riva Speech Server Version 2.1.0
Copyright (c) 2016-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Copyright (c) 2018-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
https://developer.nvidia.com/tensorrt
Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh
To install the open-source samples corresponding to this TensorRT release version
run /opt/tensorrt/install_opensource.sh. To build the open source parsers,
plugins, and samples for current top-of-tree on master or a different branch,
run /opt/tensorrt/install_opensource.sh -b <branch>
See https://github.com/NVIDIA/TensorRT for more information.
/data/artifacts /opt/riva
Directory rmir_asr_citrinet_1024_en_us_str_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_asr_citrinet_1024_en_us_ofl_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_nlp_punctuation_bert_base_en_us_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_nlp_punctuation_bert_base_en_us_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_nlp_named_entity_recognition_bert_base_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_nlp_intent_slot_bert_base_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_nlp_question_answering_bert_base_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_nlp_text_classification_bert_base_v2.1.0 already exists, skipping. Use '--force' option to override.
Directory rmir_tts_fastpitch_hifigan_en_us_female_1_v2.1.0 already exists, skipping. Use '--force' option to override.
/opt/riva
+ [[ amd64 == \a\m\d\6\4 ]]
+ [[ amd64 == \a\r\m\6\4 ]]
+ echo 'Converting RMIRs at riva-model-repo/rmir to Riva Model repository.'
Converting RMIRs at riva-model-repo/rmir to Riva Model repository.
+ docker run --privileged --init -it --rm --gpus '"device=0"' -v riva-model-repo:/data -e MODEL_DEPLOY_KEY=tlt_encode --name riva-service-maker nvcr.io/nvidia/riva/riva-speech:2.1.0-servicemaker deploy_all_models /data/rmir /data/models
==========================
=== Riva Speech Skills ===
==========================
NVIDIA Release 22.04 (build 37144275)
Riva Speech Server Version 2.1.0
Copyright (c) 2016-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Copyright (c) 2018-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
https://developer.nvidia.com/tensorrt
Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh
To install the open-source samples corresponding to this TensorRT release version
run /opt/tensorrt/install_opensource.sh. To build the open source parsers,
plugins, and samples for current top-of-tree on master or a different branch,
run /opt/tensorrt/install_opensource.sh -b <branch>
See https://github.com/NVIDIA/TensorRT for more information.
[NeMo W 2022-05-03 11:00:46 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
2022-05-03 11:00:46,542 [INFO] Writing Riva model repository to '/data/models'...
2022-05-03 11:00:46,542 [INFO] The riva model repo target directory is /data/models
2022-05-03 11:00:48,955 [INFO] Extract_binaries for language_model -> /data/models/riva-trt-riva_intent_weather-nn-bert-base-uncased/1
2022-05-03 11:00:48,955 [INFO] extracting {'ckpt': ('nemo.collections.nlp.models.intent_slot_classification.intent_slot_classification_model.IntentSlotClassificationModel', 'model_weights.ckpt'), 'bert_config_file': ('nemo.collections.nlp.models.intent_slot_classification.intent_slot_classification_model.IntentSlotClassificationModel', 'bert-base-uncased_encoder_config.json')} -> /data/models/riva-trt-riva_intent_weather-nn-bert-base-uncased/1
2022-05-03 11:00:52,328 [INFO] Printing copied artifacts:
2022-05-03 11:00:52,328 [INFO] {'ckpt': '/data/models/riva-trt-riva_intent_weather-nn-bert-base-uncased/1/model_weights.ckpt', 'bert_config_file': '/data/models/riva-trt-riva_intent_weather-nn-bert-base-uncased/1/bert-base-uncased_encoder_config.json'}
2022-05-03 11:00:52,328 [INFO] Building TRT engine from PyTorch Checkpoint
[NeMo W 2022-05-03 11:00:53 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
2022-05-03 11:01:10,295 [INFO] Reading bindings from:/tmp/bert_bindings.yaml
2022-05-03 11:01:10,333 [INFO] Intent classes: 18
2022-05-03 11:01:10,334 [INFO] Entity classes: 31
2022-05-03 11:01:10,334 [WARNING] /data/models/intent_slot_tokenizer-en-US-weather already exists, skipping deployment. To force deployment rerun with -f or remove the /data/models/intent_slot_tokenizer-en-US-weather
2022-05-03 11:01:10,334 [INFO] Extract_binaries for label_tokens -> /data/models/intent_slot_label_tokens_weather/1
2022-05-03 11:01:10,334 [INFO] extracting {'dict_slots_file': ('nemo.collections.nlp.models.intent_slot_classification.intent_slot_classification_model.IntentSlotClassificationModel', 'slot_labels.csv')} -> /data/models/intent_slot_label_tokens_weather/1
2022-05-03 11:01:10,335 [INFO] Extract_binaries for detokenizer -> /data/models/intent_slot_detokenizer/1
2022-05-03 11:01:10,336 [INFO] Extract_binaries for self -> /data/models/riva_intent_weather/1
2022-05-03 11:01:10,336 [INFO] extracting {'dict_intents_file': ('nemo.collections.nlp.models.intent_slot_classification.intent_slot_classification_model.IntentSlotClassificationModel', 'intent_labels.csv')} -> /data/models/riva_intent_weather/1
2022-05-03 11:01:10,336 [ERROR] Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/servicemaker/cli/deploy.py", line 100, in deploy_from_rmir
generator.serialize_to_disk(
File "/usr/local/lib/python3.8/dist-packages/servicemaker/triton/triton.py", line 419, in serialize_to_disk
RivaConfigGenerator.serialize_to_disk(self, repo_dir, rmir, config_only, verbose, overwrite)
File "/usr/local/lib/python3.8/dist-packages/servicemaker/triton/triton.py", line 305, in serialize_to_disk
self.generate_config(version_dir, rmir)
File "/usr/local/lib/python3.8/dist-packages/servicemaker/triton/nlp.py", line 550, in generate_config
'input_map': {tokenizer._inputs[0].name: "CLASSIFY_STR_IN__0"},
AttributeError: 'RivaTokenizer' object has no attribute '_inputs'
Hi, after spending days trying to configure a new machine, and having to wipe and start again, I felt I was finally getting somewhere.
Any help would be appreciated.
Paul