Jetpack 5.0? Riva Rasa Conv AI application

Ubuntu 20.04
Jetson Xavier NX
Running jetpack
Package: nvidia-jetpack
Version: 4.6.2-b5
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194

Im working on Riva and Rasa Domain Specific Chatbot. There is contradictory information on JetPack in all the different documents regarding Riva.

Should I upgrade jetpack and if so what is the best way. Will apt do it?

Thank you

Here is my apt upgrade attempt

nx1@NX1:~/riva$ sudo apt upgrade nvidia-jetpack
Reading package lists… Done
Building dependency tree
Reading state information… Done
nvidia-jetpack is already the newest version (4.6.2-b5).
Calculating upgrade… Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

update upon further investigation I see that my nvidia-smi is not polling metrics.

see below:

Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
5e198ba843666cf0ead399d0dfdc62ebabf999a127b20e6f7ae921c21c62fca6
I0712 01:35:36.925262 24 onnxruntime.cc:2319] TRITONBACKEND_Initialize: onnxruntime
I0712 01:35:36.925497 24 onnxruntime.cc:2329] Triton TRITONBACKEND API version: 1.8
I0712 01:35:36.925531 24 onnxruntime.cc:2335] 'onnxruntime' TRITONBACKEND API version: 1.8
I0712 01:35:36.925553 24 onnxruntime.cc:2365] backend configuration:
{}
W0712 01:35:36.986056 24 pinned_memory_manager.cc:236] Unable to allocate pinned system memory, pinned memory pool will not be available: initialization error
E0712 01:35:36.986168 24 server.cc:201] **Failed to initialize CUDA memory manager: unable to get number of CUDA devices: initialization error**
W0712 01:35:36.986194 24 server.cc:208] failed to enable peer access for some device pairs
E0712 01:35:36.991700 24 model_repository_manager.cc:1927] Poll failed for model directory 'citrinet-1024-en-US-asr-streaming-ctc-decoder-cpu-streaming': unable to get number of CUDA devices: initialization error
E0712 01:35:36.992499 24 model_repository_manager.cc:1927] Poll failed for model directory 'citrinet-1024-en-US-asr-streaming-feature-extractor-streaming': unable to get number of CUDA devices: initialization error
E0712 01:35:36.993222 24 model_repository_manager.cc:1927] Poll failed for model directory 'citrinet-1024-en-US-asr-streaming-voice-activity-detector-ctc-streaming': unable to get number of CUDA devices: initialization error
E0712 01:35:36.994644 24 model_repository_manager.cc:1927] Poll failed for model directory 'intent_slot_detokenizer': unable to get number of CUDA devices: initialization error
E0712 01:35:36.995012 24 model_repository_manager.cc:1927] Poll failed for model directory 'intent_slot_label_tokens_misty': unable to get number of CUDA devices: initialization error
E0712 01:35:36.995543 24 model_repository_manager.cc:1927] Poll failed for model directory 'intent_slot_tokenizer-en-US-misty': unable to get number of CUDA devices: initialization error
E0712 01:35:36.996004 24 model_repository_manager.cc:1927] Poll failed for model directory 'riva-onnx-fastpitch_encoder-English-US-Female-1': unable to get number of CUDA devices: initialization error
E0712 01:35:36.996610 24 model_repository_manager.cc:1927] Poll failed for model directory 'riva-punctuation-en-US': unable to get number of CUDA devices: initialization error
E0712 01:35:36.997043 24 model_repository_manager.cc:1927] Poll failed for model directory 'riva-trt-citrinet-1024-en-US-asr-streaming-am-streaming': unable to get number of CUDA devices: initialization error
E0712 01:35:36.997503 24 model_repository_manager.cc:1927] Poll failed for model directory 'riva-trt-hifigan-English-US-Female-1': unable to get number of CUDA devices: initialization error
E0712 01:35:36.997913 24 model_repository_manager.cc:1927] Poll failed for model directory 'riva-trt-riva-punctuation-en-US-nn-bert-base-uncased': unable to get number of CUDA devices: initialization error
E0712 01:35:36.998348 24 model_repository_manager.cc:1927] Poll failed for model directory 'riva-trt-riva_intent_misty-nn-bert-base-uncased': unable to get number of CUDA devices: initialization error
E0712 01:35:36.999602 24 model_repository_manager.cc:1927] Poll failed for model directory 'spectrogram_chunker-English-US-Female-1': unable to get number of CUDA devices: initialization error
E0712 01:35:37.000104 24 model_repository_manager.cc:1927] Poll failed for model directory 'tts_postprocessor-English-US-Female-1': unable to get number of CUDA devices: initialization error
E0712 01:35:37.000786 24 model_repository_manager.cc:1927] Poll failed for model directory 'tts_preprocessor-English-US-Female-1': unable to get number of CUDA devices: initialization error
E0712 01:35:37.001316 24 model_repository_manager.cc:1341] Invalid argument: ensemble citrinet-1024-en-US-asr-streaming contains models that are not available: citrinet-1024-en-US-asr-streaming-feature-extractor-streaming, riva-trt-citrinet-1024-en-US-asr-streaming-am-streaming, citrinet-1024-en-US-asr-streaming-voice-activity-detector-ctc-streaming, citrinet-1024-en-US-asr-streaming-ctc-decoder-cpu-streaming
E0712 01:35:37.001359 24 model_repository_manager.cc:1341] Invalid argument: ensemble fastpitch_hifigan_ensemble-English-US-Female-1 contains models that are not available: tts_preprocessor-English-US-Female-1, riva-onnx-fastpitch_encoder-English-US-Female-1, spectrogram_chunker-English-US-Female-1, riva-trt-hifigan-English-US-Female-1, tts_postprocessor-English-US-Female-1
E0712 01:35:37.001384 24 model_repository_manager.cc:1341] Invalid argument: ensemble riva_intent_misty contains models that are not available: intent_slot_tokenizer-en-US-misty, riva-trt-riva_intent_misty-nn-bert-base-uncased, intent_slot_label_tokens_misty, intent_slot_detokenizer
I0712 01:35:37.001505 24 server.cc:524] 
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I0712 01:35:37.001625 24 server.cc:551] 
+-------------+-----------------------------------------------------------------+--------+
| Backend     | Path                                                            | Config |
+-------------+-----------------------------------------------------------------+--------+
| onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so | {}     |
+-------------+-----------------------------------------------------------------+--------+

I0712 01:35:37.001688 24 server.cc:594] 
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+

**W0712 01:35:37.001719 24 metrics.cc:325] Neither cache metrics nor gpu metrics are enabled. Not polling for them.**

Please dont tell me I have to reflash my NX… I hope there is another way.

Hi @vanessa.crosby

Thanks for your interest in Riva

Currently we don’t support Jetpack 4.6.2
The supported Jetpack version from Riva 2.2.0 release onwards is JetPack 5.0 and JetPack 5.0.1

So we kindly request to upgrade your JetPack Version to the above stated, and let us know if you still face issues

Regarding contradictory information on JetPack present in Riva docs, we request to kindly give the link or point us to the same, So that we can update our document

Thanks

I cannot flash to 5.0 or 5.1 - it kills my Jetson Xavier NX dev kit unit.

Also is this page still relevant? All the images are broken: Virtual Assistant (with Rasa) — NVIDIA Riva

Thanks
V

You also wanted info on the documentation that needs to be updated. Riva Skills Quick Start | NVIDIA NGC does not mention the embedded option (no client run needed), should be updated to reflect this. More to follow.
V

Hi @vanessa.crosby

Thank you so much for your kind inputs, we will look and update the docs you have shared with us

Regarding the Jetson flashing and related queries, We request to kindly open a post in the below Jetson Xavier NX forum

Thanks