I’m trying to run riva on AGX orin L4t 36. However im getting the following error:
/opt/riva/bin/start-riva: line 10: curl: command not found
/opt/riva/bin/start-riva: line 11: [: -ne: unary operator expected
> Triton server is ready...
Warning: '--strict-model-config' has been deprecated! Please use '--disable-auto-complete-config' instead.
W0216 03:48:19.357550 8689 pinned_memory_manager.cc:236] Unable to allocate pinned system memory, pinned memory pool will not be available: CUDA driver version is insufficient for CUDA runtime version
I0216 03:48:19.357718 8689 cuda_memory_manager.cc:115] CUDA memory pool disabled
I0216 03:48:19.367029 8689 model_lifecycle.cc:459] loading: conformer-en-US-asr-streaming-ctc-decoder-cpu-streaming:1
I0216 03:48:19.367149 8689 model_lifecycle.cc:459] loading: conformer-en-US-asr-streaming-endpointing-streaming:1
I0216 03:48:19.367258 8689 model_lifecycle.cc:459] loading: conformer-en-US-asr-streaming-feature-extractor-streaming:1
I0216 03:48:19.367569 8689 model_lifecycle.cc:459] loading: intent_slot_detokenizer:1
I0216 03:48:19.367621 8689 model_lifecycle.cc:459] loading: intent_slot_label_tokens_misty:1
I0216 03:48:19.367677 8689 model_lifecycle.cc:459] loading: intent_slot_tokenizer-en-US-misty:1
I0216 03:48:19.367737 8689 model_lifecycle.cc:459] loading: riva-onnx-fastpitch_encoder-English-US:1
I0216 03:48:19.367945 8689 model_lifecycle.cc:459] loading: riva-punctuation-en-US:1
I0216 03:48:19.368014 8689 model_lifecycle.cc:459] loading: riva-pyt-radtts_encoder-English-US-RadTTS:1
I0216 03:48:19.368064 8689 model_lifecycle.cc:459] loading: riva-trt-conformer-en-US-asr-streaming-am-streaming:1
I0216 03:48:19.368115 8689 model_lifecycle.cc:459] loading: riva-trt-hifigan-English-US:1
I0216 03:48:19.368198 8689 model_lifecycle.cc:459] loading: riva-trt-hifigan-English-US-RadTTS:1
I0216 03:48:19.368255 8689 model_lifecycle.cc:459] loading: riva-trt-riva-punctuation-en-US-nn-bert-base-uncased:1
I0216 03:48:19.368310 8689 model_lifecycle.cc:459] loading: riva-trt-riva_intent_misty-nn-bert-base-uncased:1
I0216 03:48:19.368391 8689 model_lifecycle.cc:459] loading: spectrogram_chunker-English-US:1
I0216 03:48:19.368455 8689 model_lifecycle.cc:459] loading: spectrogram_chunker-English-US-RadTTS:1
I0216 03:48:19.368586 8689 model_lifecycle.cc:459] loading: tts_postprocessor-English-US:1
I0216 03:48:19.368645 8689 model_lifecycle.cc:459] loading: tts_postprocessor-English-US-RadTTS:1
I0216 03:48:19.368609 8691 riva_server.cc:125] Using Insecure Server Credentials
I0216 03:48:19.368809 8689 model_lifecycle.cc:459] loading: tts_preprocessor-English-US:1
I0216 03:48:19.368968 8689 model_lifecycle.cc:459] loading: tts_preprocessor-English-US-RadTTS:1
E0216 03:48:19.371394 8691 model_registry.cc:246] error: unable to get server status: failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:8001: Failed to connect to remote host: Connection refused
E0216 03:48:19.374392 8689 model_lifecycle.cc:596] failed to load 'conformer-en-US-asr-streaming-ctc-decoder-cpu-streaming' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.377045 8689 model_lifecycle.cc:596] failed to load 'conformer-en-US-asr-streaming-endpointing-streaming' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.380311 8689 model_lifecycle.cc:596] failed to load 'conformer-en-US-asr-streaming-feature-extractor-streaming' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.383014 8689 model_lifecycle.cc:596] failed to load 'intent_slot_detokenizer' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.385581 8689 model_lifecycle.cc:596] failed to load 'intent_slot_label_tokens_misty' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.388222 8689 model_lifecycle.cc:596] failed to load 'intent_slot_tokenizer-en-US-misty' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
I0216 03:48:19.390698 8689 onnxruntime.cc:2459] TRITONBACKEND_Initialize: onnxruntime
I0216 03:48:19.390736 8689 onnxruntime.cc:2469] Triton TRITONBACKEND API version: 1.10
I0216 03:48:19.390748 8689 onnxruntime.cc:2475] 'onnxruntime' TRITONBACKEND API version: 1.10
I0216 03:48:19.390757 8689 onnxruntime.cc:2505] backend configuration:
{"cmdline":{"auto-complete-config":"false","min-compute-capability":"5.300000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}}
E0216 03:48:19.407582 8689 model_lifecycle.cc:596] failed to load 'riva-onnx-fastpitch_encoder-English-US' version 1: Invalid argument: instance group riva-onnx-fastpitch_encoder-English-US_0 of model riva-onnx-fastpitch_encoder-English-US has kind KIND_GPU but no GPUs are available
I0216 03:48:19.945297 8689 libtorch.cc:1985] TRITONBACKEND_Initialize: pytorch
I0216 03:48:19.945382 8689 libtorch.cc:1995] Triton TRITONBACKEND API version: 1.10
I0216 03:48:19.945402 8689 libtorch.cc:2001] 'pytorch' TRITONBACKEND API version: 1.10
E0216 03:48:19.945461 8689 model_lifecycle.cc:596] failed to load 'riva-pyt-radtts_encoder-English-US-RadTTS' version 1: Invalid argument: instance group riva-pyt-radtts_encoder-English-US-RadTTS_0 of model riva-pyt-radtts_encoder-English-US-RadTTS has kind KIND_GPU but no GPUs are available
E0216 03:48:19.947358 8689 model_lifecycle.cc:596] failed to load 'riva-punctuation-en-US' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.949558 8689 model_lifecycle.cc:596] failed to load 'riva-trt-conformer-en-US-asr-streaming-am-streaming' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvos.so)
E0216 03:48:19.951605 8689 model_lifecycle.cc:596] failed to load 'riva-trt-hifigan-English-US' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvos.so)
E0216 03:48:19.953556 8689 model_lifecycle.cc:596] failed to load 'riva-trt-riva-punctuation-en-US-nn-bert-base-uncased' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvos.so)
E0216 03:48:19.955304 8689 model_lifecycle.cc:596] failed to load 'riva-trt-hifigan-English-US-RadTTS' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvos.so)
E0216 03:48:19.956992 8689 model_lifecycle.cc:596] failed to load 'riva-trt-riva_intent_misty-nn-bert-base-uncased' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvos.so)
E0216 03:48:19.958553 8689 model_lifecycle.cc:596] failed to load 'spectrogram_chunker-English-US' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.959793 8689 model_lifecycle.cc:596] failed to load 'spectrogram_chunker-English-US-RadTTS' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.961998 8689 model_lifecycle.cc:596] failed to load 'tts_postprocessor-English-US' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.963663 8689 model_lifecycle.cc:596] failed to load 'tts_postprocessor-English-US-RadTTS' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.966132 8689 model_lifecycle.cc:596] failed to load 'tts_preprocessor-English-US' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.967886 8689 model_lifecycle.cc:596] failed to load 'tts_preprocessor-English-US-RadTTS' version 1: Not found: unable to load shared library: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/nvidia/libnvrm_gpu.so)
E0216 03:48:19.968017 8689 model_repository_manager.cc:481] Invalid argument: ensemble 'conformer-en-US-asr-streaming' depends on 'conformer-en-US-asr-streaming-ctc-decoder-cpu-streaming' which has no loaded version
E0216 03:48:19.968045 8689 model_repository_manager.cc:481] Invalid argument: ensemble 'fastpitch_hifigan_ensemble-English-US' depends on 'tts_postprocessor-English-US' which has no loaded version
E0216 03:48:19.968056 8689 model_repository_manager.cc:481] Invalid argument: ensemble 'radtts_hifigan_ensemble-English-US-RadTTS' depends on 'tts_postprocessor-English-US-RadTTS' which has no loaded version
E0216 03:48:19.968063 8689 model_repository_manager.cc:481] Invalid argument: ensemble 'riva_intent_misty' depends on 'intent_slot_detokenizer' which has no loaded version
I0216 03:48:19.968172 8689 server.cc:563]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I0216 03:48:19.968317 8689 server.cc:590]
+-------------+----------------------------------------------------------------+----------------------------------------------------------------+
| Backend | Path | Config |
+-------------+----------------------------------------------------------------+----------------------------------------------------------------+
| onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.s | {"cmdline":{"auto-complete-config":"false","min-compute-capabi |
| | o | lity":"5.300000","backend-directory":"/opt/tritonserver/backen |
| | | ds","default-max-batch-size":"4"}} |
| | | |
| pytorch | /opt/tritonserver/backends/pytorch/libtriton_pytorch.so | {"cmdline":{"auto-complete-config":"false","min-compute-capabi |
| | | lity":"5.300000","backend-directory":"/opt/tritonserver/backen |
| | | ds","default-max-batch-size":"4"}} |
+-------------+----------------------------------------------------------------+----------------------------------------------------------------+
I0216 03:48:19.968883 8689 server.cc:633]
+-----------------------------------------------------------+---------+-------------------------------------------------------------------------+
| Model | Version | Status |
+-----------------------------------------------------------+---------+-------------------------------------------------------------------------+
| conformer-en-US-asr-streaming-ctc-decoder-cpu-streaming | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| conformer-en-US-asr-streaming-endpointing-streaming | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| conformer-en-US-asr-streaming-feature-extractor-streaming | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| intent_slot_detokenizer | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| intent_slot_label_tokens_misty | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| intent_slot_tokenizer-en-US-misty | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| riva-onnx-fastpitch_encoder-English-US | 1 | UNAVAILABLE: Invalid argument: instance group riva-onnx-fastpitch_encod |
| | | er-English-US_0 of model riva-onnx-fastpitch_encoder-English-US has kin |
| | | d KIND_GPU but no GPUs are available |
| riva-punctuation-en-US | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| riva-pyt-radtts_encoder-English-US-RadTTS | 1 | UNAVAILABLE: Invalid argument: instance group riva-pyt-radtts_encoder-E |
| | | nglish-US-RadTTS_0 of model riva-pyt-radtts_encoder-English-US-RadTTS h |
| | | as kind KIND_GPU but no GPUs are available |
| riva-trt-conformer-en-US-asr-streaming-am-streaming | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvos.so) |
| riva-trt-hifigan-English-US | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvos.so) |
| riva-trt-hifigan-English-US-RadTTS | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvos.so) |
| riva-trt-riva-punctuation-en-US-nn-bert-base-uncased | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvos.so) |
| riva-trt-riva_intent_misty-nn-bert-base-uncased | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvos.so) |
| spectrogram_chunker-English-US | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| spectrogram_chunker-English-US-RadTTS | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| tts_postprocessor-English-US | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| tts_postprocessor-English-US-RadTTS | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| tts_preprocessor-English-US | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
| tts_preprocessor-English-US-RadTTS | 1 | UNAVAILABLE: Not found: unable to load shared library: /lib/aarch64-lin |
| | | ux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/ |
| | | aarch64-linux-gnu/nvidia/libnvrm_gpu.so) |
+-----------------------------------------------------------+---------+-------------------------------------------------------------------------+
W0216 03:48:19.969793 8689 metrics.cc:354] No polling metrics (CPU, GPU, Cache) are enabled. Will not poll for them.
I0216 03:48:19.970013 8689 tritonserver.cc:2264]
+----------------------------------+------------------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.27.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configu |
| | ration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace logging |
| model_repository_path[0] | /data/models |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| rate_limit | OFF |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 1000000000 |
| response_cache_byte_size | 0 |
| min_supported_compute_capability | 5.3 |
| strict_readiness | 1 |
| exit_timeout | 30 |
+----------------------------------+------------------------------------------------------------------------------------------------------------+
I0216 03:48:19.970365 8689 server.cc:264] Waiting for in-flight requests to complete.
I0216 03:48:19.970400 8689 server.cc:280] Timeout 30: Found 0 model versions that have in-flight inferences
I0216 03:48:19.970429 8689 server.cc:295] All models are stopped, unloading models
I0216 03:48:19.970439 8689 server.cc:302] Timeout 30: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models
One of the processes has exited unexpectedly. Stopping container.
W0216 03:48:24.355655 8691 riva_server.cc:195] Signal: 15
I’ve done the following:
ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.14.0sudo bash riva_init.shbash riva_start.sh
Device:
Nvidia AGX Orin Dev Kit 64GB
OS: Ubuntu 22
Firmware: L4T 36
Please advice on how to resolve this
P.S. i’ve also tried running sudo apt-get upgrade libstdc++6, but that also didnt work