Cannot Run Riva ASR (zh-CN) on Jetson Nano Super

I’m trying to run the Riva ASR service with the Mandarin Chinese (zh-CN) model on my Jetson Nano Super Developer Kit (8GB). The service fails to start, but it works correctly under these other conditions:

  • English (en-US) ASR on the Jetson Nano Super Developer Kit.
  • Chinese (zh-CN) ASR on a Jetson AGX Orin with 64GB of memory.

The issue seems to be specific to running the zh-CN model on the Jetson Nano Super. My main question is: Is there a way to successfully run the Riva zh-CN ASR model on the Jetson Nano Super, or is this hardware not supported for this specific model?

Thanks.

Detailed Information

Hardware

  • GPU: Jetson Nano Super developer kit (8GB)
  • CPU: Jetson Nano Super developer kit (8GB)
  • Operating System: Ubuntu 22.04.5 LTS + Jetpack 6.2

Software

  • Riva Version: 2.19.0

Steps to Reproduce I followed the standard Riva deployment process, with the following modifications to config.sh:

  1. Set service_enabled_asr=true
  2. Set service_enabled_nlp=false
  3. Set service_enabled_tts=false
  4. Set service_enabled_nmt=false
  5. Set asr_language_code=("zh-CN")

After these changes, I ran the following commands:

bash ./riva_clean.sh
bash ./riva_init.sh
bash ./riva_start.sh

Observed Behavior The server gets stuck at the “Waiting for Riva server to load all models…retrying in 10 seconds” message and eventually times out.

The docker logs riva-speech output shows a clear failure to load one of the models: riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese. The error message is:

I0806 17:48:53.308918 22 tensorrt.cc:353] "TRITONBACKEND_ModelInstanceFinalize: delete instance state"
E0806 17:48:53.309015 22 backend_model.cc:692] "ERROR: Failed to create instance: unable to create TensorRT context: ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cask_fusion.cpp:initialize_cask_fusion:381] Cask unknown internal error)"
I0806 17:48:53.309044 22 tensorrt.cc:274] "TRITONBACKEND_ModelFinalize: delete model state"
E0806 17:48:53.309432 22 model_lifecycle.cc:642] "failed to load 'riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese' version 1: Internal: unable to create TensorRT context: ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cask_fusion.cpp:initialize_cask_fusion:381] Cask unknown internal error)"
I0806 17:48:53.309469 22 model_lifecycle.cc:777] "failed to load 'riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese'"

The following is the full log:


docker logs riva-speech
  > Riva waiting for Triton server to load all models...retrying in 1 second
I0806 17:48:51.325719 22 pinned_memory_manager.cc:277] "Pinned memory pool is created at '0x2040ae000' with size 268435456"
I0806 17:48:51.326250 22 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 0 with size 1000000000"
I0806 17:48:51.615222 22 model_lifecycle.cc:472] "loading: riva-trt-conformer-zh-CN-asr-streaming-am-streaming:1"
I0806 17:48:51.615352 22 model_lifecycle.cc:472] "loading: riva-punctuation-zh-CN:1"
I0806 17:48:51.615448 22 model_lifecycle.cc:472] "loading: conformer-zh-CN-asr-streaming-asr-bls-ensemble:1"
I0806 17:48:51.615505 22 model_lifecycle.cc:472] "loading: riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese:1"
I0806 17:48:51.746274 22 tensorrt.cc:65] "TRITONBACKEND_Initialize: tensorrt"
I0806 17:48:51.746342 22 tensorrt.cc:75] "Triton TRITONBACKEND API version: 1.19"
I0806 17:48:51.746353 22 tensorrt.cc:81] "'tensorrt' TRITONBACKEND API version: 1.19"
I0806 17:48:51.746361 22 tensorrt.cc:105] "backend configuration:\n{\"cmdline\":{\"auto-complete-config\":\"false\",\"backend-directory\":\"/opt/tritonserver/backends\",\"min-compute-capability\":\"5.300000\",\"default-max-batch-size\":\"4\"}}"
I0806 17:48:51.747310 22 tensorrt.cc:231] "TRITONBACKEND_ModelInitialize: riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese (version 1)"
I0806 17:48:51.747311 22 tensorrt.cc:231] "TRITONBACKEND_ModelInitialize: riva-trt-conformer-zh-CN-asr-streaming-am-streaming (version 1)"
I0806 17:48:51.750005 22 tensorrt.cc:297] "TRITONBACKEND_ModelInstanceInitialize: riva-trt-conformer-zh-CN-asr-streaming-am-streaming_0_0 (GPU device 0)"
I0806 17:48:51.750056 22 tensorrt.cc:297] "TRITONBACKEND_ModelInstanceInitialize: riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese_0_0 (GPU device 0)"
  > Riva waiting for Triton server to load all models...retrying in 1 second
I0806 17:48:52.619515 22 pipeline_library.cc:24] "TRITONBACKEND_ModelInitialize: riva-punctuation-zh-CN (version 1)"
I0806 17:48:52.621124 22 backend_model.cc:303] "model configuration:\n{\n    \"name\": \"riva-punctuation-zh-CN\",\n    \"platform\": \"\",\n    \"backend\": \"riva_nlp_pipeline\",\n    \"runtime\": \"\",\n    \"version_policy\": {\n        \"latest\": {\n            \"num_versions\": 1\n        }\n    },\n    \"max_batch_size\": 1,\n    \"input\": [\n        {\n            \"name\": \"PIPELINE_INPUT\",\n            \"data_type\": \"TYPE_STRING\",\n            \"format\": \"FORMAT_NONE\",\n            \"dims\": [\n                1\n            ],\n            \"is_shape_tensor\": false,\n            \"allow_ragged_batch\": false,\n            \"optional\": false,\n            \"is_non_linear_format_io\": false\n        }\n    ],\n    \"output\": [\n        {\n            \"name\": \"PIPELINE_OUTPUT\",\n            \"data_type\": \"TYPE_STRING\",\n            \"dims\": [\n                1\n            ],\n            \"label_filename\": \"\",\n            \"is_shape_tensor\": false,\n            \"is_non_linear_format_io\": false\n        }\n    ],\n    \"batch_input\": [],\n    \"batch_output\": [],\n    \"optimization\": {\n        \"priority\": \"PRIORITY_DEFAULT\",\n        \"input_pinned_memory\": {\n            \"enable\": true\n        },\n        \"output_pinned_memory\": {\n            \"enable\": true\n        },\n        \"gather_kernel_buffer_threshold\": 0,\n        \"eager_batching\": false\n    },\n    \"instance_group\": [\n        {\n            \"name\": \"riva-punctuation-zh-CN_0\",\n            \"kind\": \"KIND_CPU\",\n            \"count\": 1,\n            \"gpus\": [],\n            \"secondary_devices\": [],\n            \"profile\": [],\n            \"passive\": false,\n            \"host_policy\": \"\"\n        }\n    ],\n    \"default_model_filename\": \"\",\n    \"cc_model_filenames\": {},\n    \"metric_tags\": {},\n    \"parameters\": {\n        \"unk_token\": {\n            \"string_value\": \"[UNK]\"\n        },\n        \"model_api\": {\n            \"string_value\": \"PunctuateText\"\n        },\n        \"input_ids_tensor_name\": {\n            \"string_value\": \"input_ids\"\n        },\n        \"to_lower\": {\n            \"string_value\": \"true\"\n        },\n        \"capit_logits_tensor_name\": {\n            \"string_value\": \"capit_logits\"\n        },\n        \"attn_mask_tensor_name\": {\n            \"string_value\": \"attention_mask\"\n        },\n        \"tokenizer_to_lower\": {\n            \"string_value\": \"true\"\n        },\n        \"model_name\": {\n            \"string_value\": \"riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese\"\n        },\n        \"preserve_accents\": {\n            \"string_value\": \"false\"\n        },\n        \"token_type_tensor_name\": {\n            \"string_value\": \"token_type_ids\"\n        },\n        \"use_int64_nn_inputs\": {\n            \"string_value\": \"False\"\n        },\n        \"bos_token\": {\n            \"string_value\": \"[CLS]\"\n        },\n        \"delimiter\": {\n            \"string_value\": \".\"\n        },\n        \"pipeline_type\": {\n            \"string_value\": \"punctuation\"\n        },\n        \"code_point_filename\": {\n            \"string_value\": \"cp_data.json\"\n        },\n        \"capitalization_mapping_path\": {\n            \"string_value\": \"e97f0158fc914bf4ba333243227d9daf_capit_labels.txt\"\n        },\n        \"language_code\": {\n            \"string_value\": \"zh-CN\"\n        },\n        \"unicode_normalize\": {\n            \"string_value\": \"False\"\n        },\n        \"punctuation_mapping_path\": {\n            \"string_value\": \"9c726721aad54c6b981ef0d51b7fa200_punct_labels.txt\"\n        },\n        \"load_model\": {\n            \"string_value\": \"false\"\n        },\n        \"remove_spaces\": {\n            \"string_value\": \"True\"\n        },\n        \"tokenizer\": {\n            \"string_value\": \"wordpiece\"\n        },\n        \"vocab\": {\n            \"string_value\": \"83e30d7b3628459c8b25043dbc1b59de_vocab.txt\"\n        },\n        \"pad_chars_with_spaces\": {\n            \"string_value\": \"True\"\n        },\n        \"eos_token\": {\n            \"string_value\": \"[SEP]\"\n        },\n        \"model_family\": {\n            \"string_value\": \"riva\"\n        },\n        \"punct_logits_tensor_name\": {\n            \"string_value\": \"punct_logits\"\n        }\n    },\n    \"model_warmup\": []\n}"
I0806 17:48:52.621410 22 pipeline_library.cc:28] "TRITONBACKEND_ModelInstanceInitialize: riva-punctuation-zh-CN_0_0 (device 0)"
I0806 17:48:52.693967 22 pipeline_library.cc:22] "TRITONBACKEND_ModelInitialize: conformer-zh-CN-asr-streaming-asr-bls-ensemble (version 1)"
Found yaml file: /data/models/conformer-zh-CN-asr-streaming-asr-bls-ensemble/1/riva_bls_config.yaml
I0806 17:48:52.766380 22 backend_model.cc:303] "model configuration:\n{\n    \"name\": \"conformer-zh-CN-asr-streaming-asr-bls-ensemble\",\n    \"platform\": \"\",\n    \"backend\": \"riva_asr_ensemble_pipeline\",\n    \"runtime\": \"\",\n    \"version_policy\": {\n        \"latest\": {\n            \"num_versions\": 1\n        }\n    },\n    \"max_batch_size\": 1024,\n    \"input\": [\n        {\n            \"name\": \"PIPELINE_INPUT\",\n            \"data_type\": \"TYPE_STRING\",\n            \"format\": \"FORMAT_NONE\",\n            \"dims\": [\n                1\n            ],\n            \"is_shape_tensor\": false,\n            \"allow_ragged_batch\": false,\n            \"optional\": false,\n            \"is_non_linear_format_io\": false\n        }\n    ],\n    \"output\": [\n        {\n            \"name\": \"PIPELINE_OUTPUT\",\n            \"data_type\": \"TYPE_STRING\",\n            \"dims\": [\n                1\n            ],\n            \"label_filename\": \"\",\n            \"is_shape_tensor\": false,\n            \"is_non_linear_format_io\": false\n        }\n    ],\n    \"batch_input\": [],\n    \"batch_output\": [],\n    \"optimization\": {\n        \"graph\": {\n            \"level\": 0\n        },\n        \"priority\": \"PRIORITY_DEFAULT\",\n        \"cuda\": {\n            \"graphs\": false,\n            \"busy_wait_events\": false,\n            \"graph_spec\": [],\n            \"output_copy_stream\": true\n        },\n        \"input_pinned_memory\": {\n            \"enable\": true\n        },\n        \"output_pinned_memory\": {\n            \"enable\": true\n        },\n        \"gather_kernel_buffer_threshold\": 0,\n        \"eager_batching\": false\n    },\n    \"sequence_batching\": {\n        \"oldest\": {\n            \"max_candidate_sequences\": 1024,\n            \"preferred_batch_size\": [\n                64,\n                128\n            ],\n            \"max_queue_delay_microseconds\": 1000,\n            \"preserve_ordering\": false\n        },\n        \"max_sequence_idle_microseconds\": 60000000,\n        \"control_input\": [\n            {\n                \"name\": \"START\",\n                \"control\": [\n                    {\n                        \"kind\": \"CONTROL_SEQUENCE_START\",\n                        \"int32_false_true\": [\n                            0,\n                            1\n                        ],\n                        \"fp32_false_true\": [],\n                        \"bool_false_true\": [],\n                        \"data_type\": \"TYPE_INVALID\"\n                    }\n                ]\n            },\n            {\n                \"name\": \"READY\",\n                \"control\": [\n                    {\n                        \"kind\": \"CONTROL_SEQUENCE_READY\",\n                        \"int32_false_true\": [\n                            0,\n                            1\n                        ],\n                        \"fp32_false_true\": [],\n                        \"bool_false_true\": [],\n                        \"data_type\": \"TYPE_INVALID\"\n                    }\n                ]\n            },\n            {\n                \"name\": \"END\",\n                \"control\": [\n                    {\n                        \"kind\": \"CONTROL_SEQUENCE_END\",\n                        \"int32_false_true\": [\n                            0,\n                            1\n                        ],\n                        \"fp32_false_true\": [],\n                        \"bool_false_true\": [],\n                        \"data_type\": \"TYPE_INVALID\"\n                    }\n                ]\n            },\n            {\n                \"name\": \"CORRID\",\n                \"control\": [\n                    {\n                        \"kind\": \"CONTROL_SEQUENCE_CORRID\",\n                        \"int32_false_true\": [],\n                        \"fp32_false_true\": [],\n                        \"bool_false_true\": [],\n                        \"data_type\": \"TYPE_UINT64\"\n                    }\n                ]\n            }\n        ],\n        \"state\": [],\n        \"iterative_sequence\": false\n    },\n    \"instance_group\": [\n        {\n            \"name\": \"conformer-zh-CN-asr-streaming-asr-bls-ensemble_0\",\n            \"kind\": \"KIND_CPU\",\n            \"count\": 1,\n            \"gpus\": [],\n            \"secondary_devices\": [],\n            \"profile\": [],\n            \"passive\": false,\n            \"host_policy\": \"\"\n        }\n    ],\n    \"default_model_filename\": \"\",\n    \"cc_model_filenames\": {},\n    \"metric_tags\": {},\n    \"parameters\": {\n        \"offline\": {\n            \"string_value\": \"False\"\n        },\n        \"type\": {\n            \"string_value\": \"online\"\n        },\n        \"yaml_parameters_file\": {\n            \"string_value\": \"riva_bls_config.yaml\"\n        },\n        \"sample_rate\": {\n            \"string_value\": \"16000\"\n        },\n        \"streaming\": {\n            \"string_value\": \"True\"\n        },\n        \"model_family\": {\n            \"string_value\": \"riva\"\n        },\n        \"language_code\": {\n            \"string_value\": \"zh-CN\"\n        }\n    },\n    \"model_warmup\": [],\n    \"model_transaction_policy\": {\n        \"decoupled\": true\n    }\n}"
I0806 17:48:52.767045 22 pipeline_library.cc:26] "TRITONBACKEND_ModelInstanceInitialize: conformer-zh-CN-asr-streaming-asr-bls-ensemble_0_0 (device 0)"
I0806 17:48:52.951743 22 logging.cc:46] "Loaded engine size: 196 MiB"
W0806 17:48:52.992229 22 logging.cc:43] "Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors."
I0806 17:48:53.058156 22 model_lifecycle.cc:839] "successfully loaded 'riva-punctuation-zh-CN'"
I0806 17:48:53.144200 22 logging.cc:46] "The logger passed into createInferRuntime differs from one already provided for an existing builder, runtime, or refitter. Uses of the global logger, returned by nvinfer1::getLogger(), will return the existing value."
  > Riva waiting for Triton server to load all models...retrying in 1 second
E0806 17:48:53.308815 22 logging.cc:40] "ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cask_fusion.cpp:initialize_cask_fusion:381] Cask unknown internal error)"
I0806 17:48:53.308918 22 tensorrt.cc:353] "TRITONBACKEND_ModelInstanceFinalize: delete instance state"
E0806 17:48:53.309015 22 backend_model.cc:692] "ERROR: Failed to create instance: unable to create TensorRT context: ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cask_fusion.cpp:initialize_cask_fusion:381] Cask unknown internal error)"
I0806 17:48:53.309044 22 tensorrt.cc:274] "TRITONBACKEND_ModelFinalize: delete model state"
E0806 17:48:53.309432 22 model_lifecycle.cc:642] "failed to load 'riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese' version 1: Internal: unable to create TensorRT context: ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cask_fusion.cpp:initialize_cask_fusion:381] Cask unknown internal error)"
I0806 17:48:53.309469 22 model_lifecycle.cc:777] "failed to load 'riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese'"
I0806 17:48:53.513776 22 logging.cc:46] "Loaded engine size: 253 MiB"
W0806 17:48:53.513925 22 logging.cc:43] "Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors."
I0806 17:48:53.702196 22 logging.cc:46] "[MS] Running engine with multi stream info"
I0806 17:48:53.702268 22 logging.cc:46] "[MS] Number of aux streams is 1"
I0806 17:48:53.702277 22 logging.cc:46] "[MS] Number of total worker streams is 2"
I0806 17:48:53.702285 22 logging.cc:46] "[MS] The main stream provided by execute/enqueue calls is the first worker stream"
  > Riva waiting for Triton server to load all models...retrying in 1 second
I0806 17:48:54.348510 22 logging.cc:46] "[MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +10, now: CPU 0, GPU 442 (MiB)"
I0806 17:48:54.353520 22 instance_state.cc:186] "Created instance riva-trt-conformer-zh-CN-asr-streaming-am-streaming_0_0 on GPU 0 with stream priority 0 and optimization profile default[0];"
I0806 17:48:54.353937 22 model_lifecycle.cc:839] "successfully loaded 'riva-trt-conformer-zh-CN-asr-streaming-am-streaming'"
  > Riva waiting for Triton server to load all models...retrying in 1 second
  > Riva waiting for Triton server to load all models...retrying in 1 second
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0806 17:48:56.826931    41 asr_ensemble_factory.cc:278] Loading acoustic model
I0806 17:48:56.827013    41 asr_ensemble_factory.cc:284] Done loading acoustic model
I0806 17:48:56.831712    41 normalizer.cc:66] Proto String: tokenizer_grammar:  "tokenizer.ascii_proto"
verbalizer_grammar:  "verbalizer.ascii_proto"
sentence_boundary_regexp: "[\\.:!\\?] "
sentence_boundary_exceptions_file: "sentence_boundary_exceptions.txt"
I0806 17:48:56.863541 22 model_lifecycle.cc:839] "successfully loaded 'conformer-zh-CN-asr-streaming-asr-bls-ensemble'"
I0806 17:48:56.863869 22 server.cc:604] 
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I0806 17:48:56.863930 22 server.cc:631] 
+----------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Backend                    | Path                                                                                          | Config                                                                                                                                                         |
+----------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tensorrt                   | /opt/tritonserver/backends/tensorrt/libtriton_tensorrt.so                                     | {"cmdline":{"auto-complete-config":"false","backend-directory":"/opt/tritonserver/backends","min-compute-capability":"5.300000","default-max-batch-size":"4"}} |
| riva_nlp_pipeline          | /opt/tritonserver/backends/riva_nlp_pipeline/libtriton_riva_nlp_pipeline.so                   | {"cmdline":{"auto-complete-config":"false","backend-directory":"/opt/tritonserver/backends","min-compute-capability":"5.300000","default-max-batch-size":"4"}} |
| riva_asr_ensemble_pipeline | /opt/tritonserver/backends/riva_asr_ensemble_pipeline/libtriton_riva_asr_ensemble_pipeline.so | {"cmdline":{"auto-complete-config":"false","backend-directory":"/opt/tritonserver/backends","min-compute-capability":"5.300000","default-max-batch-size":"4"}} |
+----------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+

I0806 17:48:56.864079 22 server.cc:674] 
+------------------------------------------------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Model                                                | Version | Status                                                                                                                                                                                         |
+------------------------------------------------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| conformer-zh-CN-asr-streaming-asr-bls-ensemble       | 1       | READY                                                                                                                                                                                          |
| riva-punctuation-zh-CN                               | 1       | READY                                                                                                                                                                                          |
| riva-trt-conformer-zh-CN-asr-streaming-am-streaming  | 1       | READY                                                                                                                                                                                          |
| riva-trt-riva-punctuation-zh-CN-nn-bert-base-chinese | 1       | UNAVAILABLE: Internal: unable to create TensorRT context: ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cask_fusion.cpp:initialize_cask_fusion:381] Cask unknown internal error) |
+------------------------------------------------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I0806 17:48:56.864311 22 tritonserver.cc:2598] 
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option                           | Value                                                                                                                                                                                                           |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id                        | triton                                                                                                                                                                                                          |
| server_version                   | 2.50.0                                                                                                                                                                                                          |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging |
| model_repository_path[0]         | /data/models                                                                                                                                                                                                    |
| model_control_mode               | MODE_NONE                                                                                                                                                                                                       |
| strict_model_config              | 1                                                                                                                                                                                                               |
| model_config_name                |                                                                                                                                                                                                                 |
| rate_limit                       | OFF                                                                                                                                                                                                             |
| pinned_memory_pool_byte_size     | 268435456                                                                                                                                                                                                       |
| cuda_memory_pool_byte_size{0}    | 1000000000                                                                                                                                                                                                      |
| min_supported_compute_capability | 5.3                                                                                                                                                                                                             |
| strict_readiness                 | 1                                                                                                                                                                                                               |
| exit_timeout                     | 30                                                                                                                                                                                                              |
| cache_enabled                    | 0                                                                                                                                                                                                               |
+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I0806 17:48:56.864376 22 server.cc:305] "Waiting for in-flight requests to complete."
I0806 17:48:56.864396 22 server.cc:321] "Timeout 30: Found 0 model versions that have in-flight inferences"
I0806 17:48:56.865070 22 server.cc:336] "All models are stopped, unloading models"
I0806 17:48:56.865094 22 server.cc:345] "Timeout 30: Found 3 live models and 0 in-flight non-inference requests"
I0806 17:48:56.865269 22 pipeline_library.cc:31] "TRITONBACKEND_ModelInstanceFinalize: delete instance state"
I0806 17:48:56.865304 22 tensorrt.cc:353] "TRITONBACKEND_ModelInstanceFinalize: delete instance state"
I0806 17:48:56.865586 22 pipeline_library.cc:29] "TRITONBACKEND_ModelInstanceFinalize: delete instance state"
I0806 17:48:56.880498 22 pipeline_library.cc:27] "TRITONBACKEND_ModelFinalize: delete model state"
I0806 17:48:56.880695 22 model_lifecycle.cc:624] "successfully unloaded 'riva-punctuation-zh-CN' version 1"
I0806 17:48:57.043164 22 tensorrt.cc:274] "TRITONBACKEND_ModelFinalize: delete model state"
I0806 17:48:57.052405 22 model_lifecycle.cc:624] "successfully unloaded 'riva-trt-conformer-zh-CN-asr-streaming-am-streaming' version 1"
  > Riva waiting for Triton server to load all models...retrying in 1 second
I0806 17:48:57.663857 22 pipeline_library.cc:25] "TRITONBACKEND_ModelFinalize: delete model state"
I0806 17:48:57.664118 22 model_lifecycle.cc:624] "successfully unloaded 'conformer-zh-CN-asr-streaming-asr-bls-ensemble' version 1"
I0806 17:48:57.865448 22 server.cc:345] "Timeout 29: Found 0 live models and 0 in-flight non-inference requests"
error: creating server: Internal - failed to load all models
  > Riva waiting for Triton server to load all models...retrying in 1 second
  > Triton server died before reaching ready state. Terminating Riva startup.
Check Triton logs with: docker logs 
/opt/riva/bin/start-riva: line 1: kill: (22) - No such process

You need more memory to run this setup for this HW.