========================== === Riva Speech Skills === ========================== NVIDIA Release 21.10 (build 29090669) Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved. Various files include modifications (c) NVIDIA CORPORATION. All rights reserved. NVIDIA modifications are covered by the license terms that apply to the underlying project or file. NOTE: The SHMEM allocation limit is set to the default of 64MB. This may be insufficient for the inference server. NVIDIA recommends the use of the following flags: nvidia-docker run --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 ... > Riva waiting for Triton server to load all models...retrying in 1 second I1203 04:27:59.886842 75 metrics.cc:290] Collecting metrics for GPU 0: NVIDIA GeForce RTX 3060 I1203 04:27:59.889045 75 onnxruntime.cc:1970] TRITONBACKEND_Initialize: onnxruntime I1203 04:27:59.889057 75 onnxruntime.cc:1980] Triton TRITONBACKEND API version: 1.4 I1203 04:27:59.889060 75 onnxruntime.cc:1986] 'onnxruntime' TRITONBACKEND API version: 1.4 I1203 04:28:00.066177 75 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f527c000000' with size 268435456 I1203 04:28:00.066441 75 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 1000000000 I1203 04:28:00.069956 75 model_repository_manager.cc:1045] loading: riva-onnx-fastpitch_encoder:1 I1203 04:28:00.170208 75 model_repository_manager.cc:1045] loading: spectrogram_chunker:1 I1203 04:28:00.170373 75 onnxruntime.cc:2029] TRITONBACKEND_ModelInitialize: riva-onnx-fastpitch_encoder (version 1) I1203 04:28:00.171831 75 onnxruntime.cc:2072] TRITONBACKEND_ModelInstanceInitialize: riva-onnx-fastpitch_encoder_0 (GPU device 0) I1203 04:28:00.270431 75 model_repository_manager.cc:1045] loading: riva-trt-hifigan:1 I1203 04:28:00.370658 75 model_repository_manager.cc:1045] loading: tts_postprocessor:1 I1203 04:28:00.470935 75 model_repository_manager.cc:1045] loading: tts_preprocessor:1 > Riva waiting for Triton server to load all models...retrying in 1 second I1203 04:28:00.961871 75 logging.cc:49] [MemUsageChange] Init CUDA: CPU +659, GPU +0, now: CPU 873, GPU 1850 (MiB) I1203 04:28:00.962720 75 logging.cc:49] Loaded engine size: 40 MB I1203 04:28:00.962813 75 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 875 MiB, GPU 1852 MiB I1203 04:28:01.447036 75 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +18, now: CPU 1508, GPU 2160 (MiB) > Riva waiting for Triton server to load all models...retrying in 1 second I1203 04:28:02.005607 75 logging.cc:49] [MemUsageChange] Init cuDNN: CPU +323, GPU +336, now: CPU 1831, GPU 2496 (MiB) I1203 04:28:02.006308 75 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1831, GPU 2480 (MiB) I1203 04:28:02.006371 75 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine end: CPU 1831 MiB, GPU 2480 MiB I1203 04:28:02.017349 75 plan_backend.cc:456] Creating instance riva-trt-hifigan_0_0_gpu0 on GPU 0 (8.6) using model.plan I1203 04:28:02.018317 75 logging.cc:49] Loaded engine size: 40 MB I1203 04:28:02.018440 75 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 1825 MiB, GPU 2458 MiB I1203 04:28:02.039636 75 onnxruntime.cc:2106] TRITONBACKEND_ModelInstanceFinalize: delete instance state I1203 04:28:02.039710 75 onnxruntime.cc:2052] TRITONBACKEND_ModelFinalize: delete model state E1203 04:28:02.039726 75 model_repository_manager.cc:1215] failed to load 'riva-onnx-fastpitch_encoder' version 1: Invalid argument: unable to load model 'riva-onnx-fastpitch_encoder', configuration expects 1 inputs, model provides 3 I1203 04:28:02.041955 75 spectrogram-chunker.cc:239] TRITONBACKEND_ModelInitialize: spectrogram_chunker (version 1) I1203 04:28:02.042636 75 backend_model.cc:255] model configuration: { "name": "spectrogram_chunker", "platform": "", "backend": "riva_tts_chunker", "version_policy": { "latest": { "num_versions": 1 } }, "max_batch_size": 8, "input": [ { "name": "SPECTROGRAM", "data_type": "TYPE_FP32", "format": "FORMAT_NONE", "dims": [ 80, -1 ], "is_shape_tensor": false, "allow_ragged_batch": false }, { "name": "IS_LAST_SENTENCE", "data_type": "TYPE_INT32", "format": "FORMAT_NONE", "dims": [ 1 ], "is_shape_tensor": false, "allow_ragged_batch": false }, { "name": "NUM_VALID_FRAMES_IN", "data_type": "TYPE_INT64", "format": "FORMAT_NONE", "dims": [ 1 ], "is_shape_tensor": false, "allow_ragged_batch": false }, { "name": "SENTENCE_NUM", "data_type": "TYPE_INT32", "format": "FORMAT_NONE", "dims": [ 1 ], "is_shape_tensor": false, "allow_ragged_batch": false }, { "name": "DURATIONS", "data_type": "TYPE_FP32", "format": "FORMAT_NONE", "dims": [ -1 ], "is_shape_tensor": false, "allow_ragged_batch": false }, { "name": "PROCESSED_TEXT", "data_type": "TYPE_STRING", "format": "FORMAT_NONE", "dims": [ 1 ], "is_shape_tensor": false, "allow_ragged_batch": false } ], "output": [ { "name": "SPECTROGRAM_CHUNK", "data_type": "TYPE_FP32", "dims": [ 80, -1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "END_FLAG", "data_type": "TYPE_INT32", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "NUM_VALID_SAMPLES_OUT", "data_type": "TYPE_INT32", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "SENTENCE_NUM", "data_type": "TYPE_INT32", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "DURATIONS", "data_type": "TYPE_FP32", "dims": [ -1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "PROCESSED_TEXT", "data_type": "TYPE_STRING", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false } ], "batch_input": [], "batch_output": [], "optimization": { "priority": "PRIORITY_DEFAULT", "cuda": { "graphs": false, "busy_wait_events": false, "graph_spec": [], "output_copy_stream": true }, "input_pinned_memory": { "enable": true }, "output_pinned_memory": { "enable": true }, "gather_kernel_buffer_threshold": 0, "eager_batching": false }, "sequence_batching": { "oldest": { "max_candidate_sequences": 8, "preferred_batch_size": [ 8 ], "max_queue_delay_microseconds": 1000 }, "max_sequence_idle_microseconds": 60000000, "control_input": [ { "name": "START", "control": [ { "kind": "CONTROL_SEQUENCE_START", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "READY", "control": [ { "kind": "CONTROL_SEQUENCE_READY", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "END", "control": [ { "kind": "CONTROL_SEQUENCE_END", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "CORRID", "control": [ { "kind": "CONTROL_SEQUENCE_CORRID", "int32_false_true": [], "fp32_false_true": [], "data_type": "TYPE_UINT64" } ] } ] }, "instance_group": [ { "name": "spectrogram_chunker_0", "kind": "KIND_GPU", "count": 1, "gpus": [ 0 ], "secondary_devices": [], "profile": [], "passive": false, "host_policy": "" } ], "default_model_filename": "", "cc_model_filenames": {}, "metric_tags": {}, "parameters": { "num_samples_per_frame": { "string_value": "256" }, "max_execution_batch_size": { "string_value": "8" }, "chunk_length": { "string_value": "80" }, "num_mels": { "string_value": "80" } }, "model_warmup": [], "model_transaction_policy": { "decoupled": true } } I1203 04:28:02.043681 75 tts-postprocessor.cc:267] TRITONBACKEND_ModelInitialize: tts_postprocessor (version 1) W:parameter_parser.cc:118: Parameter engine_path could not be set from parameters W:parameter_parser.cc:119: Default value will be used I1203 04:28:02.044331 75 backend_model.cc:255] model configuration: { "name": "tts_postprocessor", "platform": "", "backend": "riva_tts_postprocessor", "version_policy": { "latest": { "num_versions": 1 } }, "max_batch_size": 8, "input": [ { "name": "INPUT", "data_type": "TYPE_FP32", "format": "FORMAT_NONE", "dims": [ 1, -1 ], "is_shape_tensor": false, "allow_ragged_batch": false }, { "name": "NUM_VALID_SAMPLES", "data_type": "TYPE_INT32", "format": "FORMAT_NONE", "dims": [ 1 ], "is_shape_tensor": false, "allow_ragged_batch": false } ], "output": [ { "name": "OUTPUT", "data_type": "TYPE_FP32", "dims": [ -1 ], "label_filename": "", "is_shape_tensor": false } ], "batch_input": [], "batch_output": [], "optimization": { "priority": "PRIORITY_DEFAULT", "input_pinned_memory": { "enable": true }, "output_pinned_memory": { "enable": true }, "gather_kernel_buffer_threshold": 0, "eager_batching": false }, "sequence_batching": { "oldest": { "max_candidate_sequences": 8, "preferred_batch_size": [ 8 ], "max_queue_delay_microseconds": 100 }, "max_sequence_idle_microseconds": 60000000, "control_input": [ { "name": "START", "control": [ { "kind": "CONTROL_SEQUENCE_START", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "READY", "control": [ { "kind": "CONTROL_SEQUENCE_READY", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "END", "control": [ { "kind": "CONTROL_SEQUENCE_END", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "CORRID", "control": [ { "kind": "CONTROL_SEQUENCE_CORRID", "int32_false_true": [], "fp32_false_true": [], "data_type": "TYPE_UINT64" } ] } ] }, "instance_group": [ { "name": "tts_postprocessor_0", "kind": "KIND_GPU", "count": 1, "gpus": [ 0 ], "secondary_devices": [], "profile": [], "passive": false, "host_policy": "" } ], "default_model_filename": "", "cc_model_filenames": {}, "metric_tags": {}, "parameters": { "use_denoiser": { "string_value": "False" }, "chunk_num_samples": { "string_value": "20480" }, "fade_length": { "string_value": "128" }, "max_chunk_size": { "string_value": "65536" }, "max_execution_batch_size": { "string_value": "8" }, "hop_length": { "string_value": "256" }, "filter_length": { "string_value": "1024" } }, "model_warmup": [], "model_transaction_policy": { "decoupled": false } } I1203 04:28:02.044689 75 spectrogram-chunker.cc:241] TRITONBACKEND_ModelInstanceInitialize: spectrogram_chunker_0 (device 0) I1203 04:28:02.044721 75 tts-preprocessor.cc:204] TRITONBACKEND_ModelInitialize: tts_preprocessor (version 1) I1203 04:28:02.044892 75 model_repository_manager.cc:1212] successfully loaded 'spectrogram_chunker' version 1 W:parameter_parser.cc:118: Parameter append_punctuation could not be set from parameters W:parameter_parser.cc:119: Default value will be used W:parameter_parser.cc:118: Parameter norm_proto_path could not be set from parameters W:parameter_parser.cc:119: Default value will be used W:parameter_parser.cc:118: Parameter pad_with_space could not be set from parameters W:parameter_parser.cc:119: Default value will be used W:parameter_parser.cc:118: Parameter padding_token could not be set from parameters W:parameter_parser.cc:119: Default value will be used W:parameter_parser.cc:118: Parameter split_on_sentence could not be set from parameters W:parameter_parser.cc:119: Default value will be used W1203 04:28:02.045245 75 tts-preprocessor.cc:192] Parameter abbreviation_path is deprecated WARNING: Logging before InitGoogleLogging() is written to STDERR I1203 04:28:02.045310 92 preprocessor.cc:39] TTS character mapping loaded from /data/models/tts_preprocessor/1/mapping.txt I1203 04:28:02.046537 92 preprocessor.cc:66] TTS phonetic mapping loaded from /data/models/tts_preprocessor/1/ I1203 04:28:02.046555 92 preprocessor.cc:79] Abbreviation mapping loaded from /data/models/tts_preprocessor/1/ I1203 04:28:02.046738 92 preprocessor.cc:89] TTS normalizer loaded from /opt/riva/tn_configs/sparrowhawk_configuration.ascii_proto I1203 04:28:02.116630 75 backend_model.cc:255] model configuration: { "name": "tts_preprocessor", "platform": "", "backend": "riva_tts_preprocessor", "version_policy": { "latest": { "num_versions": 1 } }, "max_batch_size": 8, "input": [ { "name": "input_string", "data_type": "TYPE_STRING", "format": "FORMAT_NONE", "dims": [ 1 ], "is_shape_tensor": false, "allow_ragged_batch": false } ], "output": [ { "name": "output", "data_type": "TYPE_INT64", "dims": [ 1, 400 ], "label_filename": "", "is_shape_tensor": false }, { "name": "output_mask", "data_type": "TYPE_FP32", "dims": [ 1, 400, 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "output_length", "data_type": "TYPE_INT32", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "is_last_sentence", "data_type": "TYPE_INT32", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "output_string", "data_type": "TYPE_STRING", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false }, { "name": "sentence_num", "data_type": "TYPE_INT32", "dims": [ 1 ], "label_filename": "", "is_shape_tensor": false } ], "batch_input": [], "batch_output": [], "optimization": { "priority": "PRIORITY_DEFAULT", "cuda": { "graphs": false, "busy_wait_events": false, "graph_spec": [], "output_copy_stream": true }, "input_pinned_memory": { "enable": true }, "output_pinned_memory": { "enable": true }, "gather_kernel_buffer_threshold": 0, "eager_batching": false }, "sequence_batching": { "oldest": { "max_candidate_sequences": 8, "preferred_batch_size": [ 8 ], "max_queue_delay_microseconds": 100 }, "max_sequence_idle_microseconds": 60000000, "control_input": [ { "name": "START", "control": [ { "kind": "CONTROL_SEQUENCE_START", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "READY", "control": [ { "kind": "CONTROL_SEQUENCE_READY", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "END", "control": [ { "kind": "CONTROL_SEQUENCE_END", "int32_false_true": [ 0, 1 ], "fp32_false_true": [], "data_type": "TYPE_INVALID" } ] }, { "name": "CORRID", "control": [ { "kind": "CONTROL_SEQUENCE_CORRID", "int32_false_true": [], "fp32_false_true": [], "data_type": "TYPE_UINT64" } ] } ] }, "instance_group": [ { "name": "tts_preprocessor_0", "kind": "KIND_GPU", "count": 1, "gpus": [ 0 ], "secondary_devices": [], "profile": [], "passive": false, "host_policy": "" } ], "default_model_filename": "", "cc_model_filenames": {}, "metric_tags": {}, "parameters": { "mapping_path": { "string_value": "/data/models/tts_preprocessor/1/mapping.txt" }, "max_input_length": { "string_value": "2000" }, "abbreviations_path": { "string_value": "/data/models/tts_preprocessor/1/" }, "dictionary_path": { "string_value": "/data/models/tts_preprocessor/1/" }, "language": { "string_value": "en-US" }, "max_sequence_length": { "string_value": "400" } }, "model_warmup": [], "model_transaction_policy": { "decoupled": true } } I1203 04:28:02.116700 75 tts-postprocessor.cc:269] TRITONBACKEND_ModelInstanceInitialize: tts_postprocessor_0 (device 0) I1203 04:28:02.121603 75 tts-preprocessor.cc:206] TRITONBACKEND_ModelInstanceInitialize: tts_preprocessor_0 (device 0) I1203 04:28:02.121729 75 model_repository_manager.cc:1212] successfully loaded 'tts_postprocessor' version 1 I1203 04:28:02.121784 75 model_repository_manager.cc:1212] successfully loaded 'tts_preprocessor' version 1 I1203 04:28:02.154573 75 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 1766, GPU 2622 (MiB) I1203 04:28:02.155053 75 logging.cc:49] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 1766, GPU 2630 (MiB) I1203 04:28:02.155650 75 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1766, GPU 2612 (MiB) I1203 04:28:02.155743 75 logging.cc:49] [MemUsageSnapshot] deserializeCudaEngine end: CPU 1766 MiB, GPU 2612 MiB I1203 04:28:02.155997 75 logging.cc:49] [MemUsageSnapshot] ExecutionContext creation begin: CPU 1766 MiB, GPU 2612 MiB I1203 04:28:02.156425 75 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 1766, GPU 2620 (MiB) I1203 04:28:02.156821 75 logging.cc:49] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 1766, GPU 2628 (MiB) I1203 04:28:02.157284 75 logging.cc:49] [MemUsageSnapshot] ExecutionContext creation end: CPU 1766 MiB, GPU 2766 MiB I1203 04:28:02.157445 75 plan_backend.cc:859] Created instance riva-trt-hifigan_0_0_gpu0 on GPU 0 with stream priority 0 and optimization profile default[0]; I1203 04:28:02.159342 75 model_repository_manager.cc:1212] successfully loaded 'riva-trt-hifigan' version 1 E1203 04:28:02.159370 75 model_repository_manager.cc:1404] Invalid argument: ensemble 'fastpitch_hifigan_ensemble' depends on 'riva-onnx-fastpitch_encoder' which has no loaded version I1203 04:28:02.159402 75 server.cc:504] +------------------+------+ | Repository Agent | Path | +------------------+------+ +------------------+------+ I1203 04:28:02.159439 75 server.cc:543] +------------------------+---------------------------------------------------------------------------------------+--------+ | Backend | Path | Config | +------------------------+---------------------------------------------------------------------------------------+--------+ | tensorrt | | {} | | onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so | {} | | riva_tts_preprocessor | /opt/tritonserver/backends/riva_tts_preprocessor/libtriton_riva_tts_preprocessor.so | {} | | riva_tts_chunker | /opt/tritonserver/backends/riva_tts_chunker/libtriton_riva_tts_chunker.so | {} | | riva_tts_postprocessor | /opt/tritonserver/backends/riva_tts_postprocessor/libtriton_riva_tts_postprocessor.so | {} | +------------------------+---------------------------------------------------------------------------------------+--------+ I1203 04:28:02.159484 75 server.cc:586] +-----------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------+ | Model | Version | Status | +-----------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------+ | riva-onnx-fastpitch_encoder | 1 | UNAVAILABLE: Invalid argument: unable to load model 'riva-onnx-fastpitch_encoder', configuration expects 1 inputs, model provides 3 | | riva-trt-hifigan | 1 | READY | | spectrogram_chunker | 1 | READY | | tts_postprocessor | 1 | READY | | tts_preprocessor | 1 | READY | +-----------------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------+ I1203 04:28:02.159556 75 tritonserver.cc:1718] +----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Option | Value | +----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | server_id | triton | | server_version | 2.13.0 | | server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics | | model_repository_path[0] | /data/models | | model_control_mode | MODE_NONE | | strict_model_config | 1 | | pinned_memory_pool_byte_size | 268435456 | | cuda_memory_pool_byte_size{0} | 1000000000 | | min_supported_compute_capability | 6.0 | | strict_readiness | 1 | | exit_timeout | 30 | +----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ I1203 04:28:02.159563 75 server.cc:234] Waiting for in-flight requests to complete. I1203 04:28:02.159567 75 model_repository_manager.cc:1078] unloading: tts_preprocessor:1 I1203 04:28:02.159594 75 model_repository_manager.cc:1078] unloading: tts_postprocessor:1 I1203 04:28:02.159642 75 model_repository_manager.cc:1078] unloading: riva-trt-hifigan:1 I1203 04:28:02.159716 75 model_repository_manager.cc:1078] unloading: spectrogram_chunker:1 I1203 04:28:02.159766 75 tts-preprocessor.cc:208] TRITONBACKEND_ModelInstanceFinalize: delete instance state I1203 04:28:02.159789 75 tts-preprocessor.cc:205] TRITONBACKEND_ModelFinalize: delete model state I1203 04:28:02.159805 75 tts-postprocessor.cc:271] TRITONBACKEND_ModelInstanceFinalize: delete instance state I1203 04:28:02.159808 75 server.cc:249] Timeout 30: Found 4 live models and 0 in-flight non-inference requests I1203 04:28:02.159904 75 spectrogram-chunker.cc:244] TRITONBACKEND_ModelInstanceFinalize: delete instance state I1203 04:28:02.159923 75 spectrogram-chunker.cc:240] TRITONBACKEND_ModelFinalize: delete model state I1203 04:28:02.160151 75 model_repository_manager.cc:1195] successfully unloaded 'spectrogram_chunker' version 1 I1203 04:28:02.160873 75 tts-postprocessor.cc:268] TRITONBACKEND_ModelFinalize: delete model state I1203 04:28:02.161302 75 model_repository_manager.cc:1195] successfully unloaded 'tts_postprocessor' version 1 I1203 04:28:02.163668 75 logging.cc:49] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU -1, GPU +0, now: CPU 1715, GPU 2718 (MiB) I1203 04:28:02.165411 75 model_repository_manager.cc:1195] successfully unloaded 'tts_preprocessor' version 1 I1203 04:28:02.175239 75 model_repository_manager.cc:1195] successfully unloaded 'riva-trt-hifigan' version 1 > Riva waiting for Triton server to load all models...retrying in 1 second I1203 04:28:03.159940 75 server.cc:249] Timeout 29: Found 0 live models and 0 in-flight non-inference requests error: creating server: Internal - failed to load all models > Riva waiting for Triton server to load all models...retrying in 1 second > Riva waiting for Triton server to load all models...retrying in 1 second > Triton server died before reaching ready state. Terminating Riva startup. Check Triton logs with: docker logs /opt/riva/bin/start-riva: line 1: kill: (75) - No such process