We have fine-tuned an ASR model with “tao speech_to_text_conformer fine-tune”. Further, we want to deploy it using Riva, hence an export was done with “tao speech_to_text_conformer export”. To build this model further, we are trying to do riva-build and deploy.
When we are using riva-deploy, we are getting below error:
- when importing initializer: onnx::MatMul_6791
[01/02/2023-14:32:31] [TRT] [E] parsers/onnx/ModelImporter.cpp:745: ERROR: parsers/onnx/ModelImporter.cpp:106 In function parseGraph:
[8] Assertion failed: convertOnnxWeights(initializer, &weights, ctx) && “Failed to import initializer.”
[01/02/2023-14:32:31] [TRT] [E] [network.cpp::addInput::1595] Error Code 3: API Usage Error (Parameter check failed at: optimizer/api/network.cpp::addInput::1595, condition: inName != knownInput->getName()
)
[01/02/2023-14:32:31] [TRT] [E] parsers/onnx/ModelImporter.cpp:745: ERROR: parsers/onnx/ModelImporter.cpp:106 In function parseGraph:
[8] Assertion failed: convertOnnxWeights(initializer, &weights, ctx) && “Failed to import initializer.”
[01/02/2023-14:32:31] [TRT] [E] parsers/onnx/ModelImporter.cpp:745: ERROR: audio_signal:269 In function importInput:
[8] Assertion failed: *tensor && “Failed to add input to the network.”
2023-01-02 14:32:31,252 [INFO] Mixed-precision net: 0 layers, 0 tensors, 0 outputs…
2023-01-02 14:32:31,252 [ERROR] Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/servicemaker/cli/deploy.py”, line 100, in deploy_from_rmir
generator.serialize_to_disk(
File “/usr/local/lib/python3.8/dist-packages/servicemaker/triton/triton.py”, line 445, in serialize_to_disk
module.serialize_to_disk(repo_dir, rmir, config_only, verbose, overwrite)
File “/usr/local/lib/python3.8/dist-packages/servicemaker/triton/triton.py”, line 311, in serialize_to_disk
self.update_binary(version_dir, rmir, verbose)
File “/usr/local/lib/python3.8/dist-packages/servicemaker/triton/asr.py”, line 159, in update_binary
RivaTRTConfigGenerator.update_binary_from_copied(self, version_dir, rmir, copied, verbose)
File “/usr/local/lib/python3.8/dist-packages/servicemaker/triton/triton.py”, line 738, in update_binary_from_copied
bindings = self.build_trt_engine_from_onnx(model_weights, engine_path=trt_file, verbose=verbose)
File “/usr/local/lib/python3.8/dist-packages/servicemaker/triton/triton.py”, line 646, in build_trt_engine_from_onnx
network = fix_fp16_network(network)
File “/usr/local/lib/python3.8/dist-packages/servicemaker/triton/trt_bindings.py”, line 249, in fix_fp16_network
sys.setrecursionlimit(len(network_definition))
ValueError: recursion limit must be greater or equal than 1
The riva-build and deploy worked fine when we used the pre-trained Nemo model (stt_hi_conformer_ctc) but with our fine-tuned model, there is an error while Riva deploy.
Steps that we performed for model fine-tuning and deployment:
-
tao speech_to_text_conformer create_tokenizer
-
tao speech_to_text_conformer finetune
-
tao speech_to_text_conformer export (export_format=RIVA)
-
inside the Riva service maker
-
riva-build speech_recognition
-
riva-deploy
Kindly, provide some guidance for the issue.