Failed to deploy citrinet nemo to riva

AI & Data Science Deep Learning (Training & Inference) Riva
1

Please provide the following information when requesting support.

Hardware - GPU V100
Operating System
Riva Version 1.7.0 beta

I tried to custom deploy stt_en_citrinet_256.nemo(STT En Citrinet 256 | NVIDIA NGC) model to riva 1.7.0 and got error while I convert to riva-trt-speech_recognition-am-streaming-offline/1/model_graph.onnx to TRT engine.

2021-12-03 08:58:40,061 [INFO] Building TRT engine from ONNX file
[TensorRT] WARNING: onnx2trt_utils.cpp:362: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] ERROR: 4: [network.cpp::validate::2712] Error Code 4: Internal Error (length: dynamic input is missing dimensions in profile 0.)
2021-12-03 08:58:54,555 [ERROR] Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/servicemaker/cli/deploy.py", line 84, in deploy_from_rmir
    generator.serialize_to_disk(
  File "/opt/conda/lib/python3.8/site-packages/servicemaker/triton/triton.py", line 396, in serialize_to_disk
    module.serialize_to_disk(repo_dir, rmir, config_only, verbose, overwrite)
  File "/opt/conda/lib/python3.8/site-packages/servicemaker/triton/triton.py", line 280, in serialize_to_disk
    self.update_binary(version_dir, rmir, verbose)
  File "/opt/conda/lib/python3.8/site-packages/servicemaker/triton/triton.py", line 676, in update_binary
    with self.build_trt_engine_from_onnx(model_weights) as engine, open(
AttributeError: __enter__

How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)

  1. Download stt_en_citrinet_256.nemo(STT En Citrinet 256 | NVIDIA NGC)
  2. install riva_quickstart_v1.7.0-beta and nemo2riva-1.7.0b0-py3-none-any.whl
  3. run nemo2riva --out stt_en_citrinet_256.riva stt_en_citrinet_256.nemo
  4. run riva servicemaker-dev 
docker run --gpus all -it --rm -v /home/ubuntu/riva/riva_quickstart_v1.7.0-beta/riva_model:/servicemaker-dev -v /home/ubuntu/riva/riva_quickstart_v1.7.0-beta/riva_output:/data --entrypoint="/bin/bash" nvcr.io/nvidia/riva/riva-speech:1.7.0-beta-servicemaker
  1. run riva-build
riva-build speech_recognition /servicemaker-dev/stt_en_citrinet_256.rmir /servicemaker-dev/stt_en_citrinet_256.riva --decoder_type greedy --offline
  1. get stt_en_citrinet_256.rmir file
  2. run riva-deploy
riva-deploy /servicemaker-dev/stt_en_citrinet_256.rmir /data/models
  1. while generate riva-deploy file I got above error.