Error when running Conformer-CTC model in Riva 1.8.0b0

Hardware - GPU RTX3090
Hardware - Xeon(R) Silver 4216
Operating System - Ubuntu Server 18.04
Riva Version 1.8.0b0

Hellow! I used the standard nemo model “stt_en_conformer_ctc_large”.
This model was converted from .nemo to .riva format using “nemo2riva-1.8.0b0-py3-none-any.whl” script
I run Riva build with parameters for Conformer-CTC model, build finished with no errors:

riva-build speech_recognition \
--force \
/servicemaker-dev/stt_en_conformer_ctc_large.rmir \
/servicemaker-dev/stt_en_conformer_ctc_large.riva \
--name=test_conformer \
--decoder_type=greedy \
--chunk_size=0.8 \
--padding_size=1.6 \
--ms_per_timestep=40 \
--nn.use_onnx_runtime \
--greedy_decoder.asr_model_delay=-1 \
--vad.residue_blanks_at_start=-2 \
--featurizer.use_utterance_norm_params=False \
--featurizer.precalc_norm_time_steps=0 \
--featurizer.precalc_norm_params=False

But when running the stream recognition example in the “riva-speech-client” container , I get the error:

!python3 …/examples/riva_streaming_asr_client.py --input-file=out.wav

Number of clients: 1
Number of iteration: 1
Input file: out.wav
File duration: 82.66s
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "../examples/riva_streaming_asr_client.py", line 137, in asr_client
    print_to_file(responses, output_file, max_alternatives, word_time_offsets)
  File "../examples/riva_streaming_asr_client.py", line 53, in print_to_file
    for response in responses:
  File "/usr/local/lib/python3.8/dist-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/usr/local/lib/python3.8/dist-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNKNOWN
	details = "in ensemble 'test_conformer', onnxruntime execute failure 6: Non-zero status code returned while running Reshape node. Name:'Reshape_349' Status Message: /workspace/onnxruntime/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:36 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, std::vector<long int>&, bool) size != 0 && (input_shape.Size() % size) == 0 was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,101,512}, requested shape:{16,-1,8,64}
"
	debug_error_string = "{"created":"@1640329690.215318026","description":"Error received from peer ipv4:127.0.0.1:50051","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"in ensemble 'test_conformer', onnxruntime execute failure 6: Non-zero status code returned while running Reshape node. Name:'Reshape_349' Status Message: /workspace/onnxruntime/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:36 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, std::vector<long int>&, bool) size != 0 && (input_shape.Size() % size) == 0 was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,101,512}, requested shape:{16,-1,8,64}\n","grpc_status":2}"
1 threads done, output written to output_<thread_id>.txt

Logs “sudo docker logs riva-speech” output: logs.txt (33.5 KB)

Hi @sltl,

Thanks for sharing the info and logs. I’ll reach out to the team and follow up with you. Folks are on holiday this week so response time may be delayed.

Hello, is there any news on my problem?

I also note that when installing the finished Conformer model, the Citrinet model is downloaded and installed.

"${riva_ngc_org}/${riva_ngc_team}/rmir_asr_conformer_en_us_str:${riva_ngc_model_version}"

It looks like support for the Conformer of models was added by the method of renaming Citrinet to the Conformer, from this and all the problems: D

The problem was solved by a clean installation of Nemo1.6.0rc0 in the NGC 21.12 Pytorch container.

1 Like