Riva providing empty transcriptions for a few audios, but nemo does not for those audios

Hardware - GPU (T4)
Hardware - CPU
Operating System
Riva Version: 2.6.0
TLT Version (if relevant)
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)

Hello everyone again!
After the riva model deployment, I’m generating transcriptions from my audio files. It’s working fine.
I wanted head to head comparison of the raw .nemo model with the riva model repositories. But I’ve noticed that, in few cases like shorter audios(not applicable for all the shorter audios) I’m not getting transcriptions, where I was getting from raw .nemo model. I’ve tried with both Quick Start Scripts way and Docker way, but with no luck. Both of these give same results, as expected.

Here’s the command I used to build rmir model:
riva-build speech_recognition /servicemaker-dev/<output name of rmir model> /servicemaker-dev/<riva model name> --name=conformer-bn-BD-asr-streaming --featurizer.use_utterance_norm_params=False --featurizer.precalc_norm_time_steps=0 --featurizer.precalc_norm_params=False --ms_per_timestep=40 --endpointing.start_history=200 --nn.fp16_needs_obey_precision_pass --endpointing.residue_blanks_at_start=-2 --chunk_size=0.16 --left_padding_size=1.92 --right_padding_size=1.92 --decoder_type=flashlight --decoding_language_model_binary=<lm_binary> --decoding_vocab=<decoder_vocab_file> --flashlight_decoder.lm_weight=0.2 --flashlight_decoder.word_insertion_score=0.2 --flashlight_decoder.beam_threshold=20. --language_code=bn-BD

What could be the underlying cause for not receiving transcriptions after the model transformation? Is it related to VAD(voice activity detection) somehow? In that case, how to control this in riva model built?

Thanks in advance!

Hi @shuvohishab

Thanks for your interest in Riva,

We have a known issue with conformer models,
Can you please try adding the following flag in your riva build command
--nn.use_trt_fp32

and let us know if it helps

Thanks

1 Like

Hi @rvinobha, I tried with your given solution.

After deploying step, I tried to start riva server, but it was getting killed due to the following error shown in the following screenshot. If you notice, the last model file is not available.

I also checked my local directory, the file isn’t available indeed. I repeated the build stage part, but this file wasn’t being generated for unknown reason.
What might be the issue here?

@rvinobha, sorry to mention you again!
Does this screenshot make sense to you? I’m stuck here.

Hi @shuvohishab

Apologies for the delay,

Request to kindly try

bash riva_clean.sh

And try the setup again and let me know if it works

Thanks