Finetuned ASR conformer returns only empty transcripts

Please provide the following information when requesting support.

Hardware - GPU : A30
Hardware - CPU : model name : AMD EPYC 7R32
Operating System: ubuntu
Riva Version: 2.5.0
TLT Version (if relevant)
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)

When serving a conformer-ctc model finetuned with TAO, I only get empty transcripts back from riva server.

When I tried inference on the finetuned tlt model using tao, transcript was returned just fine.

I generated the rmir file using this command:

    riva-build speech_recognition \
    <rmir_name>:<key> \
    <riva_name>:<key>  \
       --name=<pipeline_name>\
       --ms_per_timestep=80 \
       --featurizer.use_utterance_norm_params=False \
       --featurizer.precalc_norm_time_steps=0 \
       --featurizer.precalc_norm_params=False \
       --vad.residue_blanks_at_start=-2 \
       --chunk_size=0.8 \
       --left_padding_size=1.6 \
       --right_padding_size=1.6 \
       --decoder_type=greedy \
       --language_code=en-US 

Riva initializes and starts the server fine, my model is loaded, but returned transcripts are empty.

Are there any options to start the riva server with increased logging to debug these types of issues?

I found a few similar issues in the forums but none of the solutions would fix the issue for me.

Hi @petra1

Thanks for your interest in Riva

Can you share the following details (if possible)

  1. finetuning command used with conformer-ctc-model in TAO, i.e referred documentaion for the finetuning
  2. If possible, please share the fine tuned model directly with us via google drive, Onedrive etc (Please let me know will share my email address)
  3. Complete log output of riva-build command

Thanks

Hello, have you had a chance to get a feedback regarding this issue?
Thanks

HI @petra1

My Apologies,

I will try to provide update by next week for sure

Thanks

Hi @rvinobha and @petra1, I have similar problems - empty transcript after riva-build, riva-deploy, and riva-start. I shared complete scripts and outputs as well as docker log riva-speech in my post: No ASR text output after building riva-build to use en-GB, and the running riva-start

Please let me know if/what I’m doing wrong.
Thanks!

HI @petra1 and @edmond4

Apologies for the delay

This issue will be fixed in future

Now for workaround, Can we try the flag --use_trt_fp32 in riva build command and let us know if it works

Thanks for your patience

Hello @rvinobha ,
thank you for your help.

The solution helped with one change: instead of --use_trt_fp32 I had to use --nn.use_trt_fp32 as a part of my riva-build call.

Regards,
Petra

Hi @petra1

yes that’s absolutely correct,

Thanks for the feedback

Thanks @petra1

I was able to run riva_build.sh and riva_deploy.sh
with the new flag in riva_build --nn.use_trt_fp32
(See previous post for full script).
However, I’m now having a problem with riva_start.sh - not sure why?
(Script included here and hsa not changed from previous post.

Below is a link with the following:

  1. riva_start.sh
  2. directoy listing of /data
  3. docker logs riva-speech

One question is where do I need to keep the .rmir file generated by riva_build?
To be sure I made a copy in /data and /data/models

Here’ the link:

This line from the attached log sounds problematic:

Unavailable: unable to find '/data/models/riva-trt-conformer-en-gb-asr-offline-am-streaming-offline/1/model.plan' for model instance 'riva-trt-conformer-en-gb-asr-offline-am-streaming-offline_0'

Does the file exist?

You don’t need to copy the rmir file to the models directory, it only needs to sit in the $riva_model_loc/rmir at the time you’re deploying it. Technically, you can even remove it after the deployment. During the deployment, the artifacts are extracted and optimized and stored in the models dir.

For the deployment, I used the riva_init.sh script (as described here. I found it easier than using the riva-deploy command.

@petra1 I saw that also - I don’t know where to get that from or how to make it. I did notice that I do have that file for the en-us conformer model (i.e. the “oob” conformer model).

Any idea if I can download something from the NGC catalog and pass an option in riva-build to make the file or if I need to find that file from somewhere in the NGC catalog? Any leads would be appreciated!

@edmond4 moving this discussion back to your thread No ASR text output after building riva-build to use en-GB, and the running riva-start - #5 by edmond4 as it has more context

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.