Please provide the following information when requesting support.
Hardware - GPU : A30
Hardware - CPU : model name : AMD EPYC 7R32
Operating System: ubuntu
Riva Version: 2.5.0
TLT Version (if relevant)
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)
When serving a conformer-ctc model finetuned with TAO, I only get empty transcripts back from riva server.
When I tried inference on the finetuned tlt model using tao, transcript was returned just fine.
I was able to run riva_build.sh and riva_deploy.sh
with the new flag in riva_build --nn.use_trt_fp32
(See previous post for full script).
However, I’m now having a problem with riva_start.sh - not sure why?
(Script included here and hsa not changed from previous post.
Below is a link with the following:
riva_start.sh
directoy listing of /data
docker logs riva-speech
One question is where do I need to keep the .rmir file generated by riva_build?
To be sure I made a copy in /data and /data/models
This line from the attached log sounds problematic:
Unavailable: unable to find '/data/models/riva-trt-conformer-en-gb-asr-offline-am-streaming-offline/1/model.plan' for model instance 'riva-trt-conformer-en-gb-asr-offline-am-streaming-offline_0'
Does the file exist?
You don’t need to copy the rmir file to the models directory, it only needs to sit in the $riva_model_loc/rmir at the time you’re deploying it. Technically, you can even remove it after the deployment. During the deployment, the artifacts are extracted and optimized and stored in the models dir.
For the deployment, I used the riva_init.sh script (as described here. I found it easier than using the riva-deploy command.
@petra1 I saw that also - I don’t know where to get that from or how to make it. I did notice that I do have that file for the en-us conformer model (i.e. the “oob” conformer model).
Any idea if I can download something from the NGC catalog and pass an option in riva-build to make the file or if I need to find that file from somewhere in the NGC catalog? Any leads would be appreciated!