TensorRT Build Failure with Silero VAD ONNX Export from NeMo for Riva 2.18.0 (Int32/Int64 Mismatch)

AI & Data Science Deep Learning (Training & Inference) Riva
,
1

I successfully deployed Silero VAD with my custom ASR model on Riva 2.18.0, but I encountered a TensorRT build issue I’d like to resolve.

Current setup
Im using NeMo 24.09 with nemo2riva 2.18.0 and Riva 2.18.0.
VAD build is working with --nn.use_onnx_runtime flag but TensorRT build fails with type mismatch error during riva-deploy.

Current workaround
I am using the ONNX Runtime backend (--nn.use_onnx_runtime), which works but results in about 2–3x slower inference compared to TensorRT.

Question
Which NeMo version exports ONNX models with consistent integer types compatible with Riva 2.18.0’s TensorRT builder? I need TensorRT performance for production.

I have already tried NeMo 24.09 and encountered an Int32/Int64 mismatch. Should I try:

  • NeMo 23.11

  • NeMo 24.01

  • NeMo 24.05

Or is there an official NeMo/Riva compatibility matrix that I should reference?

3

@Vaneeza_Ahmad based on your other topic ( Silero VAD Integration with custom ASR model using riva-build in Riva 2.17.0 - #7 by Vaneeza_Ahmad ), I will assume you’re trying to do a datacenter deployment. And that you have a GPU with compute capability >7.5.

In my experience the best approach to deploy a custom ASR combined with Silero VAD served via TensorRT is to download the prebuilt Silero VAD from NGC in combination with a build via the latest (1.18.0) NIM container (e.g. Parakeet CTC 1.1b English (En-US)), just follow the instructions provided in the documentation. This will use the latest Riva 2.24.0. Note that the NIM for the 0.6b English model is currently still on Riva 2.22.0 (e.g. if you need to deploy on a Turing architecture) .

Note that from Riva 2.19.0 to Riva 2.24.0 important bugs have been resolved and new features added, so it is not worth staying on Riva 2.19.0, if not really necessary.