Question: Riva with kaldi decoder, how to

AI & Data Science Deep Learning (Training & Inference) Riva
1

I have noticed that the Riva pipeline build command has parameters termed “kaldi_decoder” (Pipeline Configuration — NVIDIA Riva), but there is no example how to use this decoder. Any hints?

Having a pre-built Kaldi model, prepared using the Kaldi framework (GitHub - kaldi-asr/kaldi: kaldi-asr/kaldi is the official location of the Kaldi project.) is it possible to use Riva with the kaldi_decoder to serve a Kaldi model?

2

Hi @ilb

Thanks for your interest in Riva

I will check with the internal team and provide you with an example

Thanks

3

Hi @ilb

I have updates from our internal team,

we have de-emphasized the kaldi decoder because we see in our internal tests that the Kaldi decoder underperforms.
Any specific reason for preferring Kaldi decoder?

ASR acoustic models supported by Riva can be found here: Speech Recognition — NVIDIA Riva

Thanks

4

@rvinobha thank you for the reply. I am aware of the official documentation and the list of ASR acoustic models that is listed there in.

Since the official documentation lists the Kaldi decoder as one of possible parameters my question was:

  1. how can I test it (I would like to see on my own if it underperforms in my case as well)?
  2. can the RIVA Kaldi decoder be used to serve a Kaldi AM (not just Nemo or TAO)?

There is no specific reason, apart from the desire to do a test run and evaluate if the decoder underperforms in my case as well (i.e. I have built my own Nemo AM models and I would like to see the difference on those, out of curiosity if nothing else).

Second, and more importantly, I recall that Kaldi ASR inference has been integrated into Triton (Integrating NVIDIA Triton Inference Server with Kaldi ASR | NVIDIA Technical Blog and DeepLearningExamples/README.md at master · NVIDIA/DeepLearningExamples · GitHub), and since Riva is based around Triton and reports a Kaldi decoder, I would like to know if I can use Riva as an inference server for a Kaldi AM. I have Nemo and Kaldi acoustic models, and regardless of the ease of training and final quality of Nemo acoustic models there are certain scenarios where I still find Kaldi acoustic models useful. Until I find a way to cover those with Nemo acoustic models, it would be very nice if I could use Riva as a Kaldi AM inference server as well.

5

Hi @ilb

Apologies for the delay,

I have updates from the team for your follow-up

Riva does not plan to support inference for Kaldi AM in the near future.

Thanks