What is the difference between Riva ASR w/wo NIM?

AI & Data Science NVIDIA NIM Models
1

I can find two documents for Riva

Seems like both use docker and triton server. When using Riva, what is the benefit of using NIM?

2

The Riva NIMs are not yet at feature parity with the classic Riva container. With the former, you can edit the config.sh script provided in the Riva Skills Quick Start resource folder to activate any combination of ASR, NMT, and TTS services; choose from among over a dozen ASR languages and 5 TTS languages; and specify your desired model architectures for any of the above services. Once you’ve edited config.sh , run riva_init.sh to download the models and deploy the pipeline, then run riva_start.sh to launch the Riva server.

Conversely, each Riva NIM gives you only one model (and, in the case of ASR and TTS, only one language) at a time. Running multiple Riva services as NIMs simultaneously is tricky, but doable. The simplest method I know is to download the constituent models of each NIM to the same directory and then launch one NIM. I demonstrate how to do so in this dev blog and this video. I imagine one could set up a docker-compose file or a Helm chart to accomplish the same task, but in the former case, you’d almost certainly have to change the port that each NIM except one listens on: by default, each Riva NIM listens on port 50051, just like the classic container.

Here’s an advantage to using Riva NIMs rather than the classic Riva container: If you don’t want to run the Riva server yourself, you can query the hosted endpoint (and associated function ID for each NIM) described in the Try API tab on each Riva NIM’s model page for inference. Moreover, since our entire software development paradigm is shifting toward NIMs, once the Riva NIMs are at feature parity with the classic Riva container, we’ll probably start developing NIMs exclusively. It can’t hurt to start familiarizing oneself with Riva NIMs now, if they support the desired model architectures and languages.

3

Thanks for your clarification. It helped me alot. Thanks!

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.