Deploying Custom Model in RIVA via NodeJS GRPC

AI & Data Science Deep Learning (Training & Inference) Riva
1

Hardware - GPU (A100/A30/T4/V100) Quadro RTX 8000
Hardware - CPU Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz
Operating System - Ubuntu 18.04.5 LTS
Riva Version - 1.9 beta
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)
Hi all, I have been following the Quick Start Guide to deploy my custom trained citrinet and quartznet model.
I have also specified the model used in my js client using the GRPC model parameter riva_asr.proto
Upon running my node js client, it seems that the ASRService.StreamingRecognize is called, but no model is selected for reference and no transcription is happening.
However, when i change the model parameter back the ‘riva-asr’ model, the client works as usual and I am able to successfully transcribe the audio file
Here is my docker logs and excerpt of my nodejs client for reference :
dockerlogs.txt (35.1 KB)
let config = new rivaMessages.RecognitionConfig()
config.setEncoding(rivaAudioMessages.AudioEncoding.LINEAR_PCM);
// Need to set sample rate hertz according to the sample rate of the audio file
config.setSampleRateHertz(48000);
config.setMaxAlternatives(1);
config.setLanguageCode(‘en-US’);
config.setModel(‘name_of_model’);
let streamingConfig = new rivaMessages.StreamingRecognitionConfig();
streamingConfig.setConfig(config);
streamingConfig.setInterimResults(true);

let streamingRequest = new rivaMessages.StreamingRecognizeRequest();
streamingRequest.setStreamingConfig(streamingConfig);

2

For reference, the models I want to run are 'riva-onnx-riva-asr-am-streaming" and “riva-trt-riva-asr-am-streaming”

3

Like I mentioned in my other post, you need to use streaming callbacks.
You should read the gRPC docs: Basics tutorial | Node | gRPC

  1. Create a streaming STT client:
// I'm using SSL, but you can create an insecure client too
// const clientCredentials = credentials.createInsecure();
const clientCredentials = credentials.createSsl();
sttClientImpl = new RivaSpeechRecognitionClient(SERVER_URL, clientCredentials);
  1. Create a gRPC stream you can listen to:
stream = sttClientImpl.streamingRecognize();
  1. Listen to data. Other streams beside data are error, status, close, and end.
// onData is a callback function when data is received from Riva
stream.on('data', onData);

Here’s an example too: How to Effectively use GRPC Streams in NodeJS | Aditya’s Blog

4

I managed to get the streaming client and callbacks to work.
The problem is specifying the model that I want to use which I managed to solve the problem by separating the citrinet and quartznet models into one folder each.
I then changed the riva_model_loc depending on which model I want to use.