Riva: Node.JS Examples

Hardware - GPU T4 and RTX3090
Hardware - CPU AMD Ryzen 5700x
Operating System: Ubuntu 20.04
Riva Version: 1.6.0-beta

Are there any examples on how to use the provided Riva Protobufs with the Riva server?

I was able to get the Python examples working out of the box, but when I try to run a Node.JS client, I only get I1028 22:45:52.961102 942 grpc_riva_asr.cc:799] ASRService.StreamingRecognize called. in the server logs and no response.

Here’s an excerpt of my code:

const grpc = require('@grpc/grpc-js');

const rivaMessages = require('./compiled/riva_asr_pb');
const rivaServices = require('./compiled/riva_asr_grpc_pb');
const rivaAudioMessages = require('./compiled/riva_audio_pb');
const fs = require('fs');

const audioFile = '/path/to/file.wav';
const serverAddress = '*****:50051';

let client = new rivaServices.RivaSpeechRecognitionClient(
    serverAddress, grpc.credentials.createInsecure()
);
let config = new rivaMessages.RecognitionConfig()
config.setEncoding(rivaAudioMessages.AudioEncoding.LINEAR_PCM);
config.setSampleRateHertz(16000);
config.setMaxAlternatives(1);
config.setLanguageCode('en-US');
let streamingConfig = new rivaMessages.StreamingRecognitionConfig();
streamingConfig.setConfig(config);
streamingConfig.setInterimResults(true);

let streamingRequest = new rivaMessages.StreamingRecognizeRequest()
streamingRequest.setStreamingConfig(streamingConfig);

client.streamingRecognize(streamingRequest, function (err, response) {
  if (err) {
    console.log(err);
  }
  console.log(response);
});

const stream = fs.createReadStream(audioFile);
stream.on('data', function (data) {
  let streamingRequest = new rivaMessages.StreamingRecognizeRequest()
  streamingRequest.setAudioContent(data);
  let response = client.streamingRecognize(streamingRequest, function (err, response) {
    if (err) {
      console.log(err);
    }
    console.log(response.results);
  });
  console.log(response);
})
stream.on('end', () => {
  console.log('Done');
})

The protos are compiled like so:

grpc_tools_node_protoc --proto_path=./protos \
  --js_out=import_style=commonjs,binary:compiled \
  --grpc_out=grpc_js:compiled \
  protos/riva_*.proto

Can the Riva team simply compile the protos into the different languages for us in the next release (like the Python wheel) and maybe even publish them? It will help with development a lot.

Update:

I was able to start communicating with the server by making my code follow the structure for bi-directional streaming gRPC

...
// Converted `streamingRecognize` from callback to event based
let call = client.streamingRecognize();
call.on('data', function (data) {
  console.log(data);
})
call.on('end', function () {
  console.log('End');
})
call.on('error', function (e) {
  console.log(e)
})

call.write(streamingRequest)  // See definition original response

const stream = fs.createReadStream(audioFile);
stream.on('data', function (data) {
  let streamingRequest = new rivaMessages.StreamingRecognizeRequest()
  streamingRequest.setAudioContent(data);
  call.write(streamingRequest);  // This is how data is written.
})
stream.on('end', () => {
  console.log('Stream End');
})

However, now I get the following error.

# Client
"3 INVALID_ARGUMENT: Error: config format doesn't match with header format"
# Server
E1029 13:28:05.143010  8354 grpc_riva_asr.cc:965] Error: config format doesn't match with header format

How should I be defining the streaming config?

Hi,

Currently we do not have node js examples, Please refer following for more details which may help you.
https://docs.nvidia.com/deeplearning/riva/user-guide/docs/development-grpc.html
https://docs.nvidia.com/deeplearning/riva/user-guide/docs/protobuf-api/protobuf-api-root.html#riva-proto-riva-asr-proto

Thank you.

Update:

I was able to get things working :)

Hopefully this thread can help others and maybe even be modified by the Riva team to be used as a sample/collect feedback.

It looks like the Riva server does not do any pre-processing of audio file headers and so if I want to transcribe a wav file in Node, I need to strip the headers.

In general, wav files have 44 bytes as part of the header, so I initialized my read file stream as such:

// Note the `highWaterMark` is the chunk size and the `start` is defined as 44,
// the size of the wav header in bytes
const stream = fs.createReadStream(audioFile, { highWaterMark: 1024, start: 44 });

It would be great if either:
a. The Riva service does this automagically (like how Google and Azure APIs work).
b. Highlight this in the documentation somewhere that audio file headers need to be stripped before sending the data over to Riva.

2 Likes