About Riva ASR Concurrency performance

AI & Data Science Deep Learning (Training & Inference) Riva
1

Hardware - GPU (T4: tegra23x )
Hardware - Jetson AGX Orin Develop kit
Operating System
ubuntu 20.04 focal, Jetpack 5.1.2

I am trying to understand concurrent Audio streams using Nvidia RIVA for transcription. Currently I use one Jetson AGX Orin 32G ram device to run riva_asr_client in docker. I get below result as the following pic:

d066e5c68d2d38a52d5ff8f49513963
d066e5c68d2d38a52d5ff8f495139631920×617 15.9 KB

I run 20 asr requests and riva takes 4.6 secs to complete all requests then return,
but I run 1 asr requests and riva takes 1.05 secs to complete .

Comparing the two results, it seems that riva will not return the results until it has processed multiple asr requests. I would like to ask if Riva Speech supports simultaneous speech recognition and translation for multiple people on Jetson orin devices?

2

Hi @fuchun.mai

Thanks for your interest in Riva

It is currently not supported

Thanks

3

@rvinobha
Concurrent Audio streams using Nvidia RIVA for transcription is not supported on Jetson AGX Orin。 For GPU server, Concurrent Audio streams using Nvidia RIVA is also not supported?

Thx