No streaming/live transcription feature for Whisper on the Riva?

AI & Data Science Deep Learning (Training & Inference) Riva
1

Please provide the following information when requesting support.

Hardware - GPU (A10)
Hardware - CPU
Operating System: Ubuntu 22.04
Riva Version: 2.18
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here):
When I try to run --list_models to inference live on whisper, whisper does not appear at all. Meanwhile it appears for offline asr binaries.
Streaming binary:
root@xfusion-server:/opt/riva# ./clients/riva_streaming_asr_client --list_models
I0120 07:24:11.533994 257 grpc.h:94]
Offline binary:
root@xfusion-server:/opt/riva# ./clients/riva_asr_client --list_models
I0120 07:26:07.435967 261 grpc.h:94] Using Insecure Server Credentials
‘en,zh,de,es,ru,ko,fr,ja,pt,tr,pl,ca,nl,ar,sv,it,id,hi,fi,vi,he,uk,el,ms,cs,ro,da,hu,ta,no,th,ur,hr,bg,lt,la,mi,ml,cy,sk,te,fa,lv,bn,sr,az,sl,kn,et,mk,br,eu,is,hy,ne,mn,bs,kk,sq,sw,gl,mr,pa,si,km,sn,yo,so,af,oc,ka,be,tg,sd,gu,am,yi,lo,uz,fo,ht,ps,tk,nn,mt,sa,lb,my,bo,tl,mg,as,tt,haw,ln,ha,ba,jw,su,yue,multi’: ‘whisper-large-v3-multi-asr-offline-asr-bls-ensemble’
Needed to confirm if whisper doesn’t support the live streaming support and if not, is there an alternate?

2

For streaming, Parakeet 1.1B is the recommended ASR model to use :

3

Just to clarify: It does work for arabic too, right? Because in the drop down I see en-us only.

4

@amargolin
details = “OfflineAsrEnsemble expects both start and end flags to be 1”
debug_error_string = “UNKNOWN:Error received from peer ipv6:%5B::1%5D:50051 {grpc_message:“OfflineAsrEnsemble expects both start and end flags to be 1”, grpc_status:3, created_time:“2025-01-23T11:14:00.468103098+03:00”}”
Also, I get this error when I run this command through the python client:
python scripts/asr/transcribe_file_.py --model-name ‘canary-1b-multi-asr-offline-asr-bls-ensemble’ --input-file ‘Paragraph_test.wav’ --language-code ar

5

Did you try the Multilingual version?
You can choose from the concat or unified tokenizer.

1 Like
6

Thanks I will try, good day!

1 Like