Riva 2.0 : Streaming scripts not working

AI & Data Science Deep Learning (Training & Inference) Riva
,
1

Riva 2.0.0 comes with default inferencing scripts for ASR/NLP/TTS.
There are streaming scipts also available like - talk_stream.py and transcribe_mic.py

I have downloaded the default models for ASR/TTS (en-US) and tried the python streaming scipts, there I got errors.

USB headset are connected to the system seen at no 24 by using the code -
python3 /work/examples/transcribe_mic.py –list-devices

15: tegra-snd-t19x-mobile-rt565x: - (hw:1,11)
16: tegra-snd-t19x-mobile-rt565x: - (hw:1,12)
17: tegra-snd-t19x-mobile-rt565x: - (hw:1,13)
18: tegra-snd-t19x-mobile-rt565x: - (hw:1,14)
19: tegra-snd-t19x-mobile-rt565x: - (hw:1,15)
20: tegra-snd-t19x-mobile-rt565x: - (hw:1,16)
21: tegra-snd-t19x-mobile-rt565x: - (hw:1,17)
22: tegra-snd-t19x-mobile-rt565x: - (hw:1,18)
23: tegra-snd-t19x-mobile-rt565x: - (hw:1,19)
24: USB Lavalier Microphone: Audio (hw:2,0)

For ASR script, python /work/examples/transcribe_mic.py –input-device=24

image
image1211×378 104 KB

For TTS, python /work/examples/talk_stream.py --output-device=24

image
image1214×346 91 KB

The above error is managed by chnaging samplerate from 44100 to 48000. But I got this below error.

image
image1214×396 112 KB

Please provide the following information when requesting support.
Hardware - GPU : Nvidia Xavier NX
Hardware - CPU : Nvidia Xavier NX
Operating System : Jetpack 4.6
Riva Version : 2.0.0

Thanks.

2

Hi @user115721,

Could you please share docker logs riva-speech output log file as well so that we can help better?

Thanks

3

Here is the logs -

xyz@xyz-desktop:~/riva_2.0/riva_quickstart_arm64_v2.0.0$ sudo docker logs riva-speech
I0531 06:04:24.192103 8 riva_server.cc:118] Using Insecure Server Credentials
E0531 06:04:24.200701 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:04:34.201865 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:04:44.203661 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:04:54.205415 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:05:04.505347 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:05:14.781774 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:05:24.854699 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:05:35.197554 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:05:45.198709 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:05:55.199358 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:06:05.199926 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
E0531 06:06:15.200497 8 model_registry.cc:238] error: unable to get server status: failed to connect to all addresses
I0531 06:06:25.246552 8 model_registry.cc:112] Successfully registered: citrinet-256-en-US-streaming for ASR
I0531 06:06:25.441211 8 model_registry.cc:112] Successfully registered: riva-punctuation-en-US for NLP
I0531 06:06:25.457564 8 model_registry.cc:112] Successfully registered: riva_intent_misty for NLP
I0531 06:06:25.770545 8 model_registry.cc:112] Successfully registered: riva-punctuation-en-US for NLP
I0531 06:06:25.786753 8 model_registry.cc:112] Successfully registered: riva_intent_misty for NLP
I0531 06:06:25.824905 8 model_registry.cc:112] Successfully registered: fastpitch_hifigan_ensemble-English-US-Female-1 for TTS
I0531 06:06:25.876750 8 riva_server.cc:158] Riva Conversational AI Server listening on 0.0.0.0:50051
W0531 06:06:25.876927 8 stats_reporter.cc:40] No API key provided. Stats reporting disabled.
I0531 06:07:31.792773 247 grpc_riva_tts.cc:466] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:07:31.793319 247 grpc_riva_tts.cc:502] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference
I0531 06:07:31.793359 247 libriva_tts.cc:166] Entering RivaTrtisTTSClient::SendStreaming for input “my name is neha”
I0531 06:08:10.163928 247 grpc_riva_tts.cc:300] TTSService.Synthesize called.
I0531 06:08:10.213165 247 grpc_riva_tts.cc:327] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:08:10.880211 247 grpc_riva_tts.cc:414] TTSService.Synthesize response contains 52992 audio samples.
I0531 06:09:39.281078 247 grpc_riva_tts.cc:300] TTSService.Synthesize called.
I0531 06:09:39.284024 247 grpc_riva_tts.cc:327] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:09:41.114997 247 grpc_riva_tts.cc:414] TTSService.Synthesize response contains 29696 audio samples.
I0531 06:10:57.295639 247 grpc_riva_tts.cc:300] TTSService.Synthesize called.
I0531 06:10:57.338589 247 grpc_riva_tts.cc:327] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:10:59.148877 247 grpc_riva_tts.cc:414] TTSService.Synthesize response contains 56576 audio samples.
I0531 06:11:39.102185 247 grpc_riva_tts.cc:300] TTSService.Synthesize called.
I0531 06:11:39.105382 247 grpc_riva_tts.cc:327] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:11:41.144372 247 grpc_riva_tts.cc:414] TTSService.Synthesize response contains 49920 audio samples.
I0531 06:11:51.428658 233 grpc_riva_tts.cc:300] TTSService.Synthesize called.
I0531 06:11:51.430666 233 grpc_riva_tts.cc:327] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:11:51.901371 233 grpc_riva_tts.cc:414] TTSService.Synthesize response contains 74496 audio samples.
I0531 06:12:02.035853 247 grpc_riva_tts.cc:300] TTSService.Synthesize called.
I0531 06:12:02.037200 247 grpc_riva_tts.cc:327] Using model fastpitch_hifigan_ensemble-English-US-Female-1 for inference with speaker_id: -1
I0531 06:12:02.316884 247 grpc_riva_tts.cc:414] TTSService.Synthesize response contains 59136 audio samples.

4

@SunilJB I have found that -
python3 talk.py --output-device=24 is working fine for streaming TTS pipeline whereas talk_stream.py gives error. I think, talk_stream and transcribe_mic scripts is broken.