Connecting Riva TTS with Audio 2 Face Streaming

AI & Data Science Digital Humans (closed) Audio2Face (closed)
1

Please provide the following information when requesting support.

Workstation
Hardware - GPU (GeForce RTX 2070)
Hardware - CPU
Operating System - Win 10
Riva Version 2.14.0
TLT Version (if relevant)
How to reproduce the issue ? (This is for errors. Please share the command and the detailed log here)

I’m trying to connect the riva-speech skills 2.14.0 with Audio 2 face 2023.2.0

  • In Audio 2 face i have enabled the Riva TTS plugin
    – I have created a streaming audio player and connected it to the time attributes in the action graph
    – I have set the URL for TTS to localhost:50051
    – I have set the URL for A2F Player to localholst:50051
    – I have set the Player instance to the player streaming audio player that was created

  • I have riva client and server up and running on docker

  • I have test sent a text from A2F and i can see it is being recieved by the Riva server. Logs below

2024-03-05 11:04:48 I0305 10:04:48.798074 11564 grpc_riva_tts.cc:502] TTSService.SynthesizeOnline called.
2024-03-05 11:04:48 I0305 10:04:48.798333 11564 grpc_riva_tts.cc:530] Using model fastpitch_hifigan_ensemble-English-US for inference with speaker_id: 0
2024-03-05 11:04:48 I0305 10:04:48.798460 11564 grpc_riva_tts.cc:567] Using model fastpitch_hifigan_ensemble-English-US for inference
2024-03-05 11:04:51 I0305 10:04:51.577692 11564 grpc_riva_tts.cc:597] TTSService.SynthesizeOnline returning OK
2024-03-05 11:04:51 I0305 10:04:51.578358 11564 stats_builder.h:164] {"specversion":"1.0","type":"riva.tts.synthesizeonline.v1","source":"","subject":"","id":"96e3e701-0766-48fe-a3a3-13bcecdff880","datacontenttype":"application/json","time":"2024-03-05T10:04:48.798039861+00:00","data":{"release_version":"2.14.0","customer_uuid":"","ngc_org":"","ngc_team":"","ngc_org_team":"","container_uuid":"","language_code":"en-US","request_count":1,"total_characters":5,"audio_duration":0.6733787059783936,"status":0,"err_msg":""}}

I’m thinking i might be connecting something wrong in with my setup in A2F
Or
I’m not listening in on the correct port for the return stream

2

Hi @W1r3d do you get any errors? I’m trying to understand what the issue is.

3

The error is that i’m not seeing the Riva TTS audio stream playing in Audio 2 Face - Streaming Audio Player

4

Have you checked this tutorial?

1 Like
5

Yes i’ve gone through that tutorial

  • I’ve set the streaming up in the same manner as in the tutorial. Except i have localhost:50051 in both the TTS and A2F Player boxes.
    image
  • It doesn’t work with the Riva Quickstart 2.14.0 Server it seems. (will post error msg in VS Code when i get home)
  • I do not have access to the demo scene that is being used so i cannot try it out(the menu option does not exist for me)
  • The script to start a local sever that they show in the video is giving me errors(will post when i get home)

I have the whole Riva Quickstart 2.14.0 setup deployed in Docker. Would be awesome if i could re-route everything through there.

6

The IP is localhost when the server is local. Default Riva port is 50051 which conflicts with A2F Player URL.

To fix this, we can modify riva_speech_api_port variable in ~/nvidia/riva_quickstart_v2.14.0/config.sh. You could use 50052 or any other ports besides 50051.

Make sure to riva_stop.sh --> edit port --> riva_start.sh

1 Like
7

That worked like a charm!

image
image410×527 21.2 KB

image

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.