Send Audio Stream to A2F over networking

vcer · May 9, 2024, 7:46am

Hello

UE 5.3
Live Link PlugIn for UE
A2F 2023.2.0

A2F runs in headless mode. I have successfully implemented live face animation in UE by calling /A2F/Player/SetTrack API to supply the audio piece by piece.

/A2F/Player/SetTrack API accepts a path to an audio file(*.wav) in local filesystem. This means that A2F cannot be deployed remotely. I intend to deploy A2F in Linux docker container accessed remotely. Hence I need a way to send audio stream(PCM samples) to A2F.

I think this is doable because A2F can accept input from microphone (but not running in headless mode).
So, my question is – is there some TCP port I can stream audio to A2F?

vcer · May 9, 2024, 11:21am

github.com

HurroWorld/text-to-audio2face/blob/main/grpcClient.js

// Import required modules and classes
import grpc from '@grpc/grpc-js';
import protoLoader from '@grpc/proto-loader';
import ffmpeg from 'fluent-ffmpeg';
import ffmpegPath from 'ffmpeg-static';
import { Transform } from 'stream';

// Configuration constants
const PROTO_PATH = "C:/Users/houar/AppData/Local/ov/pkg/audio2face-2023.2.0/exts/omni.audio2face.player/omni/audio2face/player/scripts/streaming_server/proto/audio2face.proto";
const A2F_INSTANCE_NAME = '/World/audio2face/PlayerStreaming';
const A2F_SERVER_ADDRESS = 'localhost:50051';
const audioBitrate = 24000;

// Set FFmpeg path
ffmpeg.setFfmpegPath(ffmpegPath);

// Load gRPC proto file for audio2face
const packageDefinition = protoLoader.loadSync(PROTO_PATH, {
  keepCase: true,
  longs: String,

This file has been truncated. show original

I found an example here and implement it now in Rust

system · May 23, 2024, 11:21am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Audio2face in PixelStreaming application Audio2Face (closed)	3	675	October 14, 2023
A2F 2023.2 gRPC Audio2Face (closed)	3	875	December 11, 2023
A2F to Unity streaming Audio2Face (closed)	12	2306	June 6, 2023
Audio2Face + Live Link + Pixel Streaming Audio2Face (closed) audio2face , unreal-5	35	4352	April 20, 2024
Problem about streaming and live-link Audio2Face (closed)	4	905	January 11, 2024
Client for Streaming Audio Player Audio2Face (closed)	7	1623	January 3, 2023
Streaming audio data format queston Audio2Face (closed)	2	327	June 3, 2024
Can I send data from Unreal to audio2face? Audio2Face (closed)	5	792	April 15, 2024
Streaming audio from Riva TTS to Audio 2 face Audio2Face (closed) riva	4	776	March 12, 2024
Can Audio2Face input a streamed audio via some port as if PCM or similar instead of using processed WAV files Audio2Face (closed)	1	408	November 14, 2023

Send Audio Stream to A2F over networking

Related topics