I’m trying to stream audio input from my mic to audio2face, following this tutorial, however the test_client.py script can’t find the necessary Python modules omni.client and omni.kit.
The omni.client module might be under:
C:\Users\adrian\AppData\Local\ov\pkg\audio2face-2023.2.0\kit\extscore\omni.client\omni\client
but I can’t find omni.kit.
I want to run this test_client.py script from PowerShell on Windows, so I’ll need to add those modules to my PYTHONPATH.
Where are those two modules?
Can you try something like this:
python "C:/Users/{USER}/AppData/Local/ov/pkg/prod-audio2face-2023.2.0/exts/omni.audio2face.player/omni/audio2face/player/scripts/streaming_server/test_client.py" "Path_To_Your_Wave_File" "/World/audio2face/PlayerStreaming"
Please note that you will need to install these python modules to be able to run the test_client.py
grpcio
numpy
soundfile
google
protobuf < 3.20
requests
By the way, I’m using Python 3.12
Thanks for that. I got a little further (in that the Python modules are loading now), but I’m getting protobuf errors (see below).
Note: I’m running this from a Windows PowerShell. Is there a different environment in which I should be launching this? I tried a Cmdprompt and got the same error.
PS C:\Users\adrian> py "C:/Users/adrian/AppData/Local/ov/pkg/audio2face-2023.2.0/exts/omni.audio2face.player/omni/audio2face/player/scripts/streaming_server/test_client.py" "C:/Users/adrian/bin/voice_male_p3_neutral.wav" "/World/audio2face/PlayerStreaming"
Traceback (most recent call last):
File "C:\Users\adrian\AppData\Local\ov\pkg\audio2face-2023.2.0\exts\omni.audio2face.player\omni\audio2face\player\scripts\streaming_server\test_client.py", line 15, in <module>
import audio2face_pb2
File "C:\Users\adrian\AppData\Local\ov\pkg\audio2face-2023.2.0\exts\omni.audio2face.player\omni\audio2face\player\scripts\streaming_server\audio2face_pb2.py", line 33, in <module>
_descriptor.FieldDescriptor(
File "C:\Users\adrian\AppData\Local\Programs\Python\Python312\Lib\site-packages\google\protobuf\descriptor.py", line 553, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
Update:
I followed the advice of the 2nd bullet and set this envar:
$env:PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION = 'python'
I also had to pip install soundfile and the script seems to work now.
Would downgrading protobuf package to <=3.20.x be better? My current version of protobuf is 4.25.2.
Yes, from the error message it looks like downgrading the protobuf to 3.20 or lower would make it faster.
Hello, I am trying to do the same thing, but i get following response from CMD:
I downgraded protobuf package and I am not getting any response from character. Not sure what is the issue.
this is the full response:
C:\Users\MobiS PC\AppData\Local\ov\pkg\audio2face-2023.2.0\exts\omni.audio2face.player\omni\audio2face\player\scripts\streaming_server>test_client.py C:\Users\MobiS PC\AppData\Local\ov\pkg\audio2face-2023.2.0\exts\omni.audio2face.player_deps\deps\audio2face-data\tracks\english_voice_male_g8c /World/audio2face/Player
C:\Users\MobiS PC\AppData\Local\ov\pkg\audio2face-2023.2.0\exts\omni.audio2face.player\omni\audio2face\player\scripts\streaming_server>
[main 2024-03-18T19:06:56.329Z] update#setState idle
[main 2024-03-18T19:07:26.344Z] update#setState checking for updates
[main 2024-03-18T19:07:26.403Z] update#setState idle
Not sure what the issue is here. I don’t see any error.
But here’s one way of doing this:
-
Create a folder anywhere you like. e.g. C:/a2f_stream
-
In the terminal go to this folder and create a Python environment using python -m venv .venv
-
Create a requirements.txt file containing these lines:
numpy
protobuf <= 3.20.x
grpcio
soundfile
-
Run pip install -r requirements.txt to install all the modules listed in above.
-
Activate the .venv using ./.venv/Scripts/activate
-
Copy audio2face_pb2_grpc.py, audio2face_pb2.py and test_client.py from {AUDIO2FACE DIRECTORY}\exts\omni.audio2face.player\omni\audio2face\player\scripts\streaming_server to C:/a2f_stream
-
Copy a test audio file (in .wav format) to C:/a2f_stream folder
-
Open Audio2Face app
-
From AI Models Tab select Streaming for Audio Player and click Get Started.
-
Back in the terminal run this command to send the audio to Audio2Face: python .\test_client.py .\audio_file.wav /World/audio2face/PlayerStreaming
1 Like
Thank you very much, the one more thing I had to do was to in requirements.txt file add google and now it works amazing.
can you please just show how to enable mic streaming, as adriangraham above wanted to achieve
To stream mic to A2F you can use the regular player, by selecting Record and going Live.
Thank you, this is working! I do have audio file, and it’s being overwritten with new audio constantly. How can make A2F automatically detects overwrite and playback/stream audio automatically once overwrite has been detected?
There’s no such functionality in A2F for now. You will need to create a callback system to notify A2F when the audio file writing is finished.
Do you know how to stream from UE5 through Live Link? Sth like RecieveLiveLink node? any documentation or tutorial?
In my case when I use Google Cloud TTS API, I need to have protobuf==4.25.3, whereas Audio2Face can’t work with >3.20, thus using PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION = 'python'
was the only way out.
