Omniverse Audio2Face Livelink Error on Linux

AI & Data Science Digital Humans (closed) Audio2Face (closed)
1

Hi,

I’m trying to run Audio2Face Livelink (to Unreal Engine 5) on a Linux machine, and I kept on getting the following error.

Screenshot (5)
Screenshot (5)1920×1080 186 KB

What can I do to fix this? Is the Livelink plugin not supported on Linux? Thank.

2

@tsc003 can you confirm whether you’ve installed and activated the “NVIDIA Omniverse Audio2Face LiveLink” plugin per the instruction? and if you can confirm the version of A2F you are using would be a good idea as well for the devs.

3

Hi,
We followed the instructions linked.
We are experience a crash that is related to the OmniverseLiveLink Plugin when packaging for Linux and launching the application. I believe it’s more unreal related and we may need to include a directory in the project settings.
We are using UE v5.2.1 and A2F v2023.1.1

4

We don’t currently support UE Livelink plugin for Linux. But the source code is provided in the UE Livelink plugin, so in theory, you could compile it yourself.

  • You could use Audio2Face’s livelink extension as a starting point and reference for creating the custom streaming extension. It can be found in a folder similar to this: C:\Users\<UserName>\AppData\Local\ov\pkg\prod-audio2face-2023.1.1\exts\omni.avatar.livelink

  • You could also use A2F->UE livelink plugin as a reference for receiving blendShape weights. It can be found in a folder simliar to this: C:\Users\<UserName>\AppData\Local\ov\pkg\prod-audio2face-2023.1.1\ue-plugins\audio2face-ue-plugins

This might also help: Compiling a plugin for a new engine version | Epic Developer Community (epicgames.com)

5

i actually build the unreal5.3 plugin in linux, and in unreal live link window, i have already added an audio2face livelink source, i check that unreal have listened the 12030, 12031 port by “lsof” and audio2face have transfered the data, but i can’t see the livelink subject in unreal, is there any suggestion? Thanks

6

I think I have the same issue. If it helps, I did not have to build the plugin I have used the pre compiled here and move it in my UE Plugins project directory :
.local/share/ov/pkg/audio2face-2023.2.0/ue-plugins/audio2face-ue-plugins/ACEUnrealPlugin-5.3

Plugin is then recognized in UE and enabled but in the UE livelink section I can’t find the Nvidia livelink option like I am used to in Windows.

7

Hey, I finally got Audio2Face with UE Livelink to work on Ubuntu22!
First, you have to rebuild the Plugin from source to be able to see the NVIDIA Omniverse Livelink option in the dropdown menu of the Livelink sources. Add “Linux” as a Platform in the OmniverseLiveLink.uplugin file:

"Modules": [
		{
			"Name": "OmniverseAudioMixer",
			"Type": "Runtime",
			"LoadingPhase": "Default",
			"PlatformAllowList": [
				"Linux"
			]
		},
		{
			"Name": "OmniverseLiveLink",
			"Type": "Runtime",
			"LoadingPhase": "Default",
			"PlatformAllowList": [
				"Linux"
			]
		}
	],

Now there is still a difference. We have to modify the sourcecode very little in order for the subject name to show up and Livelink to work successfully. In the OmniverseBaseListener.cpp in the function OnRawDataReceived, in the final else if statement, replace the code with the following:

else if (ReceivedSize > DataSizeInHeader.GetValue())
{
int32 DataSize = DataSizeInHeader.GetValue();
ReceivedData.RemoveAt(ReceivedSize, DataSize);
PushPackageData(ReceivedData.GetData(), ReceivedSize);
ReceivedData.RemoveAt(0, ReceivedSize);
ReceivedSize = ReceivedData.Num();
DataSizeInHeader.Reset();
}

For some reason the header is different in Linux and the logic does dot work correctly there. The main reason for the problem seems to arise somewhere in the BytesToInt function or before. It outputs way too large sizes for the data. We just use ReceivedSize instead of the DataSizeInHeader and it works. Maybe our NVidia colleagues have some more clue here on the error. :-)

Hope this work for you also and you can use one of the best plugins out there in Linux :-)

4 Likes
8

This is very interesting. I am trying to get the plugin running on Linux since days. Spent many hours debugging. Obviously I added the platform support - then I could use the NVIDIA Omniverse Livelink but the subject would not turn up (and I could not animate my Metahuman).

I sniffed the TCP traffic on port 12030 and 12031- all looks good.

Now I just tried your suggested patch in OmniverseBaseListener.cpp hoping to finally fix this issues. But - after rebuilding the plugin - I am still stuck with the same issue: the subject won’t show.

Any chance you did other source code changes you have not mentioned here that might be related? Or other ideas for me?

Thanks!

9

You are absolutely right, speaking of the BytesToInt function, I have also modified this one → using 4 bytes instead of 8. Try the following changes there:

static int32 BytesToInt(uint8* Bytes, int32 Length)
{
	int32 Return = 0;
	int32 Flag = 0;
	int32 reduced = 4;
	for (int32 Index = reduced - 1; Index >= 0; --Index)
	{
		Return += (Bytes[Index] & 0xFF) << (8 * Flag);
		++Flag;
	}
	return Return;
}
1 Like
10

Awesome - thank you so much!
All working now :-)

1 Like
11

Why not use

#if PLATFORM_WINDOWS
#define RECV_HEADER_SIZE 8
#elif PLATFORM_LINUX
#define RECV_HEADER_SIZE 4
#endif

Because the header is not only used in BytesToInt but also here

image
image713×177 23.8 KB

12

Thank you. It works like magic also for me.