Just wondering whether there are any guides for the 3ds Max → Audio2Face → Unreal Engine workflow. All the videos involving Unreal Engine seems to be for Metahuman rigs, I have no idea how to set things up in Unreal Engine for a custom rig.
So I have a custom rig in 3ds Max that I’ve already imported into Unreal Engine. I’ve set up voice acting in the Behaviour Tree and would like to add lip sync via Audio2Face. I’ve imported the face mesh of the custom rig from 3ds Max into Audio2Face, done the character transfer, set up the A2F pipeline and confirmed it works, and exported the geometry cache and blendshapes. No idea what to do with it from here.
Ok so that sounds good so far. Now if you have the rig correctly setup in A2F, record your audio and use the audio sync feature to drive your character. Then you can bake out that animation, save as a usd file and use our Omniverse Connector to Unreal to get it into Unreal.
I’m running into a problem importing the USD into Unreal. if I right-click in the Content Browser and select ‘Import USD Animation’ it crashes the editor:
The following is logged into my project’s log file:
[2024.05.03-04.41.31:681][607]LogSlate: Window 'Omniverse USD Animation Importer' being destroyed
[2024.05.03-04.41.33:132][607]LogWindows: Error: === Critical error: ===
[2024.05.03-04.41.33:132][607]LogWindows: Error:
[2024.05.03-04.41.33:132][607]LogWindows: Error: Fatal error!
[2024.05.03-04.41.33:132][607]LogWindows: Error:
[2024.05.03-04.41.33:132][607]LogWindows: Error: Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x0000000000000000
[2024.05.03-04.41.33:132][607]LogWindows: Error:
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcef8ca350 usd_gf.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffceac37091 usd_usdSkel.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffceac333d9 usd_usdSkel.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea5e76db UnrealEditor-OmniverseUSD.dll!USDConvert() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\USDConverter.cpp:5191]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea442c0d UnrealEditor-OmniverseUSD.dll!<lambda_26744bb5f100f2393189e68d24e4e65e>::operator()() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:271]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea477b73 UnrealEditor-OmniverseUSD.dll!OmniverseAnimationImporterImpl::ImportSkelAnimation() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:290]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea48deff UnrealEditor-OmniverseUSD.dll!OmniverseAnimationImporterImpl::LoadUSDAnimationInternal() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:409]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea48d735 UnrealEditor-OmniverseUSD.dll!FOmniverseAnimationImporter::LoadUSDAnimation() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:430]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffce0c686be UnrealEditor-OmniverseEditor.dll!SOmniverseImporterDialog::OkClicked() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseEditor\Private\OmniverseImporterDialog.cpp:136]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffce0c4475a UnrealEditor-OmniverseEditor.dll!TBaseSPMethodDelegateInstance<0,SOmniverseImporterDialog,1,FReply __cdecl(void),FDefaultDelegateUserPolicy>::Execute() [D:\b\0\_build\target-deps\UnrealEngine\Engine\Source\Runtime\Core\Public\Delegates\DelegateInstancesImpl.h:275]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18fc3f66 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18fea355 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18d7b552 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18d602b8 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18dc89b3 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18db7430 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18daccf0 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb50b12 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb36751 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb533c8 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb2ff43 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56bef75 USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56be9ee USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcd59c7f09 InkObj.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd70f210b8 atlthunk.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56bef75 USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56be69d USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb54856 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a71750b UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a73473c UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a73482a UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a7374f0 UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a74a4c4 UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a74d996 UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda59a7344 KERNEL32.DLL!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda61026b1 ntdll.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error:
Is there a workaround for this? If I try importing the USD by drag-and-dropping the USD into the Content Browser it does look like it’s able to recognize that there’s animation in it by labeling the import type as SkelAnimation. Unfortunately this also fails to import, just without a crash. Basically I tried following this video to import the animation even though the video is geared around Metahuman and I’m working with a custom rig.
Here’s a screenshot of the import attempt via drag-and-drop:
Technically speaking I can import the animation into Unreal Engine if I import the USD into 3ds Max first then export it as an Alembic Geometry Cache. However it’s difficult to use this format to drive facial animation on a character via Blueprint.
Tried exporting the Geometry Cache from Audio2Face in Maya format so I could bring it into 3ds Max first. In 3ds Max I can apply the Geometry Cache successfully to my face mesh using the Point Cache Binding modifier. If I scrub the timeline, the animation plays fine. I then exported the face mesh in USD format with Use Animation and Use Animation Range ticked:
Unfortunately I still can’t import it into Unreal Engine with the Omniverse right-click option ‘Import USD Animation’. I get a ‘Failed to Import’ message and the logfile shows the following:
[2024.05.06-01.49.38:454][879]LogOmniverseImporter: Error: Skeleton c14-TestRig_Skeleton isn't compatible with C:/Users/Knut99/Desktop/c14-TestRig/a2f/TestRig_GeoCacheExportedFrom3dsMax.skelanim.usd
I don’t know what else to try.
Are you using the offical Unreal Omniverse Importer ? Can you try with something very very basic, and level it up slowly to see when it crashes. Does it crash with a box, a sphere, a character with no animation, etc etc. Find out exactly what is crashing.
Can you package up the usd file and cache etc and send it to me so I can take a look please.
I’m using the official Unreal Omniverse Importer. The Plugins section in Unreal is showing version 204.0.0-rc.7+5.2.381.8f314f65.gl.
I imported a subdivided cube into Audio2Face from 3ds Max. Skipped skin mesh fitting, just did post wrap then clicked on Preview In Stage under Blendshape Generation. Under the A2F Data Conversion tab I set Geometry Cache > Export to ‘Selected Meshes’ and selected ‘BoxTest_result’ in the Stage list. Format is USD. Clicked Export Cache and exported All As GeoCache. Tried importing into Unreal however via the right-click ‘Import USD Animation’ option failed with the following message:
Failed to import ‘C:/Users/Knut99/Desktop/BoxTest/a2f_export_cache.usd’.
Please see Output Log for details.
Output log shows the following:
[2024.05.07-05.09.40:571][314]LogOmniverseImporter: Error: Cannot find skeleton animation in C:/Users/Knut99/Desktop/BoxTest/a2f_export_cache.usd
Okay, so the problem is no animation. So I went ahead and tried setting up the skin mesh fitting, however the fitting process always gets stuck at 25%:
I guess this process doesn’t like subdivided boxes?
You have to bake out your animation into the usd, using the export function
There are many great videos on exporting from A2F to UE. Here is one example
Yeah I just can’t get it to import, but I think you might be right about a bone mismatch. I ended up just exporting the animations from 3ds Max as vertex animation textures using Unreal’s VertexAnimationTools.ms script.
