How to combine CC4 A2F with A2Gesture?

I just learned how to do CC4 & A2F:
https://forums.developer.nvidia.com/t/how-to-use-new-2022-1-with-cc4-on-full-body/221868/36

Now came A2Gesture:

How to combine CC4 A2F with A2Gesture ?

1 Like

Hello @pekka.varis! Did you have a chance to check out the tutorial from this post yet?
LIVESTREAM: Getting Started: Audio2Emotion (Wed. August 31 3PM PDT / 6PM EDT)

Yes I have watched it, there is no info about this, sorry…

This currently have to to solve independently.
Use A2F to solve the the face. export the cache, then use Machima + A2G to animated the body with the same audio clip and attach the a2f cache

Sounds great!
I have to first follow the video you made me about exporting A2F to Machinima. Then I can tell how far I can go with this solution…

Hi!
I now had time to study this. It worked pretty fine as defaults!!
See here I do it the very first time :)

Before I go with combining a2f cache, I see that there are issues with Camilla cc4 characters hands. What are my options for dealing with those hands clipping inside here dress?

Pekka

Here is my first try on combining these:
I have audio2face cache imported, and also connected here already.
I thought I try to run a2gesture with the same audio straigh. It works, but why is she making those gesture movements even there is just total silence at the beginning of the audio clip?

I managed to combine them!

Here you can see the full workflow:

But still, the hands are peaking/clipping to her dress and shirt.
How do we fix this?

2 Likes

WOW, Pekka. You did it. Thank you for all the beta testing and for making the tutorial.

As for myself though I can’t say I have any interest in continuing with this product until there’s a reliable and easy way of getting the animation back into iClone (so that it can then be exported to and rendered in Unreal). I’m not interested in Omniverse Create or Machinima so I really hope this can still be accomplished by NVIDIA and Reallusion!

Hi @pekka.varis. Great work, thank you for testing A2G!

  1. An easy way to avoid intersections between hands and other parts of an asset would be to adjust/tweak retarget pose. I sometimes raise characters’ hands in retarget poses to make it more Y-like (instead T-like).
  2. I noticed that in your last video above FPS in sequencer and FPS of the animation don’t match. When they match audio timespan and animation span are of the same length.
1 Like

@etumanov thanks!!
I try this, and check the frame rates…

@esusantolim Do you know if iclone8 livelink works in a wat that it would export the A2G motions to iClone8 from Omniverse Machinima? @rbt730 is looking for that…

I’m not too familiar with iClone myself.
But I will forward the question to Reallusion team if they have answer for your question

@esusantolim also I can imagine that there is a possibility to make same kind of scrip that you gave to us; resetting the facial expression to defaults in A2F for CC4 characters.

Can you please think of that solution?
Is is quite a large work for users to adjust/tweak retarget pose for every character…

I tried the a2gesture with CC4 character Kevin:

The same thing going on…
Please can you give me all possible tutorials & documentations about this “adjust/tweak retarget pose.”

I want to know I have all info at my hands before I try to manually fix this for each character.

I opened the “Animation retargeting” window and all looks to be ok.

I watched Animation retargeting documentation, the 13 min overview video at the top of page:
https://docs.omniverse.nvidia.com/app_machinima/prod_extensions/ext_animation-retargeting.html?highlight=animation%20retargeting

At 10:16 you see the same kinda pose that I have with cc4 camilla: elbows slightly back, shoulder up like in tension…

But Maker of video John says it looks pretty ok but I think it´s not good. There is no info about adjust/tweak retarget pose. I cannot move the balls on the Animation retargeting window.

Someone please help me to get this done?

I thought there’s some sort of presets for CC characters to work with Retargetting.
I can ask the team member to confirm

There is no info about adjust/tweak retarget pose. I cannot move the balls on the Animation retargeting window.
You should be able to select the joint on the viewport, rotate it, then apply(or store) the current pose as retarget pose

1 Like

thank you @esusantolim !

A2G motions to iClone8 from Omniverse Machinima?
@pekka.varis unfortunately iClone currently can’t import USD skel / blendshape animation

like @esusantolim already told here, the combination works this way:

There we have also in use our custom tweak for “over-tension” kinda pose for cc characters with a2g: