How to use new 2022.1 with CC4 on full body?

I need information on how to use new 2022.1 with CC4 on full body?

Simple, step by step video tutorial would be the best.

As you can see, I have really spent a lot of time with the old version, trying to do this:

I succeeded in some degree but I really do not want to go again a 3 week research project with this :)

I also now write a support topic to cc4 about this need, if they answer I update the info here.

3 Likes

Hello @pekka.varis! I’ve asked the dev team for some help answering your question. I will post back here when I have more information!

2 Likes

Hi @pekka.varis ,
I’ve recorded these tutorials for CC4 character full body connection with A2F.
https://drive.google.com/drive/folders/1Fzvp_2sCkkhVF8JBZ4tdjrrr6e6u1tKX?usp=sharing

  1. Overview - Inspect what CC4 character export provided, what’s missing, and challenges 09:19
  2. Extra Mesh - Prepare missing extra meshes 16:14
  3. CharSetup - Gathering all the meshes and add them to charSetup 10:20
  4. CharTransfer - Running charTransfer with CC3Plus Preset and attach A2F Fullface pipeline 13:20
  5. Technical fix to workaround transform update trigger. 09:01
  6. Connecting to the usdSkel animation 05:58

Some of the steps are still quite involved and we are working to streamline that.

1 Like

Thanks for the information. I assume it is a much simpler process to export the Blendshape JSONs back into iClone. Can you please provide us any update on how that should be done? Nothing works. Even when I try to export a USD Facial Animation to Unreal, I get a message that says “Cannot find curve timesamples.” Everything seems to be exporting null for me. Any guidance is very appreciated.

Hi @rbt730, can you give us more information, so we can take a look closer?

  • if you export .json from the blendshape solve, are you getting all “NULL” ?

  • For unreal import part, are you using UE4 or UE5?

  • If possible, can you share the .json and .usd file that didn’t work for you?

@esusantolim thanks this is so important information!!

I get on with this and share my results here :)

From my test today, all .json export + .usd export for UE4 and UE5 all work.
@rbt730 If you show some video or any clue, I think we can find out what went wrong or is missed.

Hi there @yseol ,

It works now, after lots of difficulty. I eventually got it half-working late last night when I used my A2F result mesh as the Input Anim Mesh and my A2F mesh head export from iClone (without eyes or dentures) as the Blendshape Mesh. This way I exported multiple JSON files back to iClone and was able to successfully import them and play them in iClone’s Timeline.

At some point today I unchecked some of the Blendshapes that were being exported in the JSON files via the CC4 Extended preset. I am finding that my Reallusion characters either look completely flat or their brows and upper lip lift too much in the middle, producing a “worried chipmunk” effect. (I had hoped to clean this up in iClone, but the program understandably freezes when I try to edit any Blendshape keyframes-- it’s too much for it to handle, since I’m working with long audio files.) However, in unchecking some of the Blendshapes that were being exported from Audio2Face, this led to me back to all-“NULL” JSON files. I don’t know why it produced this error since I was simply toggling checkmarks on a Pose profile.

Unfortunately I had already saved over my file, so I had to start fresh, but in starting fresh I kept running into an issue where the A2F CC_Base_Body mesh (the head) would simply not follow along with the Input Anim after a Blendshape Solve. This lasted for hours, until literally just now, when it simply worked after I reloaded the file. This isn’t the first time reloading has solved my problems in this program, oddly. But I can confirm that I’m able to export JSONs again.

I can send you my USD file via a private message if you’d like-- Reallusion wouldn’t appreciate me sharing a CC4 character here. Otherwise, I’d really appreciate a formal YouTube tutorial on how to get all the relevant Blendshape Conversion data in JSON format-- for face and, in particular, the saccade data of the eyes-- back into iClone so that I have something to refer to step-by-step. If such a tutorial is in the works, that’s fantastic. Thanks.

Hi @rbt730, thanks for trying many tests.

We also tested many cases internally, for exmple exporting to iClone8, UE4, and UE5. We found all working well and couldn’t reproduce any error you mentioned.
Do you have any screenshot of this case? so we can try to reproduce the error on our side?

However, in unchecking some of the Blendshapes that were being exported from Audio2Face, this led to me back to all-“NULL” JSON files. I don’t know why it produced this error since I was simply toggling checkmarks on a Pose profile.

At the moment, we are investigating how to transfer all the facial motion (teeth, saccade, tongue) to other apps, once we figure that out, we will share the tutorial. Thanks for your input.

1 Like

@esusantolim I now have time to study your superb videos!

My first problem is here:

Pekka

Hi @pekka.varis ,
Audio2Face and Create shares same kit sdk when it comes to supporting usd file import. Asuming the USD file is valid, they should be importable.
Which exporter are you using to get your mesh out from blender to usd?
I think there’s some effort done on creating connector in Blender for Omniverse, but I don’t know in detail where it’s at. I’ll ping our blender expert here to chime in, but any info of what error you got from Blender would be useful

1 Like

@esusantolim please can you help me?

I hae followed the video instructions and I got it all made fine til´ the end of part 04.

See here my video, it goes STEP by STEP and exactly as you do:

But when I re-open the same scene that is working OK at the end of the above vide, I get this bad character transformation:

The error message as it opens is this:

I need help fast, I am working with a project here.

Yours,
Pekka

Ah, thank god I had the backup versions saved.
I continued from my ver 02 and this time all went by the book, even after re-opening:

So no urgent need of help here. But if you want to see the odd error above I can send the scene…

Ok, gonna make it all over again from the scratch, recording every step as I go.

@pekka.varis , on your corrupted scene, Can you double check the prox deformer nodes attributes?
I saw before that prox deformer is not maintaining its attributes values in the USD from the UI.
Under /World/LazyGraph, find all the Prox deformers node (they should all have _Prox in their name) and double check the attributes if they are as intended. Change the value and save your scene. the changes should now saved in the scene fine

I’ll make sure this is fixed on our next release

Thanks! I check that out later.
Now I follow your videos in detail, focusing 100%.

I found out that if I save here, I loose the fitted mesh selection :)

I continue!

Following this workflow, you gotta be super careful even when you save the scene…

Uh!
Again, saving at wrong point of process mess up the deformations:

Now I know 99% sure I have done all exactly like in your videos.
So I now shall try that workaround you told me - Prox deformers nodes.

Les see!

Ok, now I need your help @esusantolim :)

See, I can almost solved the issue by deactivating the very last BODY Prox deformer node:

But then we have that nice belly move :)
Can I send you this scene?

With that scene you can watch the full above video I made, it covers all the steps I made, following your every move.

You can send me the scene if you like.
But form what I see, you just need to reenable the useMaxDistance checkbox of the prox deformer. don’t need to disable it completely.
All the prox deformer affecting the body should have that option reenabled