Unsupported Rig (wrnch AI Pose Estimator)

Dear friends, I’m going crazy trying to find a skeleton compatible with the Wrnch pose AI Estimator.

I have perfectly configured the Capture Stream and machinima detects the source.

I cannot continue because there is no way for me to detect any skeleton as compatible.

I have to rename the bones. Do you have a compatible skeleton included for testing?

Do I forget to follow any procedure?

Thanks
Antoni

Hi Antoni! Sorry for your issue; I think there’s a solution, but let me clear this one thing first- so you’re using exported asset pinned to Epic skeleton, correct? If so, what’s the current skeleton restpose? Are you able to check if it’s actually T-pose and change/resave restpose in your 3d software of choice, if it isn’t?

Dear kkantiev, thank you for your message. Indeed it is the T pose of the mannequin of the Epic Skeleton. I have converted it to usd asset, from Unreal Engine. I can change the pose without problem.

Report that I have finally found the compatible characters that are tested within machinima.

This is the path of the 4 types of characters with compatible skeletons, found in the following omniverse path:

omniverse: // localhost / NVIDIA / Assets / Machinima / BannerlordII / Characters /
omniverse: // localhost / NVIDIA / Assets / Machinima / Mineways / Characters /
omniverse: // localhost / NVIDIA / Assets / Machinima / NVIDIA_Sol / Characters / Sol /
omniverse: // localhost / NVIDIA / Assets / Machinima / Squad / Characters / Models /

I have tested the animation application Wrnch pose AI Estimator and it works perfectly.

Now my intention is to get to work with my own characters and I think that for it to work I must rename the bones using the same hierarchy of bones from one of the compatible machinima example skeletons.

Do you think the same?

1 Like

Well, quite from the other end - we might add support for the Epic skeleton so it would just work. Your approach with renaming might work as well, but there are some caveats about matching skeleton topology, and I guess this skeleton being supported just out of the box sounds easier! =)
The only thing I’m unsure of is the rest pose, which is important for correct pose mapping, and it’s A-pose for default exported Epic skeleton, as far as I see. You would want a T-pose, yes.

Thanks kkantiev, it would be great if you could add support for the Epic skeleton.
I’m going to do the test renaming the bones and with the poses.
I will inform you of the result.

Dear @kkantiev, good news. After many tests I have managed to make the wrnch Ai Pose app work with its own character. The example is with the Unreal Engine 4 manquin. To inform the community, I have done the following experiments;

1-

Rename the bones of the mannequin from cinema 4d, with the same names of the character “Generic” of machinima example:

omniverse: //localhost/NVIDIA/Assets/Machinima/BannerlordII/Characters/Imperial_Legionary.usd

I discard this formula because it requires many modifications in the character and once imported into machinima it deforms, etc. We would have to study the poses, the weights, the pivots, etc.

2-

Change the names of the bones in the configuration file of the wrnch Ai Pose application. That is, a retarget of the bones name of the new character in the file “wrx_xform_generic” located in

C: \ Users \ XXXXX \ AppData \ Local \ ov \ pkg \ machinima-2021.1.0 \ exts \ wrnch.engine.wrx_decoder \ wrnch \ engine \ wrx_decoder \

This formula works, but requires position adjustments, which are reversed. The Character has his legs crossed and when he walks he goes upwards. Z, Y, Z reversed … Another problem is that the configuration is lost with the generic example characters.

I wish in future versions, you can contemplate the correct configuration for compatibility with other skeletons or that the user can make a retarget from machinima with any type of skeleton.

3-

The fastest formula with optimal results has been;
Import the sample character Imperial_Legionary.usd into Cinema 4D and bind the skeleton with the mannequin. I have very easily adapted the bones to the new figure, I have exported in fbx and then from Omniverse converted to USD.

Only I have to finish adjusting the pose, the weight of some parts of the mannequin and it would be perfect.

The attached video of the movement test is with formula 3.

Note: It is also necessary to better calibrate the camera and use it in a larger space to improve movement.

Automatic weights from C4D; (adjust very little)

2 Likes

Oh that is an impressive research! I really hope our next update is going to make things much easier for you. Basically, we’re going to add another json file next to ‘wrx_xform_generic.json’ you’ve discovered, so Epic skeleton would be picked up and used by wrnch automagically. The only thing user would need to ensure is that restpose is t-shaped body, and you can save such restpose inside Machinima now! Let me attach a short video showing the process.

1 Like
2 Likes

Actually, if you can’t wait to try it, you can use this json from the video - just put it next to others and restart the Machinima.
wrx_xform_ue4.json (2.1 KB)

4 Likes

Ohh, thank you dear kkantiev, I feel flattered.
I just tested the config file and followed the instructions in the video. Total success!!! Now it works.
I am very very happy.
From the new configuration file, now I understand how to flip the correct position with the following line:
skel_root_rotate_xyz ": [-90, 0, 0]
With the video I have also learned to modify a pose and save from machinima.
Thanks for such good support and help to users.
Machinima with Omniverse is the future.

Following this methodology, I imagine that unlimited files can be created for other types of skeletons, right?

1 Like

hello i’m blocked at the stage in the picture. “Rig Type Unsported rig”. i’m so new I have no idea what to do. I would be grateful if you could help.

Hey Atolyes, please note that your bone naming should match particular sets from known skeletons so Wrnch could recognize it. I would start from removing “mixamorig_” prefix, that might just work.

In general you can get the bone naming that will work for sure by looking at existing content examples (that are known to work with Wrnch) and picking bone naming from there.

Happy to help! Yes, in theory you can add more skeletons, but I would be careful and wouldn’t overdo that - it could interfere with the existing skeleton types and mess the recognition process.

1 Like