Eye joint transforms working on only one eye

I am using 2023.2.0 and using the exported joint information. I see one eye moving much more than the other even though they move about the same on the transferred character. Is there some kind of tuner for the joint motion export? Is there a way to preview the motion on the USD skel I am transferring to? I can see the results of the blendshape solve but I don’t see how the joint transforms could be connected to the USD skel (although my USD skel has only one joint anyway).

Hi @stu9354

Can I ask in which software are you applying the exported eye joint animation, Maya, Blender, Unreal Engine or others?

Maya and I look at the animation curves after import and see there is a difference in the movement. I am only applying rotation in my importing Python script from the exported JSON file. Translation would not make sense.

Just did a test and both eyes seem to rotate the same amount. Link to video

Here’s the script I used to apply eye animation from Audio2Face Json to joints in Maya. Just an additional note that I had to scale down the animation from 60 fps to 24 fps after importing.

import json
import maya.cmds as mc

def apply_a2f_joint_wgts():
    with open(r"C:\Users\{USER}\a2f_body_arkit_anim_bsweight.json", "r") as f:
        facs_data = json.loads(f.read())
        joints= facs_data["joints"]
        rotations= facs_data["rotations"]
        numFrames = facs_data["numFrames"]
        mc.playbackOptions(ast=0, min=0, max=numFrames-1, aet=numFrames-1)
        
        jaw_quat = mc.createNode('quatToEuler', n="jaw_quat")
        left_eye_quat = mc.createNode('quatToEuler', n="left_eye_quat")
        right_eye_quat = mc.createNode('quatToEuler', n="right_eye_quat")
        
        mc.connectAttr(jaw_quat+".outputRotate", joints[0].split("/")[-1] + ".rotate")
        mc.connectAttr(left_eye_quat+".outputRotate", joints[1].split("/")[-1] + ".rotate")
        mc.connectAttr(right_eye_quat+".outputRotate", joints[2].split("/")[-1] + ".rotate")

        quat_nodes = [jaw_quat, left_eye_quat, right_eye_quat]
        
        for fr in range(numFrames):
            for i in range(len(joints)):
                mc.setKeyframe(quat_nodes[i]+'.inputQuatX', v=rotations[fr][i][1], t=fr)
                mc.setKeyframe(quat_nodes[i]+'.inputQuatY', v=rotations[fr][i][2], t=fr)
                mc.setKeyframe(quat_nodes[i]+'.inputQuatZ', v=rotations[fr][i][3], t=fr)
                mc.setKeyframe(quat_nodes[i]+'.inputQuatW', v=rotations[fr][i][0], t=fr)

apply_a2f_joint_wgts()

What I am seeing in the exported JSON file is rotations for the jaw and left eye for all 226 frames but for the right eye the values go to zero after about frame 23.

Does it happen every time and on every character?
Can you share your Json file?

Aside from this being a bug or not, I guess the left and right eyes should rotate very similarly (with some offset of course). So the workaround would be to attach the left eye anim to right eye as well and offset it.

Here is the JSON file
face_bsweight.zip (96.9 KB)

Just imported the animation from this json file and both eyes have animation for 226 frames. Can you try using the script I provided above?

I am seeing values for all frames but only one eye is moving in Z. See attached. On another note, the jaw movement is out of sync with the jaw drop blendshape. Attached are frame 53 for the jaw drop blendshape and the jaw rotate, also attached.



Hi @stu9354

Did you use the script provided or used a custom script?

The rotation values exported from Audio2Face are Quaternion, so you have to convert them to Euler when applying them in Maya.

I use a script similar to yours but I do the conversion with this:
rot = om.MQuaternion(rotations[fr][i][0], rotations[fr][i][1], rotations[fr][i][2], rotations[fr][i][3]).asEulerRotation().asVector()
mc.setKeyframe(jointNames[i], attribute=‘rotateX’, v=rot[2], t=fr)
mc.setKeyframe(jointNames[i], attribute=‘rotateY’, v=rot[1], t=fr)
mc.setKeyframe(jointNames[i], attribute=‘rotateZ’, v=rot[0], t=fr)

Can it be because Maya quaternions are structured a little differently from USD quaternions.

USD Quat: w, x, y, z
Maya Quat: x, y, z, w

I accounted for that by swapping the Z and X rotations. The fact that the jaw and right eye rotations are correct. Do your animation curves look like mine in Maya with your script? I did use your script and got the same results as with my script.

Hmm, sorry I don’t know why you’re getting a different result. I applied your exported Json animation in Maya and the eyes seems to work as expected. Like to video