A2F: load_bs_weights_mh_rig_maya.py mapping mismatch to MH controls

[gabriel.betros]

Hi, is there an updated version of the script load_bs_weights_mh_rig_maya.py since the A2F blendshapes names are different resulting in errors in Maya when running the script like not finding the browLowerR

I could try to make new mapping and change the names in the script, but it would be more convinient to have the script updated by A2F to have proper mapping. Thanks. Any help is appreciated.

[Ehsan.HM]

Hi @gabriel.betros Are you using 52-arkit-blendshapes? browLowerR is one of the nv46 blendshape target names. So I assume you’re not using arkit shapes. That script works with arkit shapes only. But as you mentioned you could update it so it works with nv46 shapes as well. If you do, it would be great if you shared it here :)

[gabriel.betros]

@Ehsan.HM Thank you, and sorry for my late response, this was put on hold. I’m using the claire_solved_arkit so I’m not sure what number is that. But I tried to create new mapping and I’ll share it here in case someone needed it, I canged some weights values based on trial and error.

I also noticed in A2F there isnt options to controls the eyebrows section, they always have extremely subtle movement, so it would be nice to have some parameters to increase the strength of the eyebrows movement.

Another thing that would be great to have is the emphasize of phoenems shapes like “M, P, F” and to have paramteres and blendshapes for these to be mapped to MetaHuman rig controls as well, because in some examples audio files I tested these phoenems did not close the character mouth properly. Thanks for any insights on on this. Here is the new mapping script:

import maya.cmds as mc
import json

mh_ctl_list = [
    ['CTRL_R_brow_down.ty', "browDownRight", 1.0],  
    ['CTRL_L_brow_down.ty', "browDownLeft", 1.0], 
    ['CTRL_R_brow_lateral.ty', "browDownRight", 1.0],
    ['CTRL_L_brow_lateral.ty', "browDownLeft", 1.0],
    ['CTRL_R_brow_raiseIn.ty', "browInnerUp", 1.0], 
    ['CTRL_L_brow_raiseIn.ty', "browInnerUp", 1.0],
    ['CTRL_R_brow_raiseOut.ty', "browOuterUpRight", 1.0], 
    ['CTRL_L_brow_raiseOut.ty', "browOuterUpLeft", 1.0],
    ['CTRL_L_eye.ty', "eyeLookUpLeft", 1.0, "eyeLookDownLeft", -1.0],
    ['CTRL_L_eye.tx', "eyeLookOutLeft", 1.0, "eyeLookInLeft", -1.0],
    ['CTRL_R_eye.ty', "eyeLookUpRight", 1.0, "eyeLookDownRight", -1.0],
    ['CTRL_R_eye.tx', "eyeLookOutRight", 1.0, "eyeLookInRight", -1.0],
    ['CTRL_R_eye_blink.ty', "eyeBlinkRight", 1.0, "eyeWideRight", -1.0],
    ['CTRL_L_eye_blink.ty', "eyeBlinkLeft", 1.0, "eyeWideLeft", -1.0],
    ['CTRL_R_eye_squintInner.ty', "eyeSquintRight", 1.0], 
    ['CTRL_L_eye_squintInner.ty', "eyeSquintLeft", 1.0],
    ['CTRL_R_eye_cheekRaise.ty', "cheekSquintRight", 1.0], 
    ['CTRL_L_eye_cheekRaise.ty', "cheekSquintLeft", 1.0],
    ['CTRL_R_mouth_suckBlow.ty', "cheekPuff", 0.5], 
    ['CTRL_L_mouth_suckBlow.ty', "cheekPuff", 0.5],
    ['CTRL_R_nose.ty', "noseSneerRight", 1.0], 
    ['CTRL_L_nose.ty', "noseSneerLeft", 1.0],
    ['CTRL_C_jaw.ty', "jawOpen", 1.0, "jawForward", 0.6],
    ['CTRL_C_jaw.tx', "jawLeft", -1.0, "jawRight", 1.0],
    ['CTRL_R_mouth_lipsTogetherU', "mouthClose", 1.0],
    ['CTRL_L_mouth_lipsTogetherU', "mouthClose", 1.0],
    ['CTRL_R_mouth_lipsTogetherD', "mouthClose", 1.0],
    ['CTRL_L_mouth_lipsTogetherD', "mouthClose", 1.0],
    ['CTRL_C_mouth.tx', "mouthLeft", 0.5, "mouthRight", -0.5],
    ['CTRL_R_mouth_dimple.ty', "mouthDimpleRight", 1.0], 
    ['CTRL_L_mouth_dimple.ty', "mouthDimpleLeft", 1.0],
    ['CTRL_R_mouth_cornerPull.ty', "mouthSmileRight", 0.5], 
    ['CTRL_L_mouth_cornerPull.ty', "mouthSmileLeft", 0.5],
    ['CTRL_R_mouth_sharpCornerPull.ty', "mouthSmileRight", 0.5], 
    ['CTRL_L_mouth_sharpCornerPull.ty', "mouthSmileLeft", 0.5],
    ['CTRL_R_mouth_cornerDepress.ty', "mouthFrownRight", 1.0], 
    ['CTRL_L_mouth_cornerDepress.ty', "mouthFrownLeft", 1.0],
    ['CTRL_R_mouth_stretch.ty', "mouthStretchRight", 0.5], 
    ['CTRL_L_mouth_stretch.ty', "mouthStretchLeft", 0.5],
    ['CTRL_R_mouth_upperLipRaise.ty', "mouthUpperUpRight", 1.0], 
    ['CTRL_L_mouth_upperLipRaise.ty', "mouthUpperUpLeft", 1.0],
    ['CTRL_R_mouth_lowerLipDepress.ty', "mouthLowerDownRight", 1.0], 
    ['CTRL_L_mouth_lowerLipDepress.ty', "mouthLowerDownLeft", 1.0],
    ['CTRL_R_jaw_ChinRaiseD.ty', "mouthShrugLower", 1], 
    ['CTRL_L_jaw_ChinRaiseD.ty', "mouthShrugLower", 1],
    #['CTRL_R_jaw_ChinRaiseU.ty', "mouthShrugUpper", 0.5], 
    #['CTRL_L_jaw_ChinRaiseU.ty', "mouthShrugUpper", 0.5],
    ['CTRL_R_mouth_lipsPressU.ty', "mouthPressRight", 1.0], 
    ['CTRL_L_mouth_lipsPressU.ty', "mouthPressLeft", 1.0],
    ['CTRL_R_mouth_towardsU.ty', "mouthPucker", 1.0], 
    ['CTRL_L_mouth_towardsU.ty', "mouthPucker", 1.0], 
    ['CTRL_R_mouth_towardsD.ty', "mouthPucker", 1.0], 
    ['CTRL_L_mouth_towardsD.ty', "mouthPucker", 1.0], 
    ['CTRL_R_mouth_purseU.ty', "mouthPucker", 1.0], 
    ['CTRL_L_mouth_purseU.ty', "mouthPucker", 1.0], 
    ['CTRL_R_mouth_purseD.ty', "mouthPucker", 1.0], 
    ['CTRL_L_mouth_purseD.ty', "mouthPucker", 1.0],
    ['CTRL_R_mouth_funnelU.ty', "mouthFunnel", 1.0], 
    ['CTRL_L_mouth_funnelU.ty', "mouthFunnel", 1.0], 
    ['CTRL_L_mouth_funnelD.ty', "mouthFunnel", 1.0], 
    ['CTRL_R_mouth_funnelD.ty', "mouthFunnel", 1.0],
    ['CTRL_R_mouth_pressU.ty', "mouthRollUpper", 1.0], 
    ['CTRL_L_mouth_pressU.ty', "mouthRollUpper", 1.0], 
    ['CTRL_R_mouth_pressD.ty', "mouthRollLower", 1.0], 
    ['CTRL_L_mouth_pressD.ty', "mouthRollLower", 1.0],
    ['CTRL_C_tongue_bendTwist.ty', 'tongueOut', -1.0]
]

with open(r'C:\tmp\a2f_mh_controlRig_test.json', "r") as f:
    facs_data = json.loads(f.read())
    facsNames = facs_data["facsNames"]
    numPoses = facs_data["numPoses"]
    numFrames = facs_data["numFrames"]    
    weightMat = facs_data["weightMat"]
    mc.playbackOptions(ast=0, min=0, max=numFrames-1, aet=numFrames-1)
    namespace = ''
    for fr in range(numFrames):
        weightMat_fr = weightMat[fr]
        for i in range(len(mh_ctl_list)):
            ctl_value = 0
            numInputs = (len(mh_ctl_list[i])-1) / 2
            for j in range(numInputs):
                poseIdx = facsNames.index(mh_ctl_list[i][j*2+1])
                ctl_value += weightMat[fr][poseIdx] * mh_ctl_list[i][j*2+2]
            mc.setKeyframe(namespace+mh_ctl_list[i][0], v=ctl_value, t=fr)