AI Helps Transform Audio Into Music Playing Avatars

Originally published at: AI Helps Transform Audio Into Music Playing Avatars | NVIDIA Developer Blog

Researchers from Facebook, Stanford, and the University of Washington developed a deep learning based method that can transform audio of musical instruments into skeleton predictions, which can be used to animate an avatar. “The key idea is to create an animation of an avatar that moves their hands similarly to how a pianist or violinist…