Originally published at: https://developer.nvidia.com/blog/ai-helps-transform-audio-into-music-playing-avatars/
Researchers from Facebook, Stanford, and the University of Washington developed a deep learning based method that can transform audio of musical instruments into skeleton predictions, which can be used to animate an avatar. “The key idea is to create an animation of an avatar that moves their hands similarly to how a pianist or violinist…