GTC 2020: 3D Deep Learning in Function Space

GTC 2020 S21764
Presenters: Michael Niemeyer,MPI-IS and University of Tübingen
Recent advances in GPU technology and scalable algorithms have led to breakthroughs in deep learning. In particular, convolutional neural networks (CNNs) achieve state-of-the-art results in longstanding vision problems, such as image classification or object detection. However, autonomous agents that navigate and interact in our world need to reason in 3D. Unlike images in the 2D case, it is not clear how to represent 3D geometry and how to make it amenable for deep-learning techniques. We’ll introduce our approach to learning 3D representations in function space. First, we’ll show how this approach can represent arbitrary topologies without discretization at fixed memory cost. Then we’ll extend this framework to learning to predict not only the shape of an object, but also its texture and motion. Finally, we’ll show how we can scale our method to real-world scenarios using state-of-the art NVIDIA GPU technology.

Watch this session
Join in the conversation below.

Really interesting and well presented work Micheal.
You actually want to try this when you see your presentation.

What are the limitations right know and what are your future work on this?
Ia this something for NVIDIA Isaac like system and autonomous systems right know?

Hi! Amazing approach to 3D characterization. The 3D representations in function space is a really clever idea! But above all, very well explained: going into technical details, such the analytic gradient function, without losing the focus on the main idea… and only in 25 min. Bravo!

I could see this being used in game development for making models. I could start off with just a picture and then refine the GPU generated model.

Good idea using a continuous function to create a smooth object, where as the discrete methods generate ruff objects.

Is your code on github? I would love to try it on my GTX 970. Thank you!

It was actually 13 minutes. I like how it started off very simple and evolved into a very technical video.

1 Like

Nice presentation and interesting concept.