GTC 2020: Panoptic Segmentation DNN for Autonomous Vehicles

GTC 2020 S21879
Presenters: Ke Chen ,NVIDIA; Ryan Oldja,NVIDIA
Abstract
We’ll present our NVIDIA DriveAV’s Panoptic Segmentation Deep Neural Network (DNN), which can be used for semantic and instance segmentation of complex scenes for self-driving car scenarios, such as complex urban areas, congested traffic, construction zones with unusual activities, and so on. With Panoptic Segmentation DNN, input images can be accurately parsed for both semantic segmentation (which pixels represent which object class), as well as instance content (which pixels represent which object instance). Planning and control modules can use panoptic segmentation results to better inform autonomous driving decisions. We’ll cover our highly accurate GT dataset, DNN architecture, our multi-task training process, and our real-time inference (that includes post-processing steps) on vehicles’ AGX compute. Our network achieves state-of-the-art accuracy and runs at 7ms end-to-end on NVIDIA AGX GPUs. We’ll show videos of our experiments on a real vehicle in various challenging conditions.

Watch this session
Join in the conversation below.