Real-time monocular vision-based SLAM with NVIDIA Jetson, CNN and ROS

Hello, Jetson Community!

We’d like to introduce our project for real-time vSLAM with CNN and NVIDIA Jetson. Our main goal was to study different CNN architectures and techniques for depth reconstruction from single image. We evaluated all the developed algorithms (including custom layers) on NVIDIA Jetson TX2 with TensorRT to keep the balance between depth reconstruction quality and real-time performance.

Then, we used the developed CNN architectures as a part of RTAB-MAP (http://wiki.ros.org/rtabmap_ros) vSLAM algorithm pipeline to estimate the position of the moving Jetson TX2 and build the 3D map of unknown indoor environment.

This project is still work-in-progress and is a part of bigger academic research project for autonomous multi-robot vision-based SLAM, exploration, pathplanning and navigation with Jetsons, that is coming this year.

The code and models are free and open-source.

Paper: https://ieeexplore.ieee.org/document/8870936
https://arxiv.org/abs/1907.07210

Repository: https://github.com/CnnDepth/tx2_fcnn_node

Video:
https://www.youtube.com/watch?v=ayjvfzm-C7s

Feel free to use this project, contribute and ask any questions.

  • Andrey

Hi Andrey, thanks for sharing! I tried your FCRN-ResNet50 mono depth network before, it works really well!

Hello, Dustin! Glad you like it. Also, thanks for your https://github.com/dusty-nv/jetson-inference and https://github.com/dusty-nv/jetson-utils. That helps a lot.