Designing Robots with NVIDIA Isaac GEMs for ROS

Originally published at: Designing Robots with NVIDIA Isaac GEMs for ROS | NVIDIA Developer Blog

NVIDIA and Open Robotics have entered into an agreement to accelerate ROS2 performance. As a result, the NVIDIA Jetson edge AI platform now offers new NVIDIA Isaac GEMs for ROS software.

So should i forget about using NVidia Isaac the one on the Kaya robot and just go all in with ROS2 since Nvidia is improving on ROS2? Whats the use case for using ISAAc SDK if NVidia is enabling ROS2?

Im guessing NVidias original plan was to create a competitor to ROS but that is optimized for NVidia GPU. BUt now it seems Nvidia has switched gears and will be improving ROS and creating gems for ROS since the ISAAC sdk uptake was low?

Please keep it real :)

ISAAC ROS is NVIDIA’s effort to support the open-source ROS community with GPU acceleration, developers and customers are welcome to use the GEMS in their applications.
If you are using ISAAC SDK and developing on Kaya, please continue to develop using SDK.

If I want to implement a Visual SLAM algorithm using a Realsense Camera and GPU acceleration, am I pursuing the correct way in trying to plug in the Isaac Ros GEMS (isaac_common, isaac_image_pipeline, isaac_ros_visual_odometry) into an existing ROS2 installation?

As a first shot, I ran the Elbrus example
bob@desktop:~/isaac/sdk$ bazel run packages/visual_slam/apps:elbrus_visual_slam_realsense
from the Isaac SDK but to my great appointment, this example at least didn’t touch the GPU at all but was computed on CPU solely. Was this a mistake on my side, could I make it use the GPU? Initially I was planning to use the Isaac SDK, possibly alongside with its ROS bridge but since this doesn’t seem to involve the GPU I switched to the idea mentioned in the first paragraph of using the GEMs.

I want to avoid rewriting the whole ROS Visual SLAM algorithm by hand to convert it into something palatable for my Nvidia GPU.

Ubuntu 18.04; ROS Noetic; Isaac SDK
Ubuntu 20.04; ROS Foxy, Isaac GEMS only
… in either case Nvidia T1000, Cuda 11.6

Edit No. 2:

I have integrated the GEMS to my ROS2 installation and once more: only CPU is used … no errors/warnings though … how can I force the GEMS to use GPU instead? Isn’t that the whole purpose of this???


  • if GEMS solely in use: 4% GPU while 60% CPU for the Isaac Visual Odometry.
  • Isaac SDK running on Ubuntu 18.04 with the requested CUDA 10.2 – still no load on GPU in spite of the process being listed in nvtop as a computing task. No errors reported, though, on running the bazel command.

Anybody know what could be wrong here?

Hi @user150672

ROS2 middleware is limiting performance.
Improvements are coming in June for a new Isaac ROS release. This new release works with ROS2 Humble to reduce the load on the CPU.
There will continue to be a CPU component to Visual Odometry, so it will never be zero CPU.

Meanwhile, I suggest registering for the new GTC22 and scheduling the calendar for our robotics sessions: Developer Conference Session Catalog | NVIDIA GTC

Hi, is there a version of issac ros visual odometry version that supports ros1?

Actually you could use the Isaac SDK which currently to my knowledge supports only ROS1, however it is also limited to Ubuntu 18.04 (if this is not your native OS you can run it inside a container).

Thanks a heap for the information, I’m looking forward to trying the new release as soon as it’s out!