Bringing a Real iToF 3D Camera into NVIDIA Isaac Sim for Sim-to-Real Robotics Development

Omniverse Isaac Sim
1

Hi everyone,

One challenge with robotics perception development is that camera selection and placement often require physical hardware before you can properly evaluate the system.

For example, with a depth camera, you may need to determine:

  • How many cameras are required?
  • Where should they be mounted?
  • Is the field of view sufficient?
  • What areas are left uncovered?
  • Will the depth range work for the application?
  • How will the camera data integrate with ROS 2 and the perception stack?

We recently integrated DepthVista Helix, an indirect Time-of-Flight (iToF) 3D camera from e-con Systems, into NVIDIA Isaac Sim to address some of these challenges.

The integration provides a true-to-spec simulated camera with:

  • High-resolution depth mode: 1280 × 960, up to 2 m
  • Long-range depth mode: 640 × 480, up to 6 m
  • Up to 60 fps
  • 62° × 49° × 83° horizontal, vertical and diagonal FOV
  • Simulated 6-axis IMU
  • ROS 2 output for depth, point clouds, camera calibration, transforms (TF), and IMU data
  • Both GMSL and USB variants

There are also two ready-to-run reference scenarios:

1. Robotic manipulation
A UR10 arm using eye-in-hand and eye-to-hand depth cameras for workflows such as pick-and-place, bin picking, and inspection.

2. Autonomous mobile navigation
A four-camera surround configuration for an AMR, feeding the Nav2 stack for 360° obstacle avoidance.

The interesting part for me is the sim-to-real continuity. The simulated camera is modeled around the physical camera’s field of view, geometry, depth modes, resolutions, calibration, transforms, and ROS 2 topics.

That means teams can evaluate camera placement, coverage, perception algorithms, synthetic data generation, and navigation workflows before committing to physical hardware.

We documented the complete integration and the reference applications here:

Robotics Development Just Got Faster & Smarter: e-con Systems’ iToF 3D Camera Comes to NVIDIA Isaac Sim

The Isaac Sim package is also available through the access process described in the article.

For developers working with Isaac Sim and ROS 2, I’d be interested to hear how you’re currently handling the sim-to-real gap for depth cameras.

3

Thank you for this exciting contribution! I’ll leave this forum post open for responses for awhile.