Realsense T265

Hi, has there been any use of the Intel Realsense T265 with Isaac?

1 Like

I see there is this example of stereo odometry -
https://docs.nvidia.com/isaac/isaac/packages/perception/doc/visual_odometry.html
Does anyone have any samples of it working, and could it work using a realsense D435 camera (would use the RGB cameras of the D435 I imagine)
I would think that the T265 would be beneficial because it does all the pose calculations on the unit itself, not on the CPU, as I understand it. Wouldn’t this be a great help vs. Elbrus?
Thanks for the input.

Hi GOB, we have included support for D435 as a reference.
One of our partners has also brought up LIPS Realsense @ GitHub - lips-hci/stereo_ae400: LIPSedge™ AE400 Industrial 3D Camera brings RealSense™ technology to NVIDIA Isaac Robot platform.
You are free to add your own sensors.

Thank you for the reply.
I was wondering that since the t265 uses the same format to stream information as the d435, support may already be available, even if not planned.

Has anyone in the community looked into this?

Hi GOB

Why would you not use D435 + Elbrus? What’s lacking in what’s included in Isaac SDK?

Thanks for the reply.

Would like to use the T265 for a number of reasons:

  1. Elbrus uses the CPU, T265 should do most of the work on the camera.
  2. I already have a T265
  3. T265 has built in IMU that is helpful
  4. Intel has done a great job with the T265 VSLAM, it works very well. I haven’t used Elbrus that much see how it compares.

One thing that seems to be missing is that the the isaac.RealsenseCamera component doesn’t provide a left and right grayscale image needed for Elbrus. But maybe I’m missing something there?

Hello bluecamel8fusr,

https://docs.nvidia.com/isaac/isaac/doc/component_api.html#isaac-realsensecamera

The realsense driver for Isaac SDK is part of the release. Feel free to use it as a template to build out a driver for T265.

Btw, if you want Stereo Visual Odometry, you could use https://docs.nvidia.com/isaac/isaac/packages/perception/doc/visual_odometry.html#stereo-visual-inertial-odometry

We are working with a few partners to offer stereo cameras with IMU integrated. Stay tuned.

Hello!
Has anyone been able to build a codelet for T265? As @gbenel mentioned, the T265 runs the entire V-SLAM on the camera module itself, and saves a lot of compute on the Jetson device - hence would be more preferred over Elbrus.
@shrinv have there been any developments on T265?

Useful comment from our eng:
We compared T265 against our SVIO (using ZED Mini camera, as ZED doesn’t have IMU; new ZED2 has IMU too, soon to be supported in Isaac). T265 on aggressive motion (shaking camera) easily lost tracking while Isaac VIO keeps tracking very reliably with the same type of motion. The only advantage of T265 is that it has Mono camera tracking while Isaac SVIO is stereo only. But this advantage is a weird one as being in possession, physically, of the stereo camera it doesn’t make sense to track in Mono mode. So, we don’t support T265 not only for the reason of not promoting Intel’s product but because we found it inferior, quality of tracking wise, to what we offer in Isaac.

I think T265 is totally reliable, tons of robots and drones with it. I’m not sure what kind of test and in what condition was done, and I won’t enter in the results or with what software, but even being right, I don’t see why limit the possibilities because a camera was better in something. In my case T265 has very imoprtant features that nVidia is omitting when discard give support to T265

-T265 has the half price of ZED with IMU

  • It has global shutter cameras
  • It has fisheyes cameras and a lot more FOV
  • It makes the processing onboard
  • It makes fusion with motor encoders onboard
  • T265 is sold as module too that I can integrate in my project giving me a lot of freedom
  • In module, the price is 1/3 of a Zed with IMU
  • I can use it as tracking camera, but I can use it as a stereo camera just like ZED with external proccesing
    -If the robot/drone crash t265 can be repaired.

Sincerely, I don’t think is good for nVidia, Jetson and Isaac do not give support to Realsense t265 or other Realsenses. Just that lack is enough to avoid move from ROS to Isaac and even valorate choose Jetsons for future projects, I would do the same if some day I can use nVidia GPUs on Intel motherboards because in Fornite a Ati Radeon get more framerate , but I dont care I play PUBG. Mercedes maybe is better but many people want Renault, because maybe is better to deliver parcels, even if they can pay the Mercedes.
Please give support to Realsenses on Isaac, Rpi camera v2 neither is the better out there and is the official camera

2 Likes

Hello NVIDIA team @shrinv @atorabi ,

Another Realsense user here for drones. Strongly agree with the points @fpsychosis and @arjunsengupta98 made above. Particularly with the onboard processing. I know that the ArduPilot team are also currently working as part of the Vision Projects (checkout their Gitter) to support the T265 for VIO. Would really appreciate if Isaac supported the Realsense series, would be a good way to bring in existing large realsense community.

Thanks

2 Likes

what is the estimated date of availability of the support of the ZED2 IMU?
is zED2 supported by ISAAC? in my case it doesn’t appear to be detected.

is it likely that T265 will be supported in Isaac in the future?

1 Like

I hope one day, isaac will support T265, as I can use it in my production. I have nano, I have T265, I wanna use them together. I’ve tried to make them together with isaac, but failed. sorry for my poor ability.

1 Like

Only regular ZED and ZED-M (Mini) are supported in ISAAC 2020.1.
Support for ZED 2 is planned in a future release.

A robotic simulator that left aside the bigger supplier of RGB-D cameras for robotics, that leave without alternative to get global shutter and IR sensores, fisheye lens and on-camera processing, just have not sense and have not future. Same nonsense as if Intel stop to support nvidia GPUs in its motherboards.
With this decission, I’m out of Isaac and nVidia SBC. Base it in ZED was superior to t265 in a test shaking both cameras just, really…is shocking and it shows that who took the decision have not acknowledge of the targeted area of this simulator or doesn’t care.
Sad way took nVidia with its Visual Partner’s programme. At least I’m knowing before make a bigger investment.

@shrinv @atorabi could you please comment on this?

Hi Arjun, we are exploring ways to help our dev community to use existing drivers for cameras on Isaac.

One option is to use ROS driver for T265 with ROS Bridge on Isaac. Have you tried it?

https://docs.nvidia.com/isaac/isaac/packages/ros_bridge/doc/ros_bridge.html#ros-bridge