ZED x mini on isaac ros - argus problems

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Hey,

Lately I’ve been trying to run the latest version of isaac ros for JP6 with zedx mini.
I connect my zed camera via Stereolabs dual connector (GMSL) and also have it’s newest driver installed.
Trying to run isaac-ros-argus-camera, it crashed on the gxf rectification step. I then commented out the step:

  - type: nvidia::gxf::Connection
    parameters:
      source: argus_camera/left_camera_output
      target: converter_left/data_receiver
  - type: nvidia::gxf::Connection
    parameters:
      source: converter_left/data_transmitter
      #     target: rectify_parameters/left_receiver
      # - type: nvidia::gxf::Connection
      #   parameters:
      #     source: rectify_parameters/left_transmitter
      target: stereo_camera_synchronizer/rx_left_camera

and saw that I indeed get the data via /left/image_raw topic (of course the /left/camera_info isn’t published since it needs the rectification step).

I then investigated more and found out that the problem in the rectification step is that the previous step: nvidia::isaac::ArgusCamera is indeed sending the stream of the stereo camera but not the calibration data (all zeros).

Then I have installed Jetson Multimedia Api and ran “argus_syncstereo” and saw that it does identify the cameras correctly and the gets frames from both camera but the calibration data is all zeros:

Executing Argus Sample: argus_syncstereo
Argus Version: 0.99.3.3 (multi-process)
getSensorPlacement for sensor 0 is DEFAULT_SensorPlacement
Found : ZED_CAM1
getSensorPlacement for sensor 1 is SENSOR_PLACEMENT_REAR_OR_BOTTOM
Found : ZED_CAM1
getSensorPlacement for sensor 2 is SENSOR_PLACEMENT_REAR_OR_BOTTOM
Found : ZED_CAM2
getSensorPlacement for sensor 3 is SENSOR_PLACEMENT_REAR_OR_BOTTOM
Found : ZED_CAM2
Total Module Count is 2
/**************************/
Identified stereo camera module ZED_CAM1 with the following 2 sensors connected:
0
1
/**************************/
/**************************/
Identified stereo camera module ZED_CAM2 with the following 2 sensors connected:
2
3
/**************************/
Session[0] : add camera device 0
Session[0] : add camera device 1
PRODUCER: Creating stream[0].
PRODUCER: Creating stream[1].

Calibration data of sensor 0 (device 0) of session 0:

 Image size = 0, 0

 Focal Length = 0.000000, 0.000000

 Principal Point = 0.000000, 0.000000

 Skew = 0.000000

 Fish Eye mapping type = MAPPING_TYPE_EQUIDISTANT

 Lens Distortion type = DISTORTION_TYPE_POLYNOMIAL

 Radial coeffs count = 0

 Radial coefficients = 

 Tangential coeffs count = 0

 Tangential coefficients = 

 rot3d x, y, x{0.000000, 0.000000, 0.000000}

 translation 3d x, y, x{0.000000, 0.000000, 0.000000}

I suspect that this what cause the rectification step to crash on the gxf, but maybe I got it wrong.

Is this a known problem? what should I do to fix it? is there a workaround for this? this cause the whole isaac ros problem not to work and prevents me to continue.

I have also posted on zed stereolabs forums a question about it (maybe driver) and the directed me here.

Thank you.

2

Hi @shimoni

Thank you for your detailed post and for working with Isaac ROS.

At this time, ZED support for Isaac ROS 3.0 is temporarily disabled and will be available soon.

I keep you updated on other news.

Thank you in advance,
Raffaello

3

Hey, thanks!

Is there any ETA for this feature?

Adi.

4

Hi @shimoni

Unfortunately, I cannot confirm an ETA, but it will be included in a future release.

Raffaello