How to change the image dimension of realsense images in Isaac ROS!

sarthakgarg0303 · August 29, 2024, 6:51pm

These are the various image dimensions I can obtain from the Intel RealSense camera, each corresponding to a different frame rate. I want to adjust the input data frequency accordingly. Could you either suggest a method to modify the input frequency or recommend which image dimensions I should use from the camera to achieve the desired frequency?

These are the topics list I’m receiving related to camera inputs

/realsense2_camera/aligned_depth_to_color/camera_info
/realsense2_camera/aligned_depth_to_color/image_raw
/realsense2_camera/aligned_depth_to_color/image_raw/compressed
/realsense2_camera/aligned_depth_to_color/image_raw/compressedDepth
/realsense2_camera/aligned_depth_to_color/image_raw/nitros
/realsense2_camera/aligned_depth_to_color/image_raw/theora
/realsense2_camera/color/image_raw/compressed
/realsense2_camera/color/image_raw/compressedDepth
/realsense2_camera/color/image_raw/theora
/realsense2_camera/color/metadata
/realsense2_camera/depth/camera_info
/realsense2_camera/depth/image_rect_raw
/realsense2_camera/depth/image_rect_raw/compressed
/realsense2_camera/depth/image_rect_raw/compressedDepth
/realsense2_camera/depth/image_rect_raw/theora
/realsense2_camera/depth/metadata
/realsense2_camera/extrinsics/depth_to_color

Intel Realsense Camera - D435
Hardware - Nvidia Orin Agix

Raffaello · September 2, 2024, 7:46pm

Hi @sarthakgarg0303

To change the camera sensor, you need to modify the specific roslaunch.py file for the desired demo.

For example, if you are working with Isaac ROS vslam, the file: isaac_ros_visual_slam_realsense.launch.py there is an option to change the camera setup

github.com

NVIDIA-ISAAC-ROS/isaac_ros_visual_slam/blob/main/isaac_ros_visual_slam/launch/isaac_ros_visual_slam_realsense.launch.py#L36


      
              name='camera',
              namespace='camera',
              package='realsense2_camera',
              executable='realsense2_camera_node',
              parameters=[{
                      'enable_infra1': True,
                      'enable_infra2': True,
                      'enable_color': False,
                      'enable_depth': False,
                      'depth_module.emitter_enabled': 0,
                      'depth_module.profile': '640x360x90',
                      'enable_gyro': True,
                      'enable_accel': True,
                      'gyro_fps': 200,
                      'accel_fps': 200,
                      'unite_imu_method': 2
              }]
          )
          
          visual_slam_node = ComposableNode(
              name='visual_slam_node',

If you are working with Isaac ROS pose estimation the camera resolution is set to these lines:

github.com

NVIDIA-ISAAC-ROS/isaac_ros_examples/blob/main/isaac_ros_realsense/launch/isaac_ros_realsense_mono_rect_core.launch.py#L33


      
          import launch
          from launch_ros.actions import ComposableNodeContainer
          from launch_ros.descriptions import ComposableNode
          
          
          class IsaacROSRealSenseMonoRectLaunchFragment(IsaacROSLaunchFragment):
          
              @staticmethod
              def get_interface_specs() -> Dict[str, Any]:
                  return {
                      'camera_resolution': {'width': 640, 'height': 480},
                      'camera_frame': 'camera_color_optical_frame',
                      'focal_length': {
                          # Approximation - most RealSense cameras should be close to this value
                          'f_x': 380.0,
                          # Approximation - most RealSense cameras should be close to this value
                          'f_y': 380.0
                      }
                  }
          
              @staticmethod

But remember that the pose estimation demo is trained at 640x480. If you want to work at other resolutions, you need to update also the model.

Best,
Raffaello

sarthakgarg0303 · September 3, 2024, 1:41pm

Thank you @Raffaello , I’m working with foundationpose pose estimation using Isaac ROS, I changed the yaml file to get the desired inputs from realsense camera. The depth and camera_info are obtained at 60hz but still the rgb inputs are not produced at 60 hz as explicitly mentioned, they are obtained at somewhere around 40hz.

And when I run this command to get the inputs from camera, I’m getting all the three inputs of depth, rgb, camera info all at 60 hz.

ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=640x480x60 rgb_camera.color_profile:=640x480x60

This is the jetson consumption!

Raffaello · September 11, 2024, 5:32pm

Hi @sarthakgarg0303

Can you please share the launch file needed to run this demo? Also, please double-check if you have modified the correct yaml file.

I noticed that your Jetson is running other software that may be consuming CPU and GPU. Please disable any other running applications.

Best,
Raffaello