ESS Node not publishing anything

I am launching the ESS disparity node but it is not publishing anything in /disparity

full_log.txt (30.7 KB)

The node is correctly attaching to the /disparity topic as I can see it in the rqt graph.

ros2 topic hz /disparity and ros2 topic echo /disparity yield nothing

The launchfile is:

import launch
from launch_ros.actions import ComposableNodeContainer
from launch_ros.descriptions import ComposableNode


def generate_launch_description():
    output_width = 480
    output_height = 288

    argus_left_node = ComposableNode(
        name='argus_mono_left',
        package='isaac_ros_argus_camera',
        plugin='nvidia::isaac_ros::argus::ArgusMonoNode',
        parameters=[{
                'mode': 4, 
                'camera_id': 1,
                'camera_info_url': 'file:///workspaces/isaac_ros-dev/left_camera.ini',
                'camera_link_frame_name': 'camera_link',
                'optical_frame_name': 'camera_left_link_optical'
        }],
        remappings=[
                    ('/left/image_raw', '/left/image_raw'),
                    ('/left/camera_info', '/left/camera_info'),
                ],
    )
    
    argus_right_node = ComposableNode(
        name='argus_mono_right',
        package='isaac_ros_argus_camera',
        plugin='nvidia::isaac_ros::argus::ArgusMonoNode',
        parameters=[{
                'mode': 4, 
                'camera_id': 0,
                'camera_info_url': 'file:///workspaces/isaac_ros-dev/right_camera.ini',
                'camera_link_frame_name': 'camera_link',
                'optical_frame_name': 'camera_right_link_optical'
        }],
        remappings=[
                    ('/left/image_raw', '/right/image_raw'),
                    ('/left/camera_info', '/right/camera_info'),
                ],
    )

    left_resize_node = ComposableNode(
        name='left_resize_node',
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::ResizeNode',
        parameters=[{
            'output_width': output_width,
            'output_height': output_height,
            'keep_aspect_ratio': True
        }],
        remappings=[
            ('camera_info', 'left/camera_info'),
            ('image', 'left/image_raw'),
            ('resize/camera_info', 'left/camera_info_resize'),
            ('resize/image', 'left/image_resize')]
    )

    right_resize_node = ComposableNode(
        name='right_resize_node',
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::ResizeNode',
        parameters=[{
            'output_width': output_width,
            'output_height': output_height,
            'keep_aspect_ratio': True
        }],
        remappings=[
            ('camera_info', 'right/camera_info'),
            ('image', 'right/image_raw'),
            ('resize/camera_info', 'right/camera_info_resize'),
            ('resize/image', 'right/image_resize')]
    )

    left_rectify_node = ComposableNode(
        name='left_rectify_node',
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::RectifyNode',
        parameters=[{
            'output_width': output_width,
            'output_height': output_height,
        }],
        remappings=[
            ('image_raw', 'left/image_resize'),
            ('camera_info', 'left/camera_info_resize'),
            ('image_rect', 'left/image_rect'),
            ('camera_info_rect', 'left/camera_info_rect')
        ]
    )

    right_rectify_node = ComposableNode(
        name='right_rectify_node',
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::RectifyNode',
        parameters=[{
            'output_width': output_width,
            'output_height': output_height,
        }],
        remappings=[
            ('image_raw', 'right/image_resize'),
            ('camera_info', 'right/camera_info_resize'),
            ('image_rect', 'right/image_rect'),
            ('camera_info_rect', 'right/camera_info_rect')
        ]
    )

    disparity_node = ComposableNode(
        name='disparity',
        package='isaac_ros_ess',
        plugin='nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode',
        parameters=[{'engine_file_path': '/workspaces/isaac_ros-dev/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.0.0/light_ess.engine',
                     'threshold': 0.1,
                     'input_layer_width': output_width,
                     'input_layer_height': output_height}],
        remappings=[
            ('left/camera_info', 'left/camera_info_resize'),
            ('right/camera_info', 'right/camera_info_resize'),
        ('left/image_rect', 'left/image_resize'),
        ('right/image_rect', 'right/image_resize'),
        ]
    )

    
    
    argus_mono_container = ComposableNodeContainer(
            name='argus_mono_container',
            package='rclcpp_components',
            executable='component_container_mt',
            composable_node_descriptions=[argus_left_node, argus_right_node, left_resize_node, right_resize_node, left_rectify_node, right_rectify_node, disparity_node],
            namespace='',
            output='screen',
            arguments=['--ros-args', '--log-level', 'info'],
        )
    return launch.LaunchDescription([argus_mono_container])

These are the logs I am getting from the ESS node

[component_container_mt-1] [INFO] [1721038692.579305604] [argus_mono_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode>
[component_container_mt-1] [INFO] [1721038692.579368838] [argus_mono_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode>
[component_container_mt-1] [INFO] [1721038694.869944536] [disparity]: [ESSDisparityNode] preLoadGraphCallback().
[component_container_mt-1] [INFO] [1721038694.870098939] [disparity]: [ESSDisparityNode] Setting tensorrt_plugin: /workspaces/isaac_ros-dev/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.0.0/plugins/aarch64/ess_plugins.so.
[component_container_mt-1] [INFO] [1721038694.880274911] [disparity]: [ESSDisparityNode] Setting engine_file_path: /workspaces/isaac_ros-dev/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.0.0/light_ess.engine.
[component_container_mt-1] [INFO] [1721038694.880361313] [disparity]: [ESSDisparityNode] postLoadGraphCallback() with image [288 x 480]
[component_container_mt-1] [INFO] [1721038694.880377025] [disparity]: [ESSDisparityNode] postLoadGraphCallback() block_size = 552960.

The following topics display correct information:
left/camera_info_resize
right/camera_info_resize
left/image_resize
right/image_resize

Subscriptions are ok in rqt graph.

How do I make it work?

Thank you

Hi @eloi3

Thank you for your post.

To better understand your setup, which type of camera are you using? For stereo cameras, we require synchronized cameras that enable the Isaac ROS node to generate a disparity output.

Best,
Raffaello

Hi @Raffaello ,

Does it matter which camera I am using as long as the information provided in /left/image_resize and /right/image_resize are valid images guaranteed to be synchronized?