Hello,
As mentioned in Optimizing ISAAC ROS ESS Depth Estimation to run efficiently with lower GPU memory utilization I am trying to run ISAAC ROS ESS Depth Estimation on multi cameras with limited 6GB of GPU memory on my robot. I was trying a workout to merge images from two cameras with known transforms between them using perspective transform. When I feed those topics to isaac_ros_ess depth estimation it does not output any disparity.
I am guessing it is to do with
[image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
My image_format_node_right nodes do output the mono8->rgb8 image. Additionally my preprocessed image data matches input image dimensions for light_ess model.
PFA the output log file and my launch file.
Thank you.
Output Log
O] [launch]: All log files can be found below /home/admin/.ros/log/2023-10-26-18-48-06-303030-legion-120436
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [120447]
[INFO] [rviz2-2]: process started with pid [120449]
[INFO] [filter_pointcloud-3]: process started with pid [120451]
[rviz2-2] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-admin'
[rviz2-2] MESA: warning: Driver does not support the 0xa7a0 PCI ID.
[rviz2-2] libGL error: failed to create dri screen
[rviz2-2] libGL error: failed to load driver: iris
[rviz2-2] MESA: warning: Driver does not support the 0xa7a0 PCI ID.
[rviz2-2] libGL error: failed to create dri screen
[rviz2-2] libGL error: failed to load driver: iris
[rviz2-2] MESA: warning: Driver does not support the 0xa7a0 PCI ID.
[rviz2-2] libGL error: failed to create dri screen
[rviz2-2] libGL error: failed to load driver: iris
[rviz2-2] MESA: warning: Driver does not support the 0xa7a0 PCI ID.
[rviz2-2] libGL error: failed to create dri screen
[rviz2-2] libGL error: failed to load driver: iris
[component_container_mt-1] [INFO] [1698360486.686138507] [disparity.disparity_container]: Load Library: /workspaces/rm250_ros2/install/isaac_ros_ess/lib/libisaac_ros_ess_node.so
[component_container_mt-1] [INFO] [1698360486.698394278] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode>
[component_container_mt-1] [INFO] [1698360486.698414630] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode>
[component_container_mt-1] [INFO] [1698360486.700991550] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1698360486.701078082] [disparity]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1698360486.701426305] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1698360486.704798886] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1698360486.706582936] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1698360486.708523695] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1698360486.710927260] [NitrosContext]: [NitrosContext] Loading application: '/workspaces/rm250_ros2/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container_mt-1] [INFO] [1698360486.711282999] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1698360486.713905470] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-1] 2023-10-26 18:48:06.713 WARN gxf/std/program.cpp@514: No system specified. Nothing to do
[component_container_mt-1] [INFO] [1698360486.714395461] [disparity]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1698360486.714405246] [disparity]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1698360486.715309259] [disparity]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1698360486.715315864] [disparity]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1698360486.716027766] [disparity]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1698360486.716034940] [disparity]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1698360486.716037414] [disparity]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1698360486.716145460] [disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1698360486.716877109] [disparity]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1698360486.717508600] [disparity]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1698360486.718170847] [disparity]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1698360486.720228740] [disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_synchronization.so
[component_container_mt-1] [INFO] [1698360486.720970307] [disparity]: [NitrosContext] Loading extension: gxf/lib/thresholder/libgxf_thresholder.so
[component_container_mt-1] [INFO] [1698360486.721938000] [disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages.so
[component_container_mt-1] [INFO] [1698360486.722634151] [disparity]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1698360486.724919872] [disparity]: [NitrosContext] Loading extension: gxf/lib/sgm_disparity/libgxf_sgm.so
[component_container_mt-1] [INFO] [1698360486.725796824] [disparity]: [NitrosContext] Loading extension: gxf/lib/ess/libgxf_cvcore_ess.so
[component_container_mt-1] [INFO] [1698360486.738767913] [disparity]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1698360486.741180051] [disparity]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1698360486.817277557] [disparity]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1698360486.818265069] [disparity]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1698360486.824240507] [disparity]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/disparity' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.825487684] [disparity.disparity_container]: Load Library: /workspaces/rm250_ros2/install/isaac_ros_image_proc/lib/libimage_format_converter_node.so
[component_container_mt-1] [INFO] [1698360486.826447497] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ImageFormatConverterNode>
[component_container_mt-1] [INFO] [1698360486.826461234] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ImageFormatConverterNode>
[component_container_mt-1] [INFO] [1698360486.829471466] [image_format_node_left]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1698360486.829886871] [image_format_node_left]: [ImageFormatConverterNode] Set output data format to: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360486.829935588] [image_format_node_left]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1698360486.829941314] [image_format_node_left]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1698360486.831259067] [image_format_node_left]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1698360486.831268358] [image_format_node_left]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1698360486.834748584] [image_format_node_left]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1698360486.834764438] [image_format_node_left]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1698360486.834923906] [image_format_node_left]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1698360486.835388185] [image_format_node_left]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1698360486.836101097] [image_format_node_left]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1698360486.862001349] [image_format_node_left]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1698360486.865196314] [image_format_node_left]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1698360486.865221632] [image_format_node_left]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360486.866502722] [image_format_node_left]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/image_format_node_left' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.867141099] [image_format_node_left]: Negotiating
[component_container_mt-1] [INFO] [1698360486.867655821] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ImageFormatConverterNode>
[component_container_mt-1] [INFO] [1698360486.867671836] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ImageFormatConverterNode>
[rviz2-2] [INFO] [1698360486.869047637] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-2] [INFO] [1698360486.869100628] [rviz2]: OpenGl version: 3.1 (GLSL 1.4)
[component_container_mt-1] [INFO] [1698360486.870464911] [image_format_node_right]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1698360486.870772781] [image_format_node_right]: [ImageFormatConverterNode] Set output data format to: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360486.870800170] [image_format_node_right]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1698360486.870804166] [image_format_node_right]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1698360486.871931880] [image_format_node_right]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1698360486.871941226] [image_format_node_right]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1698360486.874835666] [image_format_node_right]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1698360486.874857961] [image_format_node_right]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1698360486.875005735] [image_format_node_right]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1698360486.875392304] [image_format_node_right]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1698360486.876079233] [image_format_node_right]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1698360486.881866848] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360486.888436936] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[rviz2-2] [INFO] [1698360486.901193330] [rviz2]: Stereo is NOT SUPPORTED
[component_container_mt-1] [INFO] [1698360486.902925185] [image_format_node_right]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1698360486.907158879] [image_format_node_right]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1698360486.907192934] [image_format_node_right]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360486.908481962] [image_format_node_right]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/image_format_node_right' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.909727276] [disparity.disparity_container]: Load Library: /workspaces/rm250_ros2/install/isaac_ros_stereo_image_proc/lib/libpoint_cloud_node.so
[component_container_mt-1] [INFO] [1698360486.911896603] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::stereo_image_proc::PointCloudNode>
[component_container_mt-1] [INFO] [1698360486.911908127] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::stereo_image_proc::PointCloudNode>
[component_container_mt-1] [INFO] [1698360486.914215340] [pointcloud_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1698360486.914579901] [pointcloud_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1698360486.914587509] [pointcloud_node]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1698360486.915516634] [pointcloud_node]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1698360486.915525154] [pointcloud_node]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1698360486.915922206] [pointcloud_node]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1698360486.915926975] [pointcloud_node]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1698360486.915929836] [pointcloud_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1698360486.916422990] [pointcloud_node]: [NitrosContext] Loading extension: gxf/lib/libgxf_point_cloud.so
[component_container_mt-1] [INFO] [1698360486.942386548] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360486.942743794] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_ros_messages.so
[component_container_mt-1] [INFO] [1698360486.943906161] [pointcloud_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1698360486.945310810] [pointcloud_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1698360486.961416239] [pointcloud_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1698360486.961744638] [pointcloud_node]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1698360486.965177237] [pointcloud_node]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1698360486.965437038] [image_format_node_left]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/pointcloud_node' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.965460043] [disparity]: Negotiating
[component_container_mt-1] [INFO] [1698360486.965496203] [disparity]: Could not negotiate
[component_container_mt-1] [INFO] [1698360486.965642319] [image_format_node_left]: Negotiating
[component_container_mt-1] [INFO] [1698360486.966877761] [disparity.disparity_container]: Load Library: /opt/ros/humble/lib/libstatic_transform_broadcaster_node.so
[component_container_mt-1] [INFO] [1698360486.968097537] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<tf2_ros::StaticTransformBroadcasterNode>
[component_container_mt-1] [INFO] [1698360486.968114086] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<tf2_ros::StaticTransformBroadcasterNode>
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/static_transform_publisher_1' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.973712429] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<tf2_ros::StaticTransformBroadcasterNode>
[component_container_mt-1] [INFO] [1698360486.973743118] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<tf2_ros::StaticTransformBroadcasterNode>
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/static_transform_publisher_2' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.978762051] [image_format_node_right]: Negotiating
[component_container_mt-1] [INFO] [1698360486.978820794] [disparity.disparity_container]: Found class: rclcpp_components::NodeFactoryTemplate<tf2_ros::StaticTransformBroadcasterNode>
[component_container_mt-1] [INFO] [1698360486.978843066] [disparity.disparity_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<tf2_ros::StaticTransformBroadcasterNode>
[component_container_mt-1] [INFO] [1698360486.978960187] [image_format_node_right]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/static_transform_publisher_3' in container '/disparity/disparity_container'
[component_container_mt-1] [INFO] [1698360486.988608002] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360486.988759787] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360486.988761929] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.015327235] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.015508296] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.015510891] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.055224730] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.055375431] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.055514727] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.081864794] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.082006045] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.082151644] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[rviz2-2] [INFO] [1698360487.087845480] [rviz2]: Stereo is NOT SUPPORTED
[component_container_mt-1] [INFO] [1698360487.115200137] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.115230498] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.115345893] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.121523286] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.121727745] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.121795598] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.181662944] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.181667859] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.181893392] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.189018213] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.189050847] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.189268197] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.249215949] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.249219283] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.249479313] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.255560479] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.255567465] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.256036901] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.282266163] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.282277246] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.282473877] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.314941404] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.314942705] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.315246022] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.354899560] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.354916213] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.355257256] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.381503985] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.381565061] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.381744373] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.421683764] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.421697164] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.421807331] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.448464480] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.448484411] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.448686168] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.554851765] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.554959216] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.555351880] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.581559761] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.581692259] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.581757029] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.621437528] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.621440121] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.621563405] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.648059321] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.648156615] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.648204753] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.688570287] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.688780543] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.689026451] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.714937827] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.714940095] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.715059594] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.755267935] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.755484846] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.755572114] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.781400836] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.781423528] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.781530072] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.821701023] [disparity]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.821706085] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.821872195] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.824571172] [disparity]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1698360487.824590545] [disparity]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1698360487.824599936] [disparity]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.824605113] [disparity]: [NitrosPublisher] Use only the compatible publisher: topic_name="/disparity", data_format="nitros_disparity_image_32FC1"
[component_container_mt-1] [INFO] [1698360487.824611381] [disparity]: [NitrosSubscriber] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360487.824617805] [disparity]: [NitrosSubscriber] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360487.824621574] [disparity]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.824624992] [disparity]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/transformed/infra1/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1698360487.824638343] [disparity]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.824642091] [disparity]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/transformed/infra2/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1698360487.824683761] [disparity]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1698360487.848234430] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.848781018] [image_format_node_left]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.866805319] [image_format_node_left]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1698360487.866851506] [image_format_node_left]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1698360487.866867075] [image_format_node_left]: [NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360487.866877828] [image_format_node_left]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.866885086] [image_format_node_left]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/transformed/infra1/image_rect_raw", data_format="nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360487.866945554] [image_format_node_left]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1698360487.872497423] [image_format_node_left]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/rm250_ros2/install/isaac_ros_image_proc/share/isaac_ros_image_proc/EBTHOPKXNC.yaml"
[component_container_mt-1] [INFO] [1698360487.872522453] [image_format_node_left]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1698360487.872525521] [image_format_node_left]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1698360487.872528919] [image_format_node_left]: [NitrosContext] Loading application: '/workspaces/rm250_ros2/install/isaac_ros_image_proc/share/isaac_ros_image_proc/EBTHOPKXNC.yaml'
[component_container_mt-1] [INFO] [1698360487.873534810] [image_format_node_left]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1698360487.873577986] [image_format_node_left]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1698360487.873581966] [image_format_node_left]: [ImageFormatConverterNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1698360487.873593520] [image_format_node_left]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1698360487.883581395] [image_format_node_left]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1698360487.883638843] [image_format_node_left]: [NitrosNode] Starting a heartbeat timer (eid=443)
[component_container_mt-1] [INFO] [1698360487.888678462] [disparity]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/rm250_ros2/install/isaac_ros_ess/share/isaac_ros_ess/JTETKUUUTW.yaml"
[component_container_mt-1] [INFO] [1698360487.888704199] [disparity]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1698360487.888708196] [disparity]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1698360487.888711684] [disparity]: [NitrosContext] Loading application: '/workspaces/rm250_ros2/install/isaac_ros_ess/share/isaac_ros_ess/JTETKUUUTW.yaml'
[component_container_mt-1] [INFO] [1698360487.888891506] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.889149560] [image_format_node_right]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'left_image_input' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'right_image_input' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'left_cam_input' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'right_cam_input' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'left_image_output' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'right_image_output' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'left_cam_output' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'right_cam_output' in component ''.
[component_container_mt-1] 2023-10-26 18:48:07.891 WARN gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'output_adapter' in component ''.
[component_container_mt-1] [INFO] [1698360487.892251217] [disparity]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1698360487.892334469] [disparity]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1698360487.892403916] [disparity]: [ESSDisparityNode] Setting engine_file_path: /workspaces/rm250_ros2/resources/ess_light.engine.
[component_container_mt-1] [INFO] [1698360487.892409010] [disparity]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1698360487.897466215] [disparity]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1698360487.897650593] [disparity]: [NitrosNode] Starting a heartbeat timer (eid=479)
[component_container_mt-1] [INFO] [1698360487.908880931] [image_format_node_right]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1698360487.908916551] [image_format_node_right]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1698360487.908929160] [image_format_node_right]: [NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360487.908940824] [image_format_node_right]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.908949032] [image_format_node_right]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/transformed/infra2/image_rect_raw", data_format="nitros_image_rgb8"
[component_container_mt-1] [INFO] [1698360487.908985370] [image_format_node_right]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1698360487.914849321] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.919051567] [image_format_node_right]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/rm250_ros2/install/isaac_ros_image_proc/share/isaac_ros_image_proc/JBPEKOCDDV.yaml"
[component_container_mt-1] [INFO] [1698360487.919088432] [image_format_node_right]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1698360487.919097015] [image_format_node_right]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1698360487.919116962] [image_format_node_right]: [NitrosContext] Loading application: '/workspaces/rm250_ros2/install/isaac_ros_image_proc/share/isaac_ros_image_proc/JBPEKOCDDV.yaml'
[component_container_mt-1] [INFO] [1698360487.920028678] [image_format_node_right]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1698360487.920077987] [image_format_node_right]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1698360487.920087813] [image_format_node_right]: [ImageFormatConverterNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1698360487.920098129] [image_format_node_right]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1698360487.921916292] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.926863570] [image_format_node_right]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1698360487.926939282] [image_format_node_right]: [NitrosNode] Starting a heartbeat timer (eid=585)
[component_container_mt-1] [INFO] [1698360487.957823599] [pointcloud_node]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-1] [INFO] [1698360487.965473204] [pointcloud_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1698360487.965518211] [pointcloud_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1698360487.965529152] [pointcloud_node]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.965538414] [pointcloud_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/points2", data_format="nitros_point_cloud"
[component_container_mt-1] [INFO] [1698360487.965547211] [pointcloud_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.965554385] [pointcloud_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/disparity", data_format="nitros_disparity_image_32FC1"
[component_container_mt-1] [INFO] [1698360487.965562176] [pointcloud_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.965568513] [pointcloud_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/infra1/image_rect_raw", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1698360487.965575517] [pointcloud_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.965582224] [pointcloud_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/transformed/infra1/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1698360487.965597525] [pointcloud_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1698360487.965603980] [pointcloud_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/transformed/infra2/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1698360487.965627210] [pointcloud_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1698360487.994532107] [pointcloud_node]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/rm250_ros2/install/isaac_ros_stereo_image_proc/share/isaac_ros_stereo_image_proc/AXFMYWYCFH.yaml"
[component_container_mt-1] [INFO] [1698360487.994571515] [pointcloud_node]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1698360487.994579381] [pointcloud_node]: [PointCloudNode] preLoadGraphCallback().
[component_container_mt-1] [INFO] [1698360487.994597477] [pointcloud_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1698360487.994601249] [pointcloud_node]: [NitrosContext] Loading application: '/workspaces/rm250_ros2/install/isaac_ros_stereo_image_proc/share/isaac_ros_stereo_image_proc/AXFMYWYCFH.yaml'
[component_container_mt-1] [INFO] [1698360487.996383230] [pointcloud_node]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1698360487.996436607] [pointcloud_node]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1698360487.996464079] [pointcloud_node]: [NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1698360488.001088666] [pointcloud_node]: [NitrosContext] Running application...
[component_container_mt-1] [INFO] [1698360488.001256600] [pointcloud_node]: [NitrosNode] Starting a heartbeat timer (eid=645)
Launch file:
import os
from ament_index_python.packages import get_package_share_directory
import launch
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import ComposableNodeContainer, Node
from launch_ros.descriptions import ComposableNode
def generate_launch_description():
launch_args = [
DeclareLaunchArgument(
'engine_file_path',
default_value='/workspaces/rm250_ros2/resources/ess_light.engine',
description='The absolute path to the ESS engine plan.'),
DeclareLaunchArgument(
'threshold',
default_value='0.0',
description='Threshold value ranges between 0.0 and 1.0 '
'for filtering disparity with confidence.'),
]
engine_file_path = LaunchConfiguration('engine_file_path')
threshold = LaunchConfiguration('threshold')
disparity_node = ComposableNode(
name='disparity',
package='isaac_ros_ess',
plugin='nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode',
parameters=[{'engine_file_path': engine_file_path,
'threshold': threshold}],
remappings=[
('left/camera_info', '/transformed/infra1/camera_info'),
('left/image_rect', 'infra1/image_rect_raw'),
('right/camera_info', '/transformed/infra2/camera_info'),
('right/image_rect', 'infra2/image_rect_raw')
]
)
image_format_converter_node_left = ComposableNode(
package='isaac_ros_image_proc',
plugin='nvidia::isaac_ros::image_proc::ImageFormatConverterNode',
name='image_format_node_left',
parameters=[{
'encoding_desired': 'rgb8',
}],
remappings=[
('image_raw', '/transformed/infra1/image_rect_raw'),
('image', 'infra1/image_rect_raw')]
)
image_format_converter_node_right = ComposableNode(
package='isaac_ros_image_proc',
plugin='nvidia::isaac_ros::image_proc::ImageFormatConverterNode',
name='image_format_node_right',
parameters=[{
'encoding_desired': 'rgb8',
}],
remappings=[
('image_raw', '/transformed/infra2/image_rect_raw'),
('image', 'infra2/image_rect_raw')]
)
pointcloud_node = ComposableNode(
name='pointcloud_node',
package='isaac_ros_stereo_image_proc',
plugin='nvidia::isaac_ros::stereo_image_proc::PointCloudNode',
parameters=[{
'use_color': True,
'unit_scaling': 1.0
}],
remappings=[('left/image_rect_color', 'infra1/image_rect_raw'),
('left/camera_info', '/transformed/infra1/camera_info'),
('right/camera_info', '/transformed/infra2/camera_info')]
)
node_descriptions = [disparity_node,
image_format_converter_node_left,
image_format_converter_node_right,
pointcloud_node]
container = ComposableNodeContainer(
name='disparity_container',
namespace='disparity',
package='rclcpp_components',
executable='component_container_mt',
composable_node_descriptions=node_descriptions,
output='screen',
arguments=['--ros-args', '--log-level', 'info']
)
rviz_config_path = os.path.join(get_package_share_directory(
'vision_pipeline'), 'rviz', 'isaac_ros_ess_realsense.rviz')
rviz = Node(
package='rviz2',
executable='rviz2',
arguments=['-d', rviz_config_path],
output='screen')
pointcloud_filter = Node(
package="vision_pipeline",
executable="filter_pointcloud",
output="log"
)
return (launch.LaunchDescription(launch_args + [container, rviz, pointcloud_filter]))