H i all,
In the previous release 3.1 I used to be able to run the Quickstart demo (i.e. ess_rosbag depth estimates) in the docker container using the nvidia devel image just by following the instructions in the documentation and got the expected outcome.
Now I’m struggling to get the expected results with the new release 3.2. Following the documentation here I’m using the updated debian pkgs, running the example launch file as described here and using the isaac_ros_ess_visualizer.py visualization script. The results I get look like the image below.
The logging messages of the isaac_ros_examples.launch.py launch file look as follows:
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2025-01-09-18-05-32-605250
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [44120]
[component_container_mt-1] [INFO] [1736445932.919849386] [isaac_ros_examples.container]: Load Library: /opt/ros/humble/lib/libresize_node.so
[component_container_mt-1] [INFO] [1736445932.932847040] [isaac_ros_examples.container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container_mt-1] [INFO] [1736445932.932902204] [isaac_ros_examples.container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container_mt-1] [INFO] [1736445932.967003286] [left_resize]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1736445932.967381534] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1736445932.968264761] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_gxf_helpers.so
[component_container_mt-1] [INFO] [1736445932.969895866] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_sight.so
[component_container_mt-1] [INFO] [1736445932.971589556] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_atlas.so
[component_container_mt-1] 2025-01-09 18:05:32.975 WARN gxf/std/program.cpp@538: No GXF scheduler specified.
[component_container_mt-1] [INFO] [1736445932.976100666] [left_resize]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1736445932.980868779] [left_resize]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1736445932.980916481] [left_resize]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1736445932.982037269] [left_resize]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_message_compositor.so
[component_container_mt-1] [INFO] [1736445932.982225538] [left_resize]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1736445932.983446745] [left_resize]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_tensorops.so
[component_container_mt-1] [INFO] [1736445932.985196531] [left_resize]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1736445932.987073609] [left_resize]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1736445933.084791389] [left_resize]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1736445933.091489304] [left_resize]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/left_resize' in container '/isaac_ros_examples/container'
[component_container_mt-1] [INFO] [1736445933.092881851] [isaac_ros_examples.container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container_mt-1] [INFO] [1736445933.092902639] [isaac_ros_examples.container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container_mt-1] [INFO] [1736445933.094616338] [right_resize]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1736445933.094820080] [right_resize]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1736445933.100091442] [right_resize]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1736445933.100146034] [right_resize]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1736445933.101952890] [right_resize]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1736445933.198569681] [right_resize]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1736445933.204514122] [right_resize]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/right_resize' in container '/isaac_ros_examples/container'
[component_container_mt-1] [INFO] [1736445933.206052510] [isaac_ros_examples.container]: Load Library: /opt/ros/humble/lib/libess_disparity_node.so
[component_container_mt-1] [INFO] [1736445933.217444526] [isaac_ros_examples.container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode>
[component_container_mt-1] [INFO] [1736445933.217497366] [isaac_ros_examples.container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_stereo_depth::ESSDisparityNode>
[component_container_mt-1] [INFO] [1736445933.219697722] [ess_disparity]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1736445933.219939672] [ess_disparity]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1736445933.221987347] [ess_disparity]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1736445933.222049849] [ess_disparity]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1736445933.223836043] [ess_disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_video_buffer_utils.so
[component_container_mt-1] [INFO] [1736445933.224102596] [ess_disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages.so
[component_container_mt-1] [INFO] [1736445933.224412120] [ess_disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_sgm.so
[component_container_mt-1] [INFO] [1736445933.224666492] [ess_disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages_throttler.so
[component_container_mt-1] [INFO] [1736445933.225945707] [ess_disparity]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_ess.so
[component_container_mt-1] [INFO] [1736445933.232514412] [ess_disparity]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1736445933.236046563] [ess_disparity]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1736445933.359640594] [ess_disparity]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1736445933.363767538] [ess_disparity]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/ess_disparity' in container '/isaac_ros_examples/container'
[component_container_mt-1] [INFO] [1736445933.364098665] [right_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364101546] [left_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364393519] [ess_disparity]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364424745] [ess_disparity]: Could not negotiate
[component_container_mt-1] [INFO] [1736445933.364415516] [right_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364440352] [ess_disparity]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364475567] [ess_disparity]: Could not negotiate
[component_container_mt-1] [INFO] [1736445933.364623921] [left_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364905297] [ess_disparity]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364939072] [ess_disparity]: Could not negotiate
[component_container_mt-1] [INFO] [1736445933.364952820] [ess_disparity]: Negotiating
[component_container_mt-1] [INFO] [1736445933.364960560] [ess_disparity]: Could not negotiate
[component_container_mt-1] [INFO] [1736445933.365130705] [isaac_ros_examples.container]: Load Library: /opt/ros/humble/lib/libdisparity_to_depth_node.so
[component_container_mt-1] [INFO] [1736445933.366311478] [isaac_ros_examples.container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::stereo_image_proc::DisparityToDepthNode>
[component_container_mt-1] [INFO] [1736445933.366324330] [isaac_ros_examples.container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::stereo_image_proc::DisparityToDepthNode>
[component_container_mt-1] [INFO] [1736445933.368119328] [DisparityToDepthNode]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1736445933.368301029] [DisparityToDepthNode]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1736445933.370208078] [DisparityToDepthNode]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1736445933.370271971] [DisparityToDepthNode]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_utils.so
[component_container_mt-1] [INFO] [1736445933.374063265] [DisparityToDepthNode]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1736445933.374602667] [DisparityToDepthNode]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1736445933.375678058] [DisparityToDepthNode]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1736445933.376735348] [DisparityToDepthNode]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/DisparityToDepthNode' in container '/isaac_ros_examples/container'
[component_container_mt-1] [INFO] [1736445933.376975792] [ess_disparity]: Negotiating
[component_container_mt-1] [INFO] [1736445933.377080010] [left_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.377081716] [right_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.377348286] [left_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445933.377377786] [right_resize]: Negotiating
[component_container_mt-1] [INFO] [1736445934.092094421] [left_resize]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1736445934.092193868] [left_resize]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1736445934.092214879] [left_resize]: [NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1736445934.092226868] [left_resize]: [NitrosPublisher] Use the negotiated data format: "nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.092235991] [left_resize]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1736445934.092244465] [left_resize]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/left/image_rect", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1736445934.092258179] [left_resize]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1736445934.092266332] [left_resize]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/left/camera_info_rect", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.092293975] [left_resize]: [NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component "sync/image_in" (type="nvidia::gxf::DoubleBufferReceiver") from "nitros_image_bgr8" to "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1736445934.092525078] [left_resize]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1736445934.103467403] [left_resize]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/KAOLGTQUXX/KAOLGTQUXX.yaml"
[component_container_mt-1] [INFO] [1736445934.103488827] [left_resize]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1736445934.105173976] [left_resize]: [ResizeNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1736445934.105208579] [left_resize]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1736445934.111799069] [left_resize]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1736445934.204872989] [right_resize]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1736445934.204941456] [right_resize]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1736445934.204951282] [right_resize]: [NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1736445934.204959494] [right_resize]: [NitrosPublisher] Use the negotiated data format: "nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.204965981] [right_resize]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1736445934.204971484] [right_resize]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/right/image_rect", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1736445934.204978259] [right_resize]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1736445934.204983104] [right_resize]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/right/camera_info_rect", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.205002441] [right_resize]: [NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component "sync/image_in" (type="nvidia::gxf::DoubleBufferReceiver") from "nitros_image_bgr8" to "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1736445934.205131071] [right_resize]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1736445934.214212372] [right_resize]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/DRCMJIMWIK/DRCMJIMWIK.yaml"
[component_container_mt-1] [INFO] [1736445934.214233720] [right_resize]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1736445934.216378713] [right_resize]: [ResizeNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1736445934.216409759] [right_resize]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1736445934.217501990] [right_resize]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1736445934.364980377] [ess_disparity]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1736445934.365143338] [ess_disparity]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1736445934.365176066] [ess_disparity]: [NitrosPublisher] Use the negotiated data format: "nitros_disparity_image_32FC1"
[component_container_mt-1] [INFO] [1736445934.365202525] [ess_disparity]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1736445934.365232426] [ess_disparity]: [NitrosPublisher] Use only the compatible publisher: topic_name="/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.365256075] [ess_disparity]: [NitrosSubscriber] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1736445934.365276680] [ess_disparity]: [NitrosSubscriber] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1736445934.365292748] [ess_disparity]: [NitrosSubscriber] Use the negotiated data format: "nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.365308620] [ess_disparity]: [NitrosSubscriber] Use the negotiated data format: "nitros_camera_info"
[component_container_mt-1] [INFO] [1736445934.365707218] [ess_disparity]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1736445934.377255674] [DisparityToDepthNode]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1736445934.377312542] [DisparityToDepthNode]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1736445934.377333420] [DisparityToDepthNode]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1736445934.377347018] [DisparityToDepthNode]: [NitrosPublisher] Use only the compatible publisher: topic_name="/depth", data_format="nitros_image_32FC1"
[component_container_mt-1] [INFO] [1736445934.377361376] [DisparityToDepthNode]: [NitrosSubscriber] Use the negotiated data format: "nitros_disparity_image_32FC1"
[component_container_mt-1] [INFO] [1736445934.377480100] [DisparityToDepthNode]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1736445934.378826987] [DisparityToDepthNode]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/NNGQKUABJE/NNGQKUABJE.yaml"
[component_container_mt-1] [INFO] [1736445934.378839188] [DisparityToDepthNode]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1736445934.379464189] [DisparityToDepthNode]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1736445934.380082882] [DisparityToDepthNode]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1736445934.616866632] [ess_disparity]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/JBEITDMOVZ/JBEITDMOVZ.yaml"
[component_container_mt-1] [INFO] [1736445934.616927879] [ess_disparity]: [ESSDisparityNode] preLoadGraphCallback().
[component_container_mt-1] [INFO] [1736445934.616970311] [ess_disparity]: [ESSDisparityNode] Setting tensorrt_plugin: /workspaces/isaac_ros-dev/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.1.0_onnx/plugins/x86_64/ess_plugins.so.
[component_container_mt-1] [INFO] [1736445934.616976695] [ess_disparity]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1736445934.619586702] [ess_disparity]: [ESSDisparityNode] Setting engine_file_path: /workspaces/isaac_ros-dev/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.1.0_onnx/ess.engine.
[component_container_mt-1] [INFO] [1736445934.619604272] [ess_disparity]: [ESSDisparityNode] postLoadGraphCallback() with image [576 x 960]
[component_container_mt-1] [INFO] [1736445934.619630178] [ess_disparity]: [ESSDisparityNode] postLoadGraphCallback() block_size = 2211840.
[component_container_mt-1] [INFO] [1736445934.619633252] [ess_disparity]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1736445934.620671383] [ess_disparity]: [NitrosNode] Node was started
[component_container_mt-1] hasImplicitBatchDimension is deprecated and always return false.
My system has a NVIDIA GeForce RTX 3070 Ti and my environment is running TensorRT 10.3.0. All the data seems to flow as expected through the ROS network. Changing the confidence threshold doesn’t give any different results. I had no issue at all running this demo (as-is) with the previous release (using model v4.0.0). I’ve also posted the issue in Github.
Has anyone else come across this issue? I would appreciate any insights on where I am going wrong about this. Thank you!