NITROS negotiation failed in 'isaac_ros_dnn_stereo_depth'

We have managed to run isaac_ros_dnn_stereo_depth on ORIN NANO(8GB) / JP5.1.2, with HAWK stereo camera, although the disparity fps is much lower than that on ORIN.

Here we have several questions about NITROS and ESS DNN:

  1. No outputs from cmd ‘ros2 topic hz /disparity/nitros’, or /left/image_raw/nitros, /right/image_raw/nitros, and other topics ended with /nitros.

Instead, we can get outputs from ‘ros2 topic hz /disparity’, and other topics not ended with ‘/nitros’.

Why is there no ‘nitros’ topic/msg, if the msg is transported by NITROS?

Here is the list of ROS2 topics from ‘ros2 topic list’:
/disparity
/disparity/nitros
/left/camera_info_rect
/left/camera_info_rect/nitros
/left/camerainfo
/left/camerainfo/nitros
/left/camerainfo_resize
/left/camerainfo_resize/nitros
/left/image_raw
/left/image_raw/nitros
/left/image_rect
/left/image_rect/nitros
/left/image_resize
/left/image_resize/nitros
/left/image_resize/nitros/nitros_image_rgb8
/parameter_events
/points2
/points2/nitros
/right/camera_info_rect
/right/camera_info_rect/nitros
/right/camerainfo
/right/camerainfo/nitros
/right/camerainfo_resize
/right/camerainfo_resize/nitros
/right/image_raw
/right/image_raw/nitros
/right/image_rect
/right/image_rect/nitros
/right/image_rect/nitros/nitros_image_rgb8
/right/image_resize
/right/image_resize/nitros
/right/image_resize/nitros/nitros_image_rgb8
/rosout
/tf

  1. Some type negotiation failed in NITROS

From the log file of the ESS DNN application:

[INFO] [1714056728.410497542] [left_rectify_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[INFO] [1714056728.412414512] [left_rectify_node]: [NitrosNode] Creating negotiated publishers/subscribers
[INFO] [1714056728.424427220] [left_rectify_node]: [NitrosNode] Starting negotiation…
[INFO] [1714056728.430721821] [left_rectify_node]: Negotiating
[INFO] [1714056728.430786366] [left_rectify_node]: Could not negotiate
[INFO] [1714056728.430817759] [left_rectify_node]: Negotiating
[INFO] [1714056728.430833375] [left_rectify_node]: Could not negotiate
[INFO] [1714056729.000550111] [left_rectify_node]: Negotiating
[INFO] [1714056729.006689348] [left_rectify_node]: Negotiating
[INFO] [1714056729.259069732] [left_rectify_node]: Negotiating
[INFO] [1714056729.259222887] [left_rectify_node]: Could not negotiate
[INFO] [1714056729.261737918] [left_rectify_node]: Negotiating
[INFO] [1714056729.261828800] [left_rectify_node]: Could not negotiate
[INFO] [1714056729.262812789] [left_rectify_node]: Negotiating
[INFO] [1714056729.263392514] [left_rectify_node]: Negotiating
[INFO] [1714056729.426119082] [left_rectify_node]: [NitrosNode] Starting post negotiation setup
[INFO] [1714056729.426255053] [left_rectify_node]: [NitrosNode] Getting data format negotiation results
[INFO] [1714056729.426280718] [left_rectify_node]: [NitrosPublisher] Negotiation ended with no results
[INFO] [1714056729.426299214] [left_rectify_node]: [NitrosPublisher] Use only the compatible publisher: topic_name=“/left/image_rect”, data_format=“nitros_image_bgr8”
[INFO] [1714056729.426317614] [left_rectify_node]: [NitrosPublisher] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056729.426333327] [left_rectify_node]: [NitrosSubscriber] Use the negotiated data format: “nitros_image_rgb8”
[INFO] [1714056729.428122677] [left_rectify_node]: [NitrosSubscriber] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056729.428241976] [left_rectify_node]: [NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component “image_sink/sink” (type=“nvidia::isaac_ros::MessageRelay”) from “nitros_image_bgr8” to “nitros_image_rgb8”

[INFO] [1714056730.000420045] [disparity]: [NitrosNode] Starting post negotiation setup
[INFO] [1714056730.000553264] [disparity]: [NitrosNode] Getting data format negotiation results
[INFO] [1714056730.000582897] [disparity]: [NitrosPublisher] Use the negotiated data format: “nitros_disparity_image_32FC1”
[INFO] [1714056730.000602641] [disparity]: [NitrosSubscriber] Negotiation ended with no results
[INFO] [1714056730.000617266] [disparity]: [NitrosSubscriber] Use the compatible subscriber: topic_name=“/left/image_rect”, data_format=“nitros_image_rgb8”
[INFO] [1714056730.000631794] [disparity]: [NitrosSubscriber] Use the negotiated data format: “nitros_image_rgb8”
[INFO] [1714056730.000646770] [disparity]: [NitrosSubscriber] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056730.000659923] [disparity]: [NitrosSubscriber] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056730.001036987] [disparity]: [NitrosNode] Exporting the final graph based on the negotiation results
[INFO] [1714056730.194574827] [disparity]: [NitrosNode] Wrote the final top level YAML

Why did these NITROS negotiations fail?
Is there anything wrong with the application?

Hi @jiangjie1

Let me reply in line your questions

When starting a negotiation, topics with the /nitros suffix are used, and the output comes on the original topic.

We fixed a minor bug related to negotiation for the next release. However, the current graph can still function normally even if negotiation for error messages fails.

So the underlying msg is ACTUALLY transferred by NITROS, when the negotiations succeeded?

Yes, even in the case of a negotiation error, the DNN graphy can still function normally,
i.e, the graph produces disparity as expected.

But how about the msg communication efficiency?
Is it as high as that when NITROS negotiation is completed successfully?