NITROS negotiation failed in 'isaac_ros_dnn_stereo_depth'

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

We have managed to run isaac_ros_dnn_stereo_depth on ORIN NANO(8GB) / JP5.1.2, with HAWK stereo camera, although the disparity fps is much lower than that on ORIN.

Here we have several questions about NITROS and ESS DNN:

  1. No outputs from cmd ‘ros2 topic hz /disparity/nitros’, or /left/image_raw/nitros, /right/image_raw/nitros, and other topics ended with /nitros.

Instead, we can get outputs from ‘ros2 topic hz /disparity’, and other topics not ended with ‘/nitros’.

Why is there no ‘nitros’ topic/msg, if the msg is transported by NITROS?

Here is the list of ROS2 topics from ‘ros2 topic list’:
/disparity
/disparity/nitros
/left/camera_info_rect
/left/camera_info_rect/nitros
/left/camerainfo
/left/camerainfo/nitros
/left/camerainfo_resize
/left/camerainfo_resize/nitros
/left/image_raw
/left/image_raw/nitros
/left/image_rect
/left/image_rect/nitros
/left/image_resize
/left/image_resize/nitros
/left/image_resize/nitros/nitros_image_rgb8
/parameter_events
/points2
/points2/nitros
/right/camera_info_rect
/right/camera_info_rect/nitros
/right/camerainfo
/right/camerainfo/nitros
/right/camerainfo_resize
/right/camerainfo_resize/nitros
/right/image_raw
/right/image_raw/nitros
/right/image_rect
/right/image_rect/nitros
/right/image_rect/nitros/nitros_image_rgb8
/right/image_resize
/right/image_resize/nitros
/right/image_resize/nitros/nitros_image_rgb8
/rosout
/tf

  1. Some type negotiation failed in NITROS

From the log file of the ESS DNN application:

[INFO] [1714056728.410497542] [left_rectify_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[INFO] [1714056728.412414512] [left_rectify_node]: [NitrosNode] Creating negotiated publishers/subscribers
[INFO] [1714056728.424427220] [left_rectify_node]: [NitrosNode] Starting negotiation…
[INFO] [1714056728.430721821] [left_rectify_node]: Negotiating
[INFO] [1714056728.430786366] [left_rectify_node]: Could not negotiate
[INFO] [1714056728.430817759] [left_rectify_node]: Negotiating
[INFO] [1714056728.430833375] [left_rectify_node]: Could not negotiate
[INFO] [1714056729.000550111] [left_rectify_node]: Negotiating
[INFO] [1714056729.006689348] [left_rectify_node]: Negotiating
[INFO] [1714056729.259069732] [left_rectify_node]: Negotiating
[INFO] [1714056729.259222887] [left_rectify_node]: Could not negotiate
[INFO] [1714056729.261737918] [left_rectify_node]: Negotiating
[INFO] [1714056729.261828800] [left_rectify_node]: Could not negotiate
[INFO] [1714056729.262812789] [left_rectify_node]: Negotiating
[INFO] [1714056729.263392514] [left_rectify_node]: Negotiating
[INFO] [1714056729.426119082] [left_rectify_node]: [NitrosNode] Starting post negotiation setup
[INFO] [1714056729.426255053] [left_rectify_node]: [NitrosNode] Getting data format negotiation results
[INFO] [1714056729.426280718] [left_rectify_node]: [NitrosPublisher] Negotiation ended with no results
[INFO] [1714056729.426299214] [left_rectify_node]: [NitrosPublisher] Use only the compatible publisher: topic_name=“/left/image_rect”, data_format=“nitros_image_bgr8”
[INFO] [1714056729.426317614] [left_rectify_node]: [NitrosPublisher] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056729.426333327] [left_rectify_node]: [NitrosSubscriber] Use the negotiated data format: “nitros_image_rgb8”
[INFO] [1714056729.428122677] [left_rectify_node]: [NitrosSubscriber] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056729.428241976] [left_rectify_node]: [NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component “image_sink/sink” (type=“nvidia::isaac_ros::MessageRelay”) from “nitros_image_bgr8” to “nitros_image_rgb8”

[INFO] [1714056730.000420045] [disparity]: [NitrosNode] Starting post negotiation setup
[INFO] [1714056730.000553264] [disparity]: [NitrosNode] Getting data format negotiation results
[INFO] [1714056730.000582897] [disparity]: [NitrosPublisher] Use the negotiated data format: “nitros_disparity_image_32FC1”
[INFO] [1714056730.000602641] [disparity]: [NitrosSubscriber] Negotiation ended with no results
[INFO] [1714056730.000617266] [disparity]: [NitrosSubscriber] Use the compatible subscriber: topic_name=“/left/image_rect”, data_format=“nitros_image_rgb8”
[INFO] [1714056730.000631794] [disparity]: [NitrosSubscriber] Use the negotiated data format: “nitros_image_rgb8”
[INFO] [1714056730.000646770] [disparity]: [NitrosSubscriber] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056730.000659923] [disparity]: [NitrosSubscriber] Use the negotiated data format: “nitros_camera_info”
[INFO] [1714056730.001036987] [disparity]: [NitrosNode] Exporting the final graph based on the negotiation results
[INFO] [1714056730.194574827] [disparity]: [NitrosNode] Wrote the final top level YAML

Why did these NITROS negotiations fail?
Is there anything wrong with the application?

2

Hi @jiangjie1

Let me reply in line your questions

When starting a negotiation, topics with the /nitros suffix are used, and the output comes on the original topic.

We fixed a minor bug related to negotiation for the next release. However, the current graph can still function normally even if negotiation for error messages fails.

3

So the underlying msg is ACTUALLY transferred by NITROS, when the negotiations succeeded？

Yes, even in the case of a negotiation error, the DNN graphy can still function normally,
i.e, the graph produces disparity as expected.

But how about the msg communication efficiency？
Is it as high as that when NITROS negotiation is completed successfully?

4

Yes, all messages between NITROS-based nodes will be transported by NITROS regardless of the negotiation results and the message transportation efficiency is as high as it can be in either case.

In NITROS, type negotiation is used to come up with format consensus between NITROS nodes and then help each node set up itself with this information if applicable.

Specifically to this graph, the failed negotiation doesn’t affect the graph’s functionality and performance due to how the related nodes are implemented to tolerate the failed negotiation and identify the actual format of the incoming messages and handle them correctly.