Isaac ROS UNet launch file

Hello,

We are working on a Jetson Orin Nano (8 GB) and are building an image segmentation pipeline using the following Isaac ROS packages:

  • isaac_ros_dnn_image_encoder — to preprocess raw camera images

  • isaac_ros_tensor_rt — for GPU inference

  • isaac_ros_unet — to generate the final segmentation mask

The camera stream is 640×512, 15–30 FPS, BGR8.


Problem Description

The pipeline does not behave deterministically:

  • Sometimes it starts and runs correctly

  • Sometimes it fails to start properly

The failure is intermittent, even when no launch configuration is changed.


Observed Behavior

During the failure cases, we repeatedly see messages such as:

10.0s passed while waiting to push a message entity (eid=XXX) to the receiver cam_info_in

This happens even though the camera /image_raw and /camera_info topics are synchronized and publishing normally.


Logs and Launch File

Here are the logs with DEBUG level enabled:

isaac_logs.log (2.2 MB)

Here is the launch file defining the pipeline:

isaac_ros_unet_tensor_rt.launch.txt (21.2 KB)

I tried to add a delay for the encoder runtime, but it did not solve the issue.


Software Environment

Some additional info:

  • JetPack 6.0 (Jetson Linux 36.3)
  • Isaac ROS v3.1 (base container from official installation guide)
  • ROS 2 Humble

The nodes are running inside a single component_container_mt as recommended for performance.


Hypothesis

Since the issue is intermittent and sensitive to timing, it looks like the GXF / NITROS graph may not always initialize the internal components (e.g., image encoder ports) in a deterministic manner, and running two separate containers did not solve the issue.


Could you please confirm:

  1. Whether this is a known issue in Isaac ROS 3.1 / JetPack 6.0

  2. Whether there is a recommended workaround (e.g., splitting nodes into multiple containers, adding startup synchronization, adjusting NITROS configuration, etc.)

  3. Whether this behavior is expected if camera_info arrives slightly later than image frames

  4. Whether additional GXF debug flags can help trace the graph initialization order

Any guidance on achieving a deterministic startup for this pipeline would be greatly appreciated.

Thank you in advance!

Best regards,

Simone

Hello @simone_c,

Thanks for posting in the Isaac ROS forum!

The logs you shared indicate a race / readiness issue in the GXF–NITROS graph, not a random bug in TensorRT or UNet. It means that camera_info messages are arriving, but the downstream GXF graph (resize + encoder + UNet) isn’t “ready” or isn’t draining the cam_info_in receiver, so pushes time out and the graph stalls.

You could try below steps to mitigate the problem.

  1. Delay camera startup relative to the perception graph:
    Launch segmentation (encoder + TRT + UNet) first, then start the camera node after 5–10 seconds.

  2. Or configure encoder start delay (if available in your launch), e.g.:

    encoder_startup_delay: 10.0 # increase while debugging

    so the graph is fully negotiated before it binds to live topics.

  3. Temporarily reduce FPS (e.g. from 30 → 10–15) to reduce pressure while debugging.

It’s recommended to run these three packages step by step to help identify the root cause of the problem.

  • Run isaac_ros_dnn_image_encoder consuming your camera topics and verify it publishes encoded tensors at a stable rate with no cam_info_in warnings.

  • Then add isaac_ros_tensor_rt to feed the encoder output, ensuring steady inference and no GXF timeouts.

  • Finally, add isaac_ros_unet to generate the final segmentation mask.