Image format for isaac NIT ROS and ROS2

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

Refer to nitros to create the image nitros_pub_, and set the image data format to nvidia::isaac_ros::nitros::nitros_image_nv12_t. The output of ordinary ROS2 topic channels is always in rgb format. Is there any configuration that can be set to make the topic data format of ordinary ros consistent with that of nitros?

Supplementary description:
After nitros_pub_ is created, the following two topics will be found after the cmd “ros2 topic list” done.

/left/image_raw
/left/image_raw/nitros

The encoding of /left/image_raw has always been rgb8, No matter what format it is set to nvidia::isaac_ros::nitros::nitros_image.
And I have already using this function with a different encoding but it’s still showing up as RGB8,

reference: isaac_ros_nitros/isaac_ros_managed_nitros_examples/custom_nitros_image/include/custom_nitros_image/gpu_image_builder_node.hpp at main · NVIDIA-ISAAC-ROS/isaac_ros_nitros · GitHub

2

I have tried the following two configurations, and what I got was still rgb8 from the normal ROS2 topic

// Create NitrosImage wrapping CUDA buffer
  nvidia::isaac_ros::nitros::NitrosImage nitros_image =
    nvidia::isaac_ros::nitros::NitrosImageBuilder()
    .WithHeader(header)
    .WithEncoding(img_encodings::NV24)
    .WithDimensions(msg->height, msg->width)
    .WithGpuData(buffer)
    .Build();

// Create NitrosImage wrapping CUDA buffer
  nvidia::isaac_ros::nitros::NitrosImage nitros_image =
    nvidia::isaac_ros::nitros::NitrosImageBuilder()
    .WithHeader(header)
    .WithEncoding("nv12")
    .WithDimensions(msg->height, msg->width)
    .WithGpuData(buffer)
    .Build();
3

Hi @bitren ,

Thank you for your detailed post.

I see you are testing changing the variable and the encoding, but it’s still showing up as RGB8. Looks like it may be a bug from our side.

I will keep you posted.

Best,
Raffaello

5

@Raffaello

Have you checked this already?

6

Hi @bitren

The engineers are working on this bug. I will keep you posted with any updates.

Best,
Raffaello

7

We have reviewed our NITROS image implementations, and we can confirm that the observed behavior is expected.

The NV12 and NV24 formats are treated as special cases during type adaptation. When using ros2 topic echo to subscribe to these image types, type adaptation is triggered. This process converts the message to a standard ROS image and changes the color format to rgb8 for NV12 and NV24 formats. This is likely due to NV12 encoding not being natively supported by ROS2 before the rolling update.

Here’s what this means:

  • If you use formats other than NV12 and NV24 (e.g., bgr8), the encoding will be preserved accurately.
  • For messages exchanged between NITROS nodes, NV12 and NV24 encodings are preserved and will be processed as expected.
  • However, if you attempt to convert a NITROS message to a standard ROS message—such as when using ros2 topic echo—the type adaptation will convert NV12/NV24 images to rgb8.

Best,
Raffaello