Isaac_ros_dnn_image_encoder cropping method

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

I am using the isaac_ros_dnn_image_encoder to convert a RGB8 image to a nitros_tensor_list_nchw_rgb_f32 Tensor.

isaac_ros_dnn_image_encoder — isaac_ros_docs documentation states that the image is cropped, not resized to match the network_image dimensions.

To test this, I encode a message, and then decode it back to a sensor_msgs/msg/Image.msg
On the left is the input image (1920x1080), and on the right is the Tensor, decoded back to an Image (896x896).
It can be seen that the output image is not cropped, but resized with the aspect ratio retained. (note the right image is 1:1 with letter-boxed black bars, while the left image is 16:9)

Screenshot 2024-03-06 at 4.24.12 PM
Screenshot 2024-03-06 at 4.24.12 PM1656×845 34.1 KB

    image_encoder_node = ComposableNode(
        name="video_dnn_encoder",
        namespace=[LaunchConfiguration("ns"), "/panorama_server"],
        package="isaac_ros_dnn_image_encoder",
        plugin="nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode",
        parameters=[
            {
                "input_image_width" : 1920,
                "input_image_height" : 1080,
                "network_image_width": 896,
                "network_image_height": 896,
                "image_mean": [0.0, 0.0, 0.0],
                "image_stddev": [
                    PIXEL_SCALE_INVERSE,
                    PIXEL_SCALE_INVERSE,
                    PIXEL_SCALE_INVERSE,
                ],
            }
        ],
        remappings=[
            ("encoded_tensor", "tensor_pub"),
            ("image", ["/", LaunchConfiguration("ns"), "/", VIDEO_INPUT_TOPIC]),
        ],
    )

Could the docs be updated if I am interpreting this behaviour correctly?

2

Thank you for your question. We need to update the documentation and what is happening is it is applying both resize and crop operation to get the final image dim. The enable_padding parameter is enabled by default which adds the black bars to help maintain the aspect ratio. If you disable it, you will see the final image without black bars with a shear effect on the image dims.

3

Hey Swapnesh, that makes sense. I should’ve checked the nodes parameters. bbox_loc is then not used by the node.

: ros2 param list
/nvidia_dnn_encoder:
 enable_padding
 enable_statistics
 extension_log_level
 image_mean
 image_stddev
 input_image_height
 input_image_width
 network_image_height
 network_image_width
 num_blocks