Visual Global Localization with cuVGL: How to Convert a ROS Bag to the cuVGL Data Format?

[andreas.kradolfer]

I’m using a single ZED X Mini stereo camera with the Jetson Orin as my setup. So far, VSLAM and the ZED node are running successfully. I want to integrate Isaac ROS Visual Global Localization into the existing project to determine the robot’s initial position at system startup.

As far as I understand, I should first use the Isaac Mapping ROS package and the rosbag_to_mapping_data tool to extract keyframes from the rosbag. However, I couldn’t find a specific list of required topics for data conversion in the documentation. Currently, I’m using:

ros2 bag record -o  26-03-2025-image-rosbag-1 \
/camera_front/zed_node/left_gray/camera_info \
/camera_front/zed_node/left_gray/image_compressed \
/camera_front/zed_node/right_gray/camera_info \
/camera_front/zed_node/right_gray/image_compressed \
/tf \
/tf_static \
/camera_front/zed_node/pose \
/camera_front/zed_node/pose/status \
/camera_front/zed_node/pose_with_covariance \
/visual_slam/vis/slam_odometry \
/rosout \
/diagnostics \
/camera_front/zed_node/imu/data

and for the rosbag_to_mapping_data:

ros2 run isaac_mapping_ros rosbag_to_mapping_data \
    --sensor_data_bag_file "/workspaces/isaac_ros-dev/src/fs_25_andreaskradolfer/viras_ros_global_localization/rosbag/26-03-2025-image-rosbag-1/26-03-2025-image-rosbag-1_0.db3" \
    --output_folder_path "/workspaces/isaac_ros-dev/src/fs_25_andreaskradolfer/viras_ros_global_localization/map/keyframes" \
    --pose_bag_file "/workspaces/isaac_ros-dev/src/fs_25_andreaskradolfer/viras_ros_global_localization/rosbag/2-03-2025-image-rosbag-1/26-03-2025-image-rosbag-1_0.db3" \
    --pose_topic_name "/visual_slam/vis/slam_odometry"

I receive the following error message:

W0326 09:49:38.942021 48382 video_decoder.cpp:84] Error sending packet for decoding
W0326 09:49:38.942160 48382 data_converter_utils.cpp:1000] Failed to decode frames, No frames decoded
W0326 09:49:38.941625 48383 video_decoder.cpp:84] Error sending packet for decoding
W0326 09:49:38.942505 48383 data_converter_utils.cpp:1000] Failed to decode frames, No frames decoded
[h264 @ 0xaaaada570550] A non-intra slice in an IDR NAL unit.
[h264 @ 0xaaaada570550] decode_slice_header error
[h264 @ 0xaaaada570550] no frame!
[h264 @ 0xaaaada868030] A non-intra slice in an IDR NAL unit.
[h264 @ 0xaaaada868030] decode_slice_header error
[h264 @ 0xaaaada868030] W0326 09:49:38.942885 48382 video_decoder.cpp:84] Error sending packet for decoding
no frame!
W0326 09:49:38.943284 48382 data_converter_utils.cpp:1000] Failed to decode frames, No frames decoded
W0326 09:49:38.943321 48383 video_decoder.cpp:84] Error sending packet for decoding
W0326 09:49:38.943658 48383 data_converter_utils.cpp:1000] Failed to decode frames, No frames decoded
[h264 @ 0xaaaada868030] A non-intra slice in an IDR NAL unit.
[h264 @ 0xaaaada868030] decode_slice_header error
[h264 @ 0xaaaada868030] no frame!
W0326 09:49:38.944340 48383 video_decoder.cpp:84] Error sending packet for decoding
W0326 09:49:38.944453 48383 data_converter_utils.cpp:1000] Failed to decode frames, No frames decoded
I0326 09:49:38.944540 48335 data_converter_utils.cpp:1204] For camera camera_front_zed_node_left_gray: got message 159, decoded: 0, total image written: 0
I0326 09:49:38.944586 48335 data_converter_utils.cpp:1204] For camera camera_front_zed_node_right_gray: got message 159, decoded: 0, total image written: 0
I0326 09:49:38.981148 48335 rosbag_to_mapping_data.cpp:374] Converting edex files
I0326 09:49:38.981278 48335 keyframe_metadata.cc:130] Detected this is a old frame metadata formatrunning image sync to update to new format.
F0326 09:49:38.981321 48335 keyframe_edex_utils.cc:101] Check failed: track_id_to_synced_sample_id_to_frame_ids.size() == 1 (0 vs. 1) Edex converter only support converting one track
*** Check failure stack trace: ***
    @     0xffff9f4fd41c  google::LogMessage::Fail()
    @     0xffff9f5046d0  google::LogMessage::SendToLog()
    @     0xffff9f4fd0f4  google::LogMessage::Flush()
    @     0xffff9f4feebc  google::LogMessageFatal::~LogMessageFatal()
    @     0xaaaab0da1724  (unknown)
    @     0xaaaab0d8a0bc  (unknown)
    @     0xffff8d7e73fc  (unknown)
    @     0xffff8d7e74cc  __libc_start_main
    @     0xaaaab0d8c4f0  (unknown)
[ros2run]: Aborted
admin@viras32:/workspaces/isaac_ros-dev$ 

Has anyone encountered similar error messages? Is there any additional documentation available for implementing global localization with cuVGL?

Visual Global Localization — isaac_ros_docs documentation]

Isaac ROS Mapping And Localization — isaac_ros_docs documentation

[Raffaello]

Hi @andreas.kradolfer

This error looks strange also to me. I forward to the engineering and I let you know.

Best,
Raffaello

[andreas.kradolfer]

I’ve found a workaround to get the data conversion running by using the raw/image_rect_gray topics.

For the next step—extracting image features with feature_extractor_main—I should have a frames_metadata.pb.txt file in the input_image_directory, but I only have a frames_metadata.jsonl file.

Additionally, I need to define a keypoint_creation_config.pb.txt file, but I’m unsure what its structure should be. I couldn’t find any hints in the documentation regarding this.

[dizeng]

Hi Andreas,
The issue is related to

I0326 09:49:38.944540 48335 data_converter_utils.cpp:1204] For camera camera_front_zed_node_left_gray: got message 159, decoded: 0, total image written: 0

It seems it failed to decode images.
If the compressed images are not h264 encoding, please try provide --video_codec_name XXX

Using image_raw is also fine, in that case it bypasses the image decoding.

keypoint_creation_config.pb.txt is under /opt/ros/humble/share/isaac_mapping_ros/configs/ as you probably already found in the other ticket.

in python we get the config folder by

        config_folder = pathlib.Path(
            os.path.join(
                subprocess.getoutput('ros2 pkg prefix isaac_mapping_ros'),
                'share/isaac_mapping_ros/configs'))