Nova-devkit bmi088 imu problem

hi, I have nova orin dev kit and recently encountered a problem when launching a bmi088 sensor on both front_stereo_imu(id=69, hawk stereo camera) and chasis_imu(id=68).

After launching a ros2 launch isaac_ros_imu_bmi088 bmi088.launch.py bmi_id:=68 diagnostics:=True, messages are being published on topic /imu/imu for varied period of time from 1 up to 20 minutes until it abruptly stops without any errors in the launch logs. The launch file is the standard one from the isaac_ros_nova repo
The /diagnostics would then change from OK to DIAGNOSTICS STALE:

---
header:
  stamp:
    sec: 607
    nanosec: 1419741671
  frame_id: ''
status:
- level: "\0"
  name: bmi088/imu/imu
  message: OK
  hardware_id: nvidia
  values:
  - key: num_non_increasing_msg
    value: '0'
  - key: num_jitter_outliers_msg
    value: '0'
  - key: num_jitter_outliers_node
    value: '0'
  - key: max_abs_jitter_msg
    value: '18'
  - key: max_abs_jitter_node
    value: '0'
  - key: mean_abs_jitter_msg
    value: '17'
  - key: mean_abs_jitter_node
    value: '0'
  - key: frame_rate_msg
    value: '99.828293'
  - key: frame_rate_node
    value: '99.825302'
  - key: total_dropped_frames
    value: '0'
---
header:
  stamp:
    sec: 608
    nanosec: 2419729304
  frame_id: ''
status:
- level: "\x03"
  name: bmi088/imu/imu
  message: DIAGNOSTICS STALE
  hardware_id: nvidia
  values: []
---
header:
  stamp:
    sec: 609
    nanosec: 3419724171
  frame_id: ''
status:
- level: "\x03"
  name: bmi088/imu/imu
  message: DIAGNOSTICS STALE
  hardware_id: nvidia
  values: []
---

launch file console output

admin@nvidia-desktop:/workspaces/isaac_ros-dev$ ros2 launch isaac_ros_imu_bmi088 bmi088.launch.py  bmi_id:=68 diagnostics:=True
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2025-07-17-12-47-27-439709-nvidia-desktop-49453
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [49485]
[component_container_mt-1] [INFO] [1752745647.827934169] [bmi088_container]: Load Library: /opt/ros/humble/lib/libcorrelated_timestamp_driver_node.so
[component_container_mt-1] [INFO] [1752745647.870428375] [bmi088_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::correlated_timestamp_driver::CorrelatedTimestampDriverNode>
[component_container_mt-1] [INFO] [1752745647.870510231] [bmi088_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::correlated_timestamp_driver::CorrelatedTimestampDriverNode>
[component_container_mt-1] [INFO] [1752745647.881103478] [correlated_timestamp_driver]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1752745647.881360790] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1752745647.882732982] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_gxf_helpers.so
[component_container_mt-1] [INFO] [1752745647.886905910] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_sight.so
[component_container_mt-1] [INFO] [1752745647.891499254] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_atlas.so
[component_container_mt-1] 2025-07-17 12:47:27.902 WARN  gxf/std/program.cpp@538: No GXF scheduler specified.
[component_container_mt-1] [INFO] [1752745647.903356021] [correlated_timestamp_driver]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1752745647.906589301] [correlated_timestamp_driver]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1752745647.906735445] [correlated_timestamp_driver]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1752745647.909766229] [correlated_timestamp_driver]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_message_compositor.so
[component_container_mt-1] [INFO] [1752745647.910239829] [correlated_timestamp_driver]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_timestamp_correlator.so
[component_container_mt-1] [INFO] [1752745647.918442389] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1752745647.919022837] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages.so
[component_container_mt-1] [INFO] [1752745647.919583509] [correlated_timestamp_driver]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1752745647.920667509] [correlated_timestamp_driver]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1752745647.922024789] [correlated_timestamp_driver]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1752745647.924464149] [correlated_timestamp_driver]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1752745647.925449429] [bmi088_container]: Load Library: /opt/ros/humble/lib/libimu_bmi088_node.so
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/correlated_timestamp_driver' in container 'bmi088_container'
[component_container_mt-1] [INFO] [1752745647.927828372] [bmi088_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::imu_bmi088::Bmi088Node>
[component_container_mt-1] [INFO] [1752745647.927871380] [bmi088_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::imu_bmi088::Bmi088Node>
[component_container_mt-1] [INFO] [1752745647.932501364] [imu.bmi088]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1752745647.933017652] [imu.bmi088]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1752745647.937208820] [imu.bmi088]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1752745647.937362836] [imu.bmi088]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_bmi088_imu.so
[component_container_mt-1] [INFO] [1752745647.938505908] [imu.bmi088]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_imu_utils.so
[component_container_mt-1] [INFO] [1752745647.939178868] [imu.bmi088]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1752745647.943168148] [imu.bmi088]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1752745647.974325330] [imu.bmi088]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1752745647.990346642] [imu.bmi088]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1752745647.990890322] [correlated_timestamp_driver]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/imu/bmi088' in container 'bmi088_container'
[component_container_mt-1] [INFO] [1752745648.925155607] [correlated_timestamp_driver]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1752745648.925233911] [correlated_timestamp_driver]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1752745648.925258295] [correlated_timestamp_driver]: [NitrosPublisher] Use the negotiated data format: "nitros_correlated_timestamp"
[component_container_mt-1] [INFO] [1752745648.925485847] [correlated_timestamp_driver]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1752745648.927242296] [correlated_timestamp_driver]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/DEHMQAMDNA/DEHMQAMDNA.yaml"
[component_container_mt-1] [INFO] [1752745648.927322936] [correlated_timestamp_driver]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1752745648.928512888] [correlated_timestamp_driver]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1752745648.929421336] [correlated_timestamp_driver]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1752745652.990880971] [imu.bmi088]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1752745652.990957067] [imu.bmi088]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1752745652.990972011] [imu.bmi088]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1752745652.990992555] [imu.bmi088]: [NitrosPublisher] Use only the compatible publisher: topic_name="/imu/imu", data_format="nitros_imu"
[component_container_mt-1] [INFO] [1752745652.991016107] [imu.bmi088]: [NitrosSubscriber] Use the negotiated data format: "nitros_correlated_timestamp"
[component_container_mt-1] [INFO] [1752745652.991187371] [imu.bmi088]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1752745652.997819595] [imu.bmi088]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/STLKBBJPNZ/STLKBBJPNZ.yaml"
[component_container_mt-1] [INFO] [1752745652.997874059] [imu.bmi088]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1752745653.001371627] [imu.bmi088]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1752745653.004411019] [imu.bmi088]: [NitrosNode] Node was started
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)
[component_container_mt-1] [INFO] [1752746446.123469129] [rclcpp]: signal_handler(signum=2)
[component_container_mt-1] [INFO] [1752746446.126196009] [imu.bmi088]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1752746446.126222409] [imu.bmi088]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1752746446.164227304] [imu.bmi088]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1752746446.164245320] [imu.bmi088]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1752746446.165097416] [imu.bmi088]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1752746446.165354632] [imu.bmi088]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1752746446.170410407] [correlated_timestamp_driver]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1752746446.170429447] [correlated_timestamp_driver]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2025-07-17 13:00:46.164 WARN  gxf/std/double_buffer_receiver.cpp@40: Unprocessed num of message 10 in queue: STLKBBJPNZ_imu_combiner:rx_gyroscope
[component_container_mt-1] 2025-07-17 13:00:46.164 WARN  gxf/std/double_buffer_receiver.cpp@40: Unprocessed num of message 10 in queue: STLKBBJPNZ_imu_combiner:rx_accelerometer
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |                                    STLKBBJPNZ_sink |   60073 |     0.03 |     0.03 |     0.11 |    0.3 % |   1839.1 |         4608 |         5248 |        58912 |
[component_container_mt-1] |                        STLKBBJPNZ_accel_translator |   79555 |     0.05 |     0.05 |     0.19 |    0.5 % |   4155.6 |        11200 |        16320 |        98240 |
[component_container_mt-1] |                            STLKBBJPNZ_imu_combiner | 6190192 |     0.01 |     0.02 |     0.79 |   11.5 % |   91428.2 |   92418665607 |   160025060609 |   191154869361 |
[component_container_mt-1] |                         STLKBBJPNZ_gyro_translator |   79162 |     0.04 |     0.05 |     0.30 |    0.5 % |   3631.0 |        11872 |        14144 |       250752 |
[component_container_mt-1] |                           STLKBBJPNZ_bmi088_driver |  158737 |     0.06 |     0.07 |     0.26 |    1.1 % |   8781.1 |            0 |            0 |            0 |
[component_container_mt-1] |                    STLKBBJPNZ_correlator_broadcast |     775 |     0.04 |     0.05 |     0.10 |    0.0 % |     33.8 |        19392 |        23264 |        80576 |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |         STLKBBJPNZ_sink  |              StopPending |       1 |  0.00669 |  0.00669 |  0.00669                                                                   |
[component_container_mt-1] |         STLKBBJPNZ_sink  |                     Idle |   60073 |  9.99050 | 10.01830 | 191212.67185                                                                   |
[component_container_mt-1] |         STLKBBJPNZ_sink  |                  Ticking |   60073 |  0.01875 |  0.02509 |  0.09872                                                                   |
[component_container_mt-1] |         STLKBBJPNZ_sink  |              TickPending |   60073 |  0.00435 |  0.00454 |  0.05210                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] | STLKBBJPNZ_accel_transl  |              StopPending |       1 |  0.00602 |  0.00602 |  0.00602                                                                   |
[component_container_mt-1] | STLKBBJPNZ_accel_transl  |                     Idle |   79555 |  9.93203 |  9.95824 | 42.49328                                                                   |
[component_container_mt-1] [INFO] [1752746446.612501978] [correlated_timestamp_driver]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1752746446.612528602] [correlated_timestamp_driver]: [NitrosContext] Deinitializing...
[component_container_mt-1] | STLKBBJPNZ_accel_transl  |                  Ticking |   79555 |  0.03405 |  0.03709 |  0.17142                                                                   |
[component_container_mt-1] | STLKBBJPNZ_accel_transl  |              TickPending |   79555 |  0.00838 |  0.01027 |  0.11290                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] | STLKBBJPNZ_imu_combiner  |              StopPending |       1 |  0.11437 |  0.11437 |  0.11437                                                                   |
[component_container_mt-1] | STLKBBJPNZ_imu_combiner  |                     Idle | 6190192 |  0.02314 |  0.02445 | 37.75133                                                                   |
[component_container_mt-1] | STLKBBJPNZ_imu_combiner  |                  Ticking | 6190192 |  0.00397 |  0.00413 |  0.37818                                                                   |
[component_container_mt-1] | STLKBBJPNZ_imu_combiner  |              TickPending | 6190192 |  0.00544 |  0.00589 |  0.09258                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] | STLKBBJPNZ_gyro_transla  |              StopPending |       1 |  0.01520 |  0.01520 |  0.01520                                                                   |
[component_container_mt-1] | STLKBBJPNZ_gyro_transla  |                     Idle |   79162 |  9.96665 | 10.00979 | 44.11011                                                                   |
[component_container_mt-1] | STLKBBJPNZ_gyro_transla  |                  Ticking |   79162 |  0.02938 |  0.03693 |  0.28352                                                                   |
[component_container_mt-1] | STLKBBJPNZ_gyro_transla  |              TickPending |   79162 |  0.00710 |  0.01005 |  0.06118                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] | STLKBBJPNZ_bmi088_drive  |              StopPending |       1 | 37.64099 | 37.64099 | 37.64099                                                                   |
[component_container_mt-1] | STLKBBJPNZ_bmi088_drive  |                     Idle |  158737 |  5.30019 |  8.34601 |  9.77149                                                                   |
[component_container_mt-1] | STLKBBJPNZ_bmi088_drive  |                  Ticking |  158737 |  0.04298 |  0.05046 |  0.18691                                                                   |
[component_container_mt-1] | STLKBBJPNZ_bmi088_drive  |              TickPending |  158737 |  0.00771 |  0.01091 |  0.11968                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] | STLKBBJPNZ_correlator_b  |              StopPending |       1 |  0.00496 |  0.00496 |  0.00496                                                                   |
[component_container_mt-1] | STLKBBJPNZ_correlator_b  |                     Idle |     775 | 1023.96918 | 1024.00328 | 1028.15016                                                                   |
[component_container_mt-1] | STLKBBJPNZ_correlator_b  |                  Ticking |     775 |  0.02352 |  0.02544 |  0.06355                                                                   |
[component_container_mt-1] | STLKBBJPNZ_correlator_b  |              TickPending |     775 |  0.01226 |  0.01456 |  0.05901                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] [INFO] [1752746446.612928602] [correlated_timestamp_driver]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1752746446.613061626] [correlated_timestamp_driver]: [NitrosNode] Application termination done
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |                                    DEHMQAMDNA_sink |     779 |     0.06 |     0.08 |     0.12 |    0.0 % |     49.6 |        24160 |        34432 |        58752 |
[component_container_mt-1] |                              DEHMQAMDNA_correlator |     779 |     0.11 |     0.12 |     0.17 |    0.0 % |     85.5 |            0 |            0 |            0 |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |         DEHMQAMDNA_sink  |              StopPending |       1 |  0.01709 |  0.01709 |  0.01709                                                                   |
[component_container_mt-1] |         DEHMQAMDNA_sink  |                     Idle |     779 | 1023.94686 | 1023.97955 | 1028.05026                                                                   |
[component_container_mt-1] |         DEHMQAMDNA_sink  |                  Ticking |     779 |  0.03520 |  0.04941 |  0.07866                                                                   |
[component_container_mt-1] |         DEHMQAMDNA_sink  |              TickPending |     779 |  0.01482 |  0.02019 |  0.04486                                                                   |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |   DEHMQAMDNA_correlator  |              StopPending |       1 | 441.80354 | 441.80354 | 441.80354                                                                   |
[component_container_mt-1] |   DEHMQAMDNA_correlator  |                     Idle |     779 | 1023.89368 | 1023.92843 | 1028.00764                                                                   |
[component_container_mt-1] |   DEHMQAMDNA_correlator  |                  Ticking |     779 |  0.07482 |  0.08006 |  0.12506                                                                   |
[component_container_mt-1] |   DEHMQAMDNA_correlator  |              TickPending |     779 |  0.02381 |  0.02659 |  0.04755                    admin@nvidia-desktop:/workspace

logs from GXF_LOG_LEVEL=debug ros2 launch ros2 launch isaac_ros_imu_bmi088 bmi088.launch.py bmi_id:=68 diagnostics:=True
bmi_gxf_log_debug_part1.log (97.9 MB)
bmi_gxf_log_debug_part2.log (2.5 MB)

Jetpack 6.2+b77
Ubuntu 22.04
ROS2 Humble
Kernel 5.15.148-tegra
Nova Version 1.3.2+b4
Nova Config nova-devkit

Hi @mdemid

Welcome to Isaac ROS forum and thanks for your post.

From the attached log, it seems that demo was stopped by ctrl-C from user. Otherwise, the demo shall keep running. Can you provide more infomation how you observed the stop before you enter ctrl-C? is it because no message is published to /imu/imu ?

Best,
Ahung

Hi @mdemid

We can reproduce the problem here. It stops publishing message to imu topic.
I forward this issue to developer.

Best,
Ahung

Hi @Ahung,

  1. Can you confirm if this is a software issue, not a hardware one? This will help us avoid bothering our supplier unnecessarily.

  2. Should this issue remain open until we get some feedback from the developer?

Thanks,
Maxim

Hi @mdemid

Unfortunately, I don’t have the update from developer yet. Yes, we can keep this issue open.

Best,
Ahung