Isaac_ros_nvblox obtacle detection issue

Hello,

I have a request for clarification regarding the isaac_ros_nvblox package and its integration with Nav2 for local obstacle detection and avoidance.

While utilizing a RealSense D435i camera (mounted ~0.33m from the ground) with the isaac_ros realsense_splitter package and remapping the topics so that nvblox_ros is receiving the streams, we have set up the robot so that it has a clear path in front of it, and configured nvblox with the following parameters:

nvblox_ros:
  ros__parameters:
    global_frame: 'odom'
    # miscellaneous
    voxel_size: 0.05
    use_tf_transforms: true
    # multi mapper
    mapping_type: "dynamic"  # ["static_tsdf", "static_occupancy"]
    connected_mask_component_size_threshold: 2000
    # esdf settings
    compu_esdf: true
    esdf_update_rate_hz: 10.0
    esdf_2d: true
    publish_esdf_distance_slice: true
    esdf_slice_height: 0.3
    esdf_2d_min_height: 0.1
    esdf_2d_max_height: 1.3
    # mesh settings
    compute_mesh: false
    mesh_update_rate_hz: 5.0
    # Rviz visualization
    slice_visualization_attachment_frame_id: "odom"
    slice_visualization_side_length: 10.0
    # color settings
    use_color: false
    max_color_update_hz: 5.0
    # depth settings
    use_depth: true
    max_depth_update_hz: 30.0
    # lidar settings
    use_lidar: false
    max_lidar_update_hz: 100.0
    lidar_width: 1800
    lidar_height: 31
    use_non_equal_vertical_fov_lidar_params: false
    # static occupancy
    static_occupancy_publication_rate_hz: 2.0
    # dynamic occupancy
    dynamic_occupancy_decay_rate_hz: 10.0
    # Input queues
    max_poll_rate_hz: 100.0
    maximum_sensor_message_queue_length: 30
    # Map clearing settings
    map_clearing_radius_m: 10.0 # no map clearing if < 0.0
    map_clearing_frame_id: "base_footprint"
    clear_outside_radius_rate_hz: 1.0
    # QoS settings
    depth_qos: "SENSOR_DATA"
    color_qos: "SENSOR_DATA"
    # Mapper(s) settings
    static_mapper:
      # mapper
      do_depth_preprocessing: false
      depth_preprocessing_num_dilations: 3
      # projective integrator (tsdf/color/occupancy)
      projective_integrator_max_integration_distance_m: 4.0
      # lidar_projective_integrator_max_integration_distance_m: 10.0
      projective_integrator_truncation_distance_vox: 4.0
      weighting_mode: "inverse_square_tsdf_distance_penalty"
      projective_integrator_max_weight: 5.0
      # freespace integrator
      max_tsdf_distance_for_occupancy_m: 0.15
      max_unobserved_to_keep_consecutive_occupancy_ms: 200
      min_duration_since_occupied_for_freespace_ms: 250
      min_consecutive_occupancy_duration_for_reset_ms: 2000
      check_neighborhood: true
      # occupancy integrator
      free_region_occupancy_probability: 0.45
      occupied_region_occupancy_probability: 0.55
      unobserved_region_occupancy_probability: 0.5
      occupied_region_half_width_m: 0.1
      # esdf integrator
      esdf_integrator_min_weight: 0.0001
      esdf_integrator_max_site_distance_vox: 5.0
      esdf_integrator_max_distance_m: 2.0
      # mesh integrator
      mesh_integrator_min_weight: 0.0001
      mesh_integrator_weld_vertices: true
      # tsdf decay integrator
      tsdf_decay_factor: 0.95
    dynamic_mapper:
      # occupancy integrator
      projective_integrator_max_integration_distance_m: 4.0
      projective_integrator_truncation_distance_vox: 4.0
      free_region_occupancy_probability: 0.2
      occupied_region_occupancy_probability: 0.9
      unobserved_region_occupancy_probability: 0.35
      occupied_region_half_width_m: 0.15
      # esdf integrator
      esdf_integrator_min_weight: 2.0
      esdf_integrator_max_distance_m: 2.0
      esdf_integrator_max_site_distance_vox: 5.0
      # occupancy decay integrator
      free_region_decay_probability: 0.55
      occupied_region_decay_probability: 0.25

To my understanding, the esdf settings parameters above define the boundaries and configuration for the obstacle detection. In my case, since the path ahead is completely clear for a number of meters, nvblox should be showing and clear path ahead. However, nvblox is instead identifying the path ahead as completely blocked (the local costmap visualization of the FOV in rviz is completely red), and the robot cannot navitgate…

Please clarify perhaps if any of the parameters are being misunderstood or not utilized correctly, or if there is some additional configuration needed here to get accurate results?

Thanks.

Hi @arik6

I noticed that the camera is mounted very low to the ground, close to the minimum sensor height. Have you tried testing at different heights?

Best,
Raffaello

Hi,

What is the ninimum sensors height in this case? Is that documented somewhere?

Hi @arik6

I’m still reproducing your bug on my setup, and I keep you posted.

My reference was on d435i documentation, where the minimal range is from .3 to 3m, and I thought if the camera viewing cone was too close to the ground.

Hi @arik6

I created a similar setup and did not notice any unusual behavior. Have you followed all the documentation on using nvblox with a Realsense camera? I have attached a link to all related documentation. It could be a symptom of incorrect installation of Realsense firmware

  1. RealSense Issues — isaac_ros_docs documentation
  2. Isaac ROS RealSense Setup — isaac_ros_docs documentation