When reproducing the tutorial “Tutorial for cuMotion with Perception” (version 3.1), the robot can follow the planned trajectory for movement but fails to perform real-time obstacle avoidance. Most importantly, I need to use the visual obstacle avoidance feature, which currently isn’t working - it can only avoid predefined obstacles in Rviz. I want to identify where the problem lies. The “depth with segmented robot” node in Rviz displays “no image” - could this be related? Many thanks if anyone can help resolve this issue! :)
The issue you’re experiencing with the visual obstacle avoidance feature may indeed be related to the “no image” display in the “depth with segmented robot” node in Rviz. This could indicate a problem with the camera feed or the depth sensor input, which is crucial for real-time obstacle detection. Ensure that the camera or sensor is properly configured and is transmitting data. You might want to check if the sensor drivers are correctly installed, the camera calibration is accurate, and the node is correctly subscribed to the appropriate topics. Debugging these configurations should help resolve the issue with visual obstacle avoidance.
Hi @wendymatsui and thank you for your suggestion @sitecountryy
Do you have other logs from the terminals? Can help us to figure out where the issue is coming from.
I would like also to share that we made some improvements in Isaac ROS 3.2 related to cumotion, that can also impact the performance of the node Release Notes — isaac_ros_docs documentation
Best,
Raffaello
Here is part of the log content:
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-2]: process started with pid [1218549]
[INFO] [component_container_mt-1]: process started with pid [1218547]
[INFO] [cumotion_planner_node-3]: process started with pid [1218551]
[INFO] [robot_segmenter_node-4]: process started with pid [1218553]
[INFO] [isaac_ros_moveit_goal_setter-5]: process started with pid [1218555]
[INFO] [pose_to_pose.py-6]: process started with pid [1218557]
[INFO] [static_transform_publisher-7]: process started with pid [1218559]
[INFO] [static_transform_publisher-8]: process started with pid [1218568]
[INFO] [static_transform_publisher-9]: process started with pid [1218571]
[INFO] [rviz2-10]: process started with pid [1218580]
[static_transform_publisher-8] [INFO] [1740554934.685465923] [static_transform_publisher_k1rFvnwZXrCOkaur]: Spinning until stopped - publishing transform
[static_transform_publisher-8] translation: (‘-0.315590’, ‘0.219510’, ‘0.300830’)
[static_transform_publisher-8] rotation: (‘0.032303’, ‘0.998730’, ‘-0.006797’, ‘-0.037990’)
[static_transform_publisher-8] from ‘world’ to ‘target1_frame’
[static_transform_publisher-9] [INFO] [1740554934.775602462] [static_transform_publisher_paExkr0mZBJmLQIZ]: Spinning until stopped - publishing transform
[static_transform_publisher-9] translation: (‘-0.342700’, ‘-0.233490’, ‘0.333020’)
[static_transform_publisher-9] rotation: (‘0.661120’, ‘0.749260’, ‘-0.029517’, ‘-0.025912’)
[static_transform_publisher-9] from ‘elfin_base’ to ‘target2_frame’
[static_transform_publisher-7] [INFO] [1740554934.795667531] [static_transform_publisher_mpxIUMuCgpE8UAA5]: Spinning until stopped - publishing transform
[static_transform_publisher-7] translation: (‘-0.392076’, ‘1.236060’, ‘0.824618’)
[static_transform_publisher-7] rotation: (‘0.060985’, ‘0.122796’, ‘-0.651488’, ‘0.746167’)
[static_transform_publisher-7] from ‘elfin_base’ to ‘camera_1_link’
[isaac_ros_moveit_goal_setter-5] [ERROR] [1740554935.277772605] [rcl]: Failed to parse global arguments
[isaac_ros_moveit_goal_setter-5] terminate called after throwing an instance of ‘rclcpp::exceptions::RCLInvalidROSArgsError’
[isaac_ros_moveit_goal_setter-5] what(): failed to initialize rcl: Couldn’t parse params file: ‘–params-file /workspaces/isaac_ros-dev/install/elfin3_ros2_moveit2/share/elfin3_ros2_moveit2/config/kinematics.yaml’. Error: Cannot have a value before ros__parameters at line 2, at ./src/parse.c:793, at ./src/rcl/arguments.c:406
[component_container_mt-1] ERROR: flag ‘logtostderr’ was defined more than once (in files ‘./src/logging.cc’ and ‘/workspaces/isaac_ros-dev/src/glog-0.6.0/src/logging.cc’).
[rviz2-10] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to ‘/tmp/runtime-admin’
[ERROR] [component_container_mt-1]: process has died [pid 1218547, exit code 1, cmd ‘/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args --log-level error --ros-args -r __node:=manipulation_container’].
[ERROR] [isaac_ros_moveit_goal_setter-5]: process has died [pid 1218555, exit code -6, cmd ‘/workspaces/isaac_ros-dev/install/isaac_ros_moveit_goal_setter/lib/isaac_ros_moveit_goal_setter/isaac_ros_moveit_goal_setter --ros-args -r __node:=isaac_ros_moveit_goal_setter --params-file /tmp/launch_params_8ss2xme1 --params-file /tmp/launch_params_yn2navlh --params-file /workspaces/isaac_ros-dev/install/elfin3_ros2_moveit2/share/elfin3_ros2_moveit2/config/kinematics.yaml’].
[realsense2_camera_node-2] [INFO] [1740554938.201530300] [camera_1.camera_1]: RealSense ROS v4.51.1
[realsense2_camera_node-2] [INFO] [1740554938.201863298] [camera_1.camera_1]: Built with LibRealSense v2.55.1
[realsense2_camera_node-2] [INFO] [1740554938.201911779] [camera_1.camera_1]: Running with LibRealSense v2.55.1
[realsense2_camera_node-2] 26/02 15:28:58,363 WARNING [281473139402976] (handle-libusb.h:110) failed to claim usb interface, interface 0, is busy - retrying…
[realsense2_camera_node-2] 26/02 15:28:58,373 WARNING [281473139402976] (handle-libusb.h:110) failed to claim usb interface, interface 0, is busy - retrying…
[realsense2_camera_node-2] 26/02 15:28:58,383 ERROR [281473139402976] (handle-libusb.h:125) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
[realsense2_camera_node-2] 26/02 15:28:58,389 ERROR [281473164765408] (uvc-sensor.cpp:412) acquire_power failed: failed to set power state
[realsense2_camera_node-2] 26/02 15:28:58,402 ERROR [281473164765408] (rs.cpp:237) [rs2_create_device( info_list:0xffff88013400, index:0 ) UNKNOWN] failed to set power state
[realsense2_camera_node-2] 26/02 15:28:58,413 ERROR [281473164765408] (rs.cpp:237) [rs2_delete_device( device:nullptr ) UNKNOWN] null pointer passed for argument “device”
[realsense2_camera_node-2] 26/02 15:28:58,413 WARNING [281473164765408] (rs.cpp:373) null pointer passed for argument “device”
[realsense2_camera_node-2] [WARN] [1740554938.413228228] [camera_1.camera_1]: Device 1/1 failed with exception: failed to set power state
[realsense2_camera_node-2] [ERROR] [1740554938.413421895] [camera_1.camera_1]: The requested device with is NOT found. Will Try again.
Here is the latest log. Now it cannot perform path planning anymore, and the depth with segmented robot node still cannot see the content.
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2025-02-28-15-50-19-732188-bjut-desktop-1369998
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-2]: process started with pid [1370014]
[INFO] [component_container_mt-1]: process started with pid [1370012]
[INFO] [cumotion_planner_node-3]: process started with pid [1370016]
[INFO] [robot_segmenter_node-4]: process started with pid [1370018]
[INFO] [isaac_ros_moveit_goal_setter-5]: process started with pid [1370020]
[INFO] [pose_to_pose.py-6]: process started with pid [1370022]
[INFO] [static_transform_publisher-7]: process started with pid [1370028]
[INFO] [static_transform_publisher-8]: process started with pid [1370034]
[INFO] [static_transform_publisher-9]: process started with pid [1370036]
[INFO] [rviz2-10]: process started with pid [1370045]
[static_transform_publisher-7] [INFO] [1740729021.211719366] [static_transform_publisher_lmwst8kKtZ9nRCYA]: Spinning until stopped - publishing transform
[static_transform_publisher-7] translation: (‘-0.392076’, ‘1.236060’, ‘0.824618’)
[static_transform_publisher-7] rotation: (‘0.060985’, ‘0.122796’, ‘-0.651488’, ‘0.746167’)
[static_transform_publisher-7] from ‘elfin_base’ to ‘camera_1_link’
[static_transform_publisher-8] [INFO] [1740729021.654773966] [static_transform_publisher_ABX24yXwBM1nlk7v]: Spinning until stopped - publishing transform
[static_transform_publisher-8] translation: (‘-0.315590’, ‘0.219510’, ‘0.300830’)
[static_transform_publisher-8] rotation: (‘0.032303’, ‘0.998730’, ‘-0.006797’, ‘-0.037990’)
[static_transform_publisher-8] from ‘elfin_base’ to ‘target1_frame’
[isaac_ros_moveit_goal_setter-5] [ERROR] [1740729021.977658259] [rcl]: Failed to parse global arguments
[isaac_ros_moveit_goal_setter-5] terminate called after throwing an instance of ‘rclcpp::exceptions::RCLInvalidROSArgsError’
[isaac_ros_moveit_goal_setter-5] what(): failed to initialize rcl: Couldn’t parse params file: ‘–params-file /workspaces/isaac_ros-dev/install/elfin3_ros2_moveit2/share/elfin3_ros2_moveit2/config/kinematics.yaml’. Error: Cannot have a value before ros__parameters at line 2, at ./src/parse.c:793, at ./src/rcl/arguments.c:406
[static_transform_publisher-9] [INFO] [1740729022.139095862] [static_transform_publisher_bL1mi92m2WqiLOMC]: Spinning until stopped - publishing transform
[static_transform_publisher-9] translation: (‘-0.342700’, ‘-0.233490’, ‘0.333020’)
[static_transform_publisher-9] rotation: (‘0.661120’, ‘0.749260’, ‘-0.029517’, ‘-0.025912’)
[static_transform_publisher-9] from ‘elfin_base’ to ‘target2_frame’
[rviz2-10] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to ‘/tmp/runtime-admin’
[component_container_mt-1] ERROR: flag ‘logtostderr’ was defined more than once (in files ‘./src/logging.cc’ and ‘/workspaces/isaac_ros-dev/src/glog-0.6.0/src/logging.cc’).
[ERROR] [component_container_mt-1]: process has died [pid 1370012, exit code 1, cmd ‘/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args --log-level error --ros-args -r __node:=manipulation_container’].
[ERROR] [isaac_ros_moveit_goal_setter-5]: process has died [pid 1370020, exit code -6, cmd ‘/workspaces/isaac_ros-dev/install/isaac_ros_moveit_goal_setter/lib/isaac_ros_moveit_goal_setter/isaac_ros_moveit_goal_setter --ros-args -r __node:=isaac_ros_moveit_goal_setter --params-file /tmp/launch_params_poiqm20o --params-file /tmp/launch_params_o190j0_e --params-file /workspaces/isaac_ros-dev/install/elfin3_ros2_moveit2/share/elfin3_ros2_moveit2/config/kinematics.yaml’].
[realsense2_camera_node-2] [INFO] [1740729024.914566158] [camera_1.camera_1]: RealSense ROS v4.51.1
[realsense2_camera_node-2] [INFO] [1740729024.915208767] [camera_1.camera_1]: Built with LibRealSense v2.55.1
[realsense2_camera_node-2] [INFO] [1740729024.915299362] [camera_1.camera_1]: Running with LibRealSense v2.55.1
[realsense2_camera_node-2] [INFO] [1740729026.105836329] [camera_1.camera_1]: Device with serial number 109622072621 was found.
[realsense2_camera_node-2]
[realsense2_camera_node-2] [INFO] [1740729026.106051055] [camera_1.camera_1]: Device with physical ID 2-1.3-10 was found.
[realsense2_camera_node-2] [INFO] [1740729026.106103344] [camera_1.camera_1]: Device with name Intel RealSense D435I was found.
[realsense2_camera_node-2] [INFO] [1740729026.106809603] [camera_1.camera_1]: Device with port number 2-1.3 was found.
[realsense2_camera_node-2] [INFO] [1740729026.106887397] [camera_1.camera_1]: Device USB type: 3.2
[realsense2_camera_node-2] [INFO] [1740729026.154191012] [camera_1.camera_1]: getParameters…
[realsense2_camera_node-2] [INFO] [1740729026.244310349] [camera_1.camera_1]: JSON file is not provided
[realsense2_camera_node-2] [INFO] [1740729026.245770324] [camera_1.camera_1]: Device Name: Intel RealSense D435I
[realsense2_camera_node-2] [INFO] [1740729026.247048277] [camera_1.camera_1]: Device Serial No: 109622072621
[realsense2_camera_node-2] [INFO] [1740729026.249771229] [camera_1.camera_1]: Device physical port: 2-1.3-10
[realsense2_camera_node-2] [INFO] [1740729026.251928982] [camera_1.camera_1]: Device FW version: 5.12.7.150
[realsense2_camera_node-2] [INFO] [1740729026.254329301] [camera_1.camera_1]: Device Product ID: 0x0B3A
[realsense2_camera_node-2] [INFO] [1740729026.254401911] [camera_1.camera_1]: Sync Mode: Off
[pose_to_pose.py-6] [INFO] [1740729028.530057188] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[rviz2-10] [INFO] [1740729028.692114934] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-10] [INFO] [1740729028.694374322] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
[rviz2-10] [INFO] [1740729028.854644372] [rviz2]: Stereo is NOT SUPPORTED
[pose_to_pose.py-6] [INFO] [1740729029.535050346] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729030.540178225] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729031.546212623] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[realsense2_camera_node-2] [WARN] [1740729031.576416748] [camera_1.camera_1]: re-enable the stream for the change to take effect.
[pose_to_pose.py-6] [INFO] [1740729032.550997579] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[realsense2_camera_node-2] [INFO] [1740729032.898108331] [camera_1.camera_1]: Set ROS param rgb_camera.profile to default: 1280x720x30
[realsense2_camera_node-2] [INFO] [1740729032.957771632] [camera_1.camera_1]: Set ROS param gyro_fps to default: 200
[realsense2_camera_node-2] [INFO] [1740729032.965228500] [camera_1.camera_1]: Set ROS param accel_fps to default: 63
[realsense2_camera_node-2] [INFO] [1740729033.453788357] [camera_1.camera_1]: Stopping Sensor: Stereo Module
[pose_to_pose.py-6] [INFO] [1740729033.559093117] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[realsense2_camera_node-2] [INFO] [1740729033.820486536] [camera_1.camera_1]: Starting Sensor: Stereo Module
[realsense2_camera_node-2] [INFO] [1740729034.039647834] [camera_1.camera_1]: Open profile: stream_type: Depth(0), Format: Z16, Width: 1280, Height: 720, FPS: 15
[realsense2_camera_node-2] [INFO] [1740729034.040121735] [camera_1.camera_1]: Stopping Sensor: RGB Camera
[realsense2_camera_node-2] 28/02 15:50:34,041 WARNING [281472862513376] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-2] 28/02 15:50:34,052 WARNING [281472862513376] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-2] [WARN] [1740729034.173149466] [camera_1.camera_1]:
[realsense2_camera_node-2] [INFO] [1740729034.201231870] [camera_1.camera_1]: Starting Sensor: RGB Camera
[realsense2_camera_node-2] [INFO] [1740729034.309465443] [camera_1.camera_1]: Open profile: stream_type: Color(0), Format: RGB8, Width: 1280, Height: 720, FPS: 30
[realsense2_camera_node-2] [INFO] [1740729034.328285779] [camera_1.camera_1]: RealSense Node Is Up!
[rviz2-10] [INFO] [1740729034.350121523] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-10] [INFO] [1740729034.528851003] [rviz2]: Stereo is NOT SUPPORTED
[pose_to_pose.py-6] [INFO] [1740729034.564658059] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729035.569388897] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729036.574582787] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729037.581886970] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729038.593572036] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729039.599529974] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729040.605375107] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729041.618286921] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729042.623257917] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729043.637924430] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729044.642437683] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729045.653741706] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[realsense2_camera_node-2] 28/02 15:50:45,952 ERROR [281471889696992] (uvc-streamer.cpp:105) uvc streamer watchdog triggered on endpoint: 132
[pose_to_pose.py-6] [INFO] [1740729046.681692246] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729047.689267719] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729048.696607153] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[pose_to_pose.py-6] [INFO] [1740729049.703103172] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[robot_segmenter_node-4] [INFO] [1740729050.547235041] [robot_segmenter]: Node initialized with 1 cameras
[cumotion_planner_node-3] [INFO] [1740729050.661325470] [cumotion_planner]: Service(/nvblox_node/get_esdf_and_gradient) not available, waiting again…
[pose_to_pose.py-6] [INFO] [1740729050.708629820] [pose_to_pose_node]: Service set_target_pose not available! Waiting…
[robot_segmenter_node-4] [INFO] [1740729050.730105778] [robot_segmenter]: Reading TF from cameras
[cumotion_planner_node-3] [INFO] [1740729051.671245256] [cumotion_planner]: Service(/nvblox_node/get_esdf_and_gradient) not available, waiting again…
Thank you, I forward to the engineers.