Isaac_ros_foundationpose Quickstart with RealSense Camera D435i

Robotics & Edge Computing Robotics - Isaac Isaac ROS
1

I’m trying to follow the Isaac_ros_foundationpose Quickstart with RealSense Camera D435i+Jetson AGX Orin Devkit +NVMe SSD installed on JetPack6 GA.
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#quickstart

I have already completed both of Isaac_ros_foundationpose Quickstart and isaac_ros_rtdetr Quickstart with Rosbag.
In addition, I have already checked the operation of isaac_ros_rtdetr Quickstart with RealSense Camera D435i like below:

However, when I follow the Isaac_ros_foundationpose Quickstart with RealSense Camera D435i to run the launch file like the attached file, the command’s work stops.
error_foundationpose_RealsenseCamera_launch_script.txt (108.5 KB)

So, could you please solve out this issue?

thanks

1 Like
2

Hi @A_Y

Thank you for your post, looking your log I noticed some errors:

[component_container_mt-1]  21/08 14:07:30,963 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1]  21/08 14:07:30,974 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1]  21/08 14:07:30,985 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1]  21/08 14:07:30,996 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1]  21/08 14:07:31,007 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[component_container_mt-1]  21/08 14:07:31,018 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11

This issue is usually related using a wrong USB cable on your realsense.
I suggest to check if your USB cable works for your camera, you can also use RealSenseViewer to verify that your camera works.

[component_container_mt-1] 2024-08-21 14:07:35.174 ERROR ./gxf/foundationpose/foundationpose_sampling.cpp@410: [FoundationposeSampling] Input depth image and segmentation image have different dimension
[component_container_mt-1] 2024-08-21 14:07:35.179 ERROR gxf/std/entity_executor.cpp@552: Failed to tick codelet  in entity: SEJSPLYNZD_sampling code: GXF_FAILURE
[component_container_mt-1] 2024-08-21 14:07:35.179 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E1826 named 'SEJSPLYNZD_sampling': GXF_FAILURE
[component_container_mt-1] 2024-08-21 14:07:35.216 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 1784 not found!

The error Input depth image and segmentation image have different dimension looks related to a wrong camera input. I’ll check if this error is coming from the demo.

I keep you posted.

Raffaello

1 Like
3

Hi @Raffaello

Thank you for your comments.

This issue is usually related using a wrong USB cable on your realsense.
I suggest to check if your USB cable works for your camera, you can also use RealSenseViewer to verify that your camera works.

I have already checked with other USB cables but, this issue still remains. In addition, my camera works with RealSenseViewer

Screenshot from 2024-08-19 14-01-03
Screenshot from 2024-08-19 14-01-031337×792 201 KB

[component_container_mt-1] 21/08 14:07:31,018 WARNING [281469885016288] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11

Do you have any ideas to solve out this error?

[component_container_mt-1] 2024-08-21 14:07:35.174 ERROR ./gxf/foundationpose/foundationpose_sampling.cpp@410: [FoundationposeSampling] Input depth image and segmentation image have different dimension
[component_container_mt-1] 2024-08-21 14:07:35.179 ERROR gxf/std/entity_executor.cpp@552: Failed to tick codelet  in entity: SEJSPLYNZD_sampling code: GXF_FAILURE
[component_container_mt-1] 2024-08-21 14:07:35.179 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E1826 named 'SEJSPLYNZD_sampling': GXF_FAILURE
[component_container_mt-1] 2024-08-21 14:07:35.216 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 1784 not found!

Please help to check it too. I’m looking forward to your update.

thanks

4

I found another topic related to the same issue, and we are investigating the cause of this bug. Meanwhile, could you please share the log by adding the command below to your demo?

--- a/isaac_ros_realsense/launch/isaac_ros_realsense_mono_rect_depth_core.launch.py
+++ b/isaac_ros_realsense/launch/isaac_ros_realsense_mono_rect_depth_core.launch.py
@@ -84,7 +84,8 @@ def generate_launch_description():
         composable_node_descriptions=(
             IsaacROSRealSenseMonoRectDepthLaunchFragment.get_composable_nodes()
         ),
-        output='screen'
+        output='screen',
+        arguments=['--ros-args', '--log-level', 'debug']
     )

Thank in in advance
Raffaello

5

@Raffaello ,

I followed this quickstart with not “Build from Source” but “Binary Package” so, I need to do with “Build from Source”. Is my understanding correct?

In addition, when I followed the quickstart with “Build from Source”, I faced another issue like below:

--- stderr: isaac_ros_dope                                                                                                                                                                            
/workspaces/isaac_ros-dev/src/isaac_ros_pose_estimation/isaac_ros_dope/src/dope_decoder_node.cpp:28:10: fatal error: isaac_ros_nitros_pose_array_type/nitros_pose_array.hpp: No such file or directory
   28 | #include "isaac_ros_nitros_pose_array_type/nitros_pose_array.hpp"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
gmake[2]: *** [CMakeFiles/dope_decoder_node.dir/build.make:76: CMakeFiles/dope_decoder_node.dir/src/dope_decoder_node.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:140: CMakeFiles/dope_decoder_node.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
---
Failed   <<< isaac_ros_dope [7.96s, exited with code 2]
Aborted  <<< gxf_isaac_centerpose [50.4s]                                                                                                                                       
Aborted  <<< realsense2_camera [1min 16s]                                                                                                         
Aborted  <<< gxf_isaac_foundationpose [1min 54s]                                           

Summary: 13 packages finished [2min 12s]
  1 package failed: isaac_ros_dope
  3 packages aborted: gxf_isaac_centerpose gxf_isaac_foundationpose realsense2_camera
  8 packages had stderr output: isaac_ros_apriltag_interfaces isaac_ros_bi3d_interfaces isaac_ros_dope isaac_ros_nitros_bridge_interfaces isaac_ros_nova_interfaces isaac_ros_pointcloud_interfaces isaac_ros_tensor_list_interfaces realsense2_camera_msgs
  2 packages not processed

Could you please advise me to avoid this another issue?

thanks

6

I have the same problem, there a multiply errors in the demo launch files. I found I needed to change the resolution of the realsense config file: realsense_mono_depth.yaml → depth_module.depth_profile: 640x480x15 Then you will have to modify the launch for the realsense as the topics are wrongly map due to name space. Still haven’t gotten it to run 100% with any real cameras.

7

Hi, I had the same problem and solved by manually setting the depth and rgb resolution as follows:

Solution: In realsense_mono_depth.yaml file, add the following configurations:
depth_model.profile: 640x480x15
rgb_camera.profile: 640x480x15

The resolution profiling in Realsense camera ROS driver version 4.51.1 seems to be defined a little differently from newer version.

ref: About how to set the resolution of /aligned_depth_to_color/image_raw in custom launch.py. · Issue #3194 · IntelRealSense/realsense-ros · GitHub

1 Like