Super slow inference using isaac_ros_foundationpose following its tutorial

[mona.jalal]

I am getting super slow inference and also wrong ones in isaac_ros_foundationpose following the example.

Here is the log:

isaac_ros_foundationpose_test.txt (136.8 KB)

here is rviz2 log

admin@DOS:/workspaces/isaac_ros-dev/isaac_ros_assets/models/synthetica_detr$ rviz2 -d  $(ros2 pkg prefix isaac_ros_foundationpose --share)/rviz/foundationpose_realsense.rviz
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-admin'
[INFO] [1739389371.321226750] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1739389371.321312486] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
[INFO] [1739389371.348097414] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1739389371.491140505] [rviz2]: Stereo is NOT SUPPORTED
Segmentation fault (core dumped)

I am in this part of training (finished synthetica DETR part and attached screencast of it).
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html

here is the screencast of the inference visualization:

sys info:

2025-02-12 14:40:35 [mona@DOS ~]$ uname -a
Linux DOS 6.8.0-52-generic #53~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jan 15 19:18:46 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
2025-02-12 14:40:37 [mona@DOS ~]$ lsb_release -a
LSB Version:	core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.5 LTS
Release:	22.04
Codename:	jammy
2025-02-12 14:40:43 [mona@DOS ~]$ nvidia-smi
Wed Feb 12 14:40:51 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.15              Driver Version: 570.86.15      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080 ...    Off |   00000000:01:00.0  On |                  N/A |
| N/A   77C    P0             46W /   90W |   12676MiB /  16384MiB |     98%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            5988      G   /usr/lib/xorg/Xorg                      133MiB |
|    0   N/A  N/A           25117      C   ...onents/component_container_mt      12514MiB |
+-----------------------------------------------------------------------------------------+


2025-02-12 14:49:06 [mona@DOS ~]$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

I also notice that fps for depth is 15 while intel realsense depth fps is 60.

[component_container_mt-1] [INFO] [1739389332.951787581] [realsense2_camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 640, Height: 480, FPS: 15

[Ahung]

Hi @mona.jalal ,

This tutorial detects the 3D pose of the Mac and Cheese Box mentioned in the SyntheticaDETR Object Classes section. In order to detect other objects from SyntheticaDETR Object Classes , you need to create a mesh and replace the mesh_file_path and texture_path arguments to this tutorials launch file with the mesh you generated. The tutorial on how to create a mesh using an iPhone can be found here.

In order to detect other objects not supported by SyntheticaDETR , you will have to modify the launch file and replace the 2D object detection pipeline with one that supports the object you are trying to detect.

Best,
Ahung

[mona.jalal]

Thanks for your response. Is there a benchmark for speed on AGX ORIN 64G for isaac_ros_foundationpose for objects in Synthetica_DERT?