Missing onnx model files for foundationpose, only etlt

kcz · June 18, 2024, 3:37am

After following the instructions in the website, it reports error when running the API launch command for foundationpose (replaced the path with correct path):

Command

ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py refine_model_file_path:=<path to refine onnx model> refine_engine_file_path:=<path to refine model .plan> score_model_file_path:=<path to score onnx model> score_engine_file_path:=<path to score model .plan> mesh_file_path:=<path to object mesh file> texture_path:=<path to texture map> launch_rviz:=<enable rviz> launch_bbox_to_mask:=<enable bbox to mask converter> mask_height:=<converted mask height> mask_width:=<converted mask width>

Error

[component_container_mt-1] 2024-06-17 20:32:37.319 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@151: TRT ERROR: ModelImporter.cpp:733: Failed to parse ONNX model from file: /tmp/score_model.onnx
[component_container_mt-1] 2024-06-17 20:32:37.319 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@472: Failed to parse ONNX file /tmp/score_model.onnx
[component_container_mt-1] 2024-06-17 20:32:37.442 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@287: Failed to create engine plan for model /tmp/score_model.onnx.

The downloaded model files have no onnx files. May I know how we can get it? Thanks

Raffaello · June 18, 2024, 1:32pm

Hi @kcz

I think you have successfully run the Issac ROS foundation point example, and now you are trying to make your own launch file.

If not, please check point 4 in isaac_ros_foundationpose — isaac_ros_docs documentation

This command will start the foundationpose demo.

If you are working with this command

ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py refine_model_file_path:=<path to refine onnx model> refine_engine_file_path:=<path to refine model .plan> score_model_file_path:=<path to score onnx model> score_engine_file_path:=<path to score model .plan> mesh_file_path:=<path to object mesh file> texture_path:=<path to texture map> launch_rviz:=<enable rviz> launch_bbox_to_mask:=<enable bbox to mask converter> mask_height:=<converted mask height> mask_width:=<converted mask width>

like is written, you need to change the <path to refine onnx model>, <path to refine model .plan>, <path to score onnx model>, <path to score model .plan>, <path to object mesh file>, <path to texture map>, <enable rviz>, <enable bbox to mask converter>, <converted mask height>, <converted mask width> with the right path.

Here is an example you can follow

github.com

NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation/blob/main/isaac_ros_foundationpose/launch/isaac_ros_foundationpose.launch.py#L106-L133


      
          foundationpose_node = ComposableNode(
              name='foundationpose',
              package='isaac_ros_foundationpose',
              plugin='nvidia::isaac_ros::foundationpose::FoundationPoseNode',
              parameters=[{
                  'mesh_file_path': mesh_file_path,
                  'texture_path': texture_path,
          
                  'refine_model_file_path': refine_model_file_path,
                  'refine_engine_file_path': refine_engine_file_path,
                  'refine_input_tensor_names': ['input_tensor1', 'input_tensor2'],
                  'refine_input_binding_names': ['input1', 'input2'],
                  'refine_output_tensor_names': ['output_tensor1', 'output_tensor2'],
                  'refine_output_binding_names': ['output1', 'output2'],
          
                  'score_model_file_path': score_model_file_path,
                  'score_engine_file_path': score_engine_file_path,
                  'score_input_tensor_names': ['input_tensor1', 'input_tensor2'],
                  'score_input_binding_names': ['input1', 'input2'],
                  'score_output_tensor_names': ['output_tensor'],

This file has been truncated. show original

kcz · June 18, 2024, 3:56pm

Thanks for the quick response and the instructions. Actually, I replaced the placeholder path with the correct ones (but not putting them in this thread). The problem is that I don’t have the onnx model files after following the instructions on the website. It only has the etlt and trt engine.

Download the pre-trained FoundationPose models from NGC:

wget --content-disposition https://api.ngc.nvidia.com/v2/models/nvidia/isaac/foundationpose/versions/1.0.0/zip -O foundationpose_1.0.0.zip

Copy the models to the required directory:

mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose && \ unzip foundationpose_1.0.0.zip -d ${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose

Can you be advise how to get the onnx files? Thanks

kcz · June 18, 2024, 8:34pm

Hi @Raffaello, could you please take a look at the problem again? Thanks

Raffaello · June 19, 2024, 2:16pm

Hi @kcz

I tested again running the command on my desktop:

wget --content-disposition https://api.ngc.nvidia.com/v2/models/nvidia/isaac/foundationpose/versions/1.0.0/zip -O foundationpose_1.0.0.zip

and

mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose && \
   unzip foundationpose_1.0.0.zip -d ${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose

and everything looks normal.

Do you have some errors when you download the foundationpose_1.0.0.zip file? or when you unpack the zip file?

kcz · June 19, 2024, 2:38pm

@Raffaello Thanks for checking but it seems that you missed the point of the issue.

After downloading the model files, there are NO onnx model files, which is REQUIRED when running the launch file of foundationpose. There are only etlt model files and trt engine files.

Can you please take a look? Thanks

Raffaello · June 21, 2024, 11:13am

Hi @kcz

I forwarded your question to the engineers.

If you follow the tutorial to finish the model preparation, you should have the trt engines under isaac_ros_assets and you can pass the trt engine files to the launch file and ignore the onnx file.

Let me know if the demo starts again.

Best,
Raffaello

grdec · June 24, 2024, 2:29pm

Hello,

For me it also happened. I have just tried to follow the tutorial and after unzipping there is no files such ‘model.onnx’ and I also receive errors:

[component_container_mt-1] Could not open file model.onnx
[component_container_mt-1] Could not open file model.onnx
[component_container_mt-1] 2024-06-24 16:08:58.393 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@151: TRT ERROR: ModelImporter.cpp:733: Failed to parse ONNX model from file: model.onnx
[component_container_mt-1] 2024-06-24 16:08:58.393 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@472: Failed to parse ONNX file model.onnx
[component_container_mt-1] 2024-06-24 16:08:58.502 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@287: Failed to create engine plan for model model.onnx.

Was that issue solved locally by anyone?

Raffaello · June 27, 2024, 5:37pm

This demo does not require the ONNX file. If you follow the quickstart, the files are converted to .etlt files and loaded on the foundationpose launch file.

https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#build-package-name

We are working to update the documentation to make more simple to usage the provided launch file.

Best,
Raffaello

sarthakgarg0303 · August 23, 2024, 6:28pm

@grdec Have you solved the issue, I’m encountering the same issue!

[component_container_mt-1] 2024-08-23 23:51:56.340 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@151: TRT ERROR: ModelImporter.cpp:730: Failed to parse ONNX model from file: /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 23:51:56.340 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@472: Failed to parse ONNX file /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 23:51:56.634 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@287: Failed to create engine plan for model /tmp/refine_model.onnx.
[component_container_mt-1] 2024-08-23 23:51:56.634 WARN  gxf/std/entity_executor.cpp@495: Failed to start entity [OXJWUVBWKM_refine_inference]
[component_container_mt-1] 2024-08-23 23:51:56.634 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E449 named 'OXJWUVBWKM_refine_inference': GXF_FAILURE
[component_container_mt-1] 2024-08-23 23:51:56.640 ERROR gxf/std/entity_executor.cpp@586: Entity [OXJWUVBWKM_refine_inference] must be in Started, Tick Pending, Ticking or Idle stage before stopping. Current state is StartPending
[component_container_mt-1] 2024-08-23 23:51:57.373 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 330 not found!
[component_container_mt-1] [WARN] [1724437317.373317852] [foundationpose_node]: [NitrosNode] The heartbeat entity (eid=330) was stopped. The graph may have been terminated.
[component_container_mt-1] [INFO] [1724437317.404714505] [foundationpose_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724437317.404749612] [foundationpose_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-08-23 23:51:57.404 ERROR gxf/core/runtime.cpp@1468: Graph interrupt failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1724437317.404849844] [foundationpose_node]: [NitrosContext] GxfGraphInterrupt Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1724437317.404853876] [foundationpose_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] 2024-08-23 23:51:57.404 ERROR gxf/std/program.cpp@574: wait failed. Deactivating...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] 2024-08-23 23:51:57.458 WARN  gxf/std/entity_warden.cpp@464: Component of type nvidia::gxf::MultiThreadScheduler, cid 574 failed to deinitialize with code GXF_FAILURE
[component_container_mt-1] 2024-08-23 23:51:57.458 ERROR gxf/core/runtime.cpp@751: Could not deinitialize entity 'OXJWUVBWKM_utils' (E572): GXF_FAILURE
[component_container_mt-1] 2024-08-23 23:51:57.458 ERROR gxf/std/program.cpp@576: Deactivation failed.
[component_container_mt-1] 2024-08-23 23:51:57.458 ERROR gxf/core/runtime.cpp@1476: Graph wait failed with error: GXF_FAILURE

Topic		Replies	Views
Can't complete isaac_ros_foundationpose quickstart Isaac ROS	5	543	July 1, 2024
Use custom RT-DETR model for FoundationPose Isaac ROS tensorrt , cuda	3	418	August 28, 2024
Foundation Pose for custom segmentation Isaac ROS	4	231	November 24, 2024
Launch isaac-ros-foundationpose node failed Isaac ROS isaac-ros-pose-estimation	4	93	December 15, 2024
Foundationpose package using orbbec camera Isaac ROS isaac-ros-pose-estimation	0	64	November 23, 2024
Unsupported ONNX data type: UINT8 (2) TensorRT	24	8967	May 6, 2021
An Three error occurred during Foundationpose Quickstart Isaac ROS	18	439	September 1, 2024
How to use onnx file with deepstream-test1-usbcam + Custom models DeepStream SDK	30	4652	October 12, 2021
Fake quantization ONNX model parse ERROR using TensorRT TensorRT	6	1779	September 26, 2021
ERORR with ONNX2TRT : Unknown embedded device detected Jetson Xavier NX onnx	18	4574	April 27, 2022

Missing onnx model files for foundationpose, only etlt

Related topics