Use custom RT-DETR model for FoundationPose

Robotics & Edge Computing Robotics - Isaac Isaac ROS
,
1

Hi,
System Specifications:

  • Operating System: Jetson AGX ORIN JetPack 6.0

I trained a rt-detr model follow the RT-DETR (Realtime Detection Transformer) - Ultralytics YOLO Docs, and covert it using yolo export model=./weights/rt-detr-l.pt format=onnx
then

 ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
 launch_fragments:=foundationpose \
 interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/quickstart_interface_specs.json \
 mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/tx/my_tx.obj \
 texture_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/tx/my_tx.jpg \
 score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan \
 rt_detr_model_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rt-detr-l.onnx \
 refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rt-detr-l.plan

error

[component_container_mt-1] [INFO] [1724421025.299676381] [tensor_rt]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1724421025.304128165] [tensor_rt]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421025.304218080] [tensor_rt]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-08-23 21:50:25.304 ERROR gxf/std/program.cpp@551: Attempted interrupting when not running (state=0).
[component_container_mt-1] 2024-08-23 21:50:25.304 ERROR gxf/core/runtime.cpp@1468: Graph interrupt failed with error: GXF_INVALID_EXECUTION_SEQUENCE
[component_container_mt-1] [ERROR] [1724421025.304299292] [tensor_rt]: [NitrosContext] GxfGraphInterrupt Error: GXF_INVALID_EXECUTION_SEQUENCE
[component_container_mt-1] [INFO] [1724421025.304319643] [tensor_rt]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421025.304339610] [tensor_rt]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421025.304357401] [tensor_rt]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421025.304380536] [tensor_rt]: [NitrosNode] Application termination done
[component_container_mt-1] [ERROR] [1724421025.317514297] [isaac_ros_examples.container]: Component constructor threw an exception: [TensorRTNode] Empty engine_file_path_, this needs to be set per the engine
[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'tensor_rt' of type 'nvidia::isaac_ros::dnn_inference::TensorRTNode' in container '/isaac_ros_examples/container': Component constructor threw an exception: [TensorRTNode] Empty engine_file_path_, this needs to be set per the engine
[component_container_mt-1] [INFO] [1724421025.324423496] [isaac_ros_examples.container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_rtdetr/lib/librtdetr_decoder_node.so
[component_container_mt-1] [INFO] [1724421025.329629259] [isaac_ros_examples.container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::rtdetr::RtDetrDecoderNode>
[component_container_mt-1] [INFO] [1724421025.329727942] [isaac_ros_examples.container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::rtdetr::RtDetrDecoderNode>
[component_container_mt-1] [INFO] [1724421025.351558399] [rtdetr_decoder.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/rtdetr_decoder' in container '/isaac_ros_examples/container'

[component_container_mt-1] 2024-08-23 21:50:35.073 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dummy_rx' in component ''.
[component_container_mt-1] 2024-08-23 21:50:35.074 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dummy_rx' in component ''.
[component_container_mt-1] 2024-08-23 21:50:35.076 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dev_id' in component 'stream'.
[component_container_mt-1] [INFO] [1724421035.079313486] [foundationpose_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.103 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] 2024-08-23 21:50:35.107 WARN  gxf/std/scheduling_terms.cpp@333: 'min_size' parameter in MultiMessageAvailableSchedulingTerm is deprecated. Use 'min_sum' with SumOfAll sampling mode instead
[component_container_mt-1] [INFO] [1724421035.128228896] [foundationpose_node]: [NitrosNode] Node was started
[component_container_mt-1] 2024-08-23 21:50:35.811 WARN  ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@281: Rebuilding CUDA engine /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.plan (forced by config). Note: this process may take up to several minutes.
[component_container_mt-1] Could not open file /tmp/refine_model.onnx
[component_container_mt-1] Could not open file /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 21:50:42.242 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@151: TRT ERROR: ModelImporter.cpp:730: Failed to parse ONNX model from file: /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 21:50:42.242 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@472: Failed to parse ONNX file /tmp/refine_model.onnx
[component_container_mt-1] 2024-08-23 21:50:42.621 ERROR ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@287: Failed to create engine plan for model /tmp/refine_model.onnx.
[component_container_mt-1] 2024-08-23 21:50:42.622 WARN  gxf/std/entity_executor.cpp@495: Failed to start entity [IFHCSCJKZE_refine_inference]
[component_container_mt-1] 2024-08-23 21:50:42.622 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E415 named 'IFHCSCJKZE_refine_inference': GXF_FAILURE
[component_container_mt-1] 2024-08-23 21:50:42.625 ERROR gxf/std/entity_executor.cpp@586: Entity [IFHCSCJKZE_refine_inference] must be in Started, Tick Pending, Ticking or Idle stage before stopping. Current state is StartPending
[component_container_mt-1] 2024-08-23 21:50:43.128 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 296 not found!
[component_container_mt-1] [WARN] [1724421043.128632355] [foundationpose_node]: [NitrosNode] The heartbeat entity (eid=296) was stopped. The graph may have been terminated.
[component_container_mt-1] [INFO] [1724421043.168373716] [foundationpose_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.168430993] [foundationpose_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] 2024-08-23 21:50:43.168 ERROR gxf/core/runtime.cpp@1468: Graph interrupt failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1724421043.168616776] [foundationpose_node]: [NitrosContext] GxfGraphInterrupt Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1724421043.168628744] [foundationpose_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] 2024-08-23 21:50:43.168 ERROR gxf/std/program.cpp@574: wait failed. Deactivating...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] 2024-08-23 21:50:43.211 WARN  gxf/std/entity_warden.cpp@464: Component of type nvidia::gxf::MultiThreadScheduler, cid 540 failed to deinitialize with code GXF_FAILURE
[component_container_mt-1] 2024-08-23 21:50:43.211 ERROR gxf/core/runtime.cpp@751: Could not deinitialize entity 'IFHCSCJKZE_utils' (E538): GXF_FAILURE
[component_container_mt-1] 2024-08-23 21:50:43.211 ERROR gxf/std/program.cpp@576: Deactivation failed.
[component_container_mt-1] 2024-08-23 21:50:43.211 ERROR gxf/core/runtime.cpp@1476: Graph wait failed with error: GXF_FAILURE
[component_container_mt-1] [ERROR] [1724421043.211704406] [foundationpose_node]: [NitrosContext] GxfGraphWait Error: GXF_FAILURE
[component_container_mt-1] [INFO] [1724421043.211727989] [foundationpose_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.255598172] [resize_left_viz]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.255647642] [resize_left_viz]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.256447219] [resize_left_viz]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.256476786] [resize_left_viz]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421043.258284538] [resize_left_viz]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.258637160] [resize_left_viz]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.285778335] [resize_mask_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.285822045] [resize_mask_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.287726880] [resize_mask_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.287767454] [resize_mask_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421043.290581173] [resize_mask_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.290948323] [resize_mask_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.351694357] [reshape_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.351736435] [reshape_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.352350165] [reshape_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.352375828] [reshape_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] [INFO] [1724421043.358963315] [reshape_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.359194056] [reshape_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.372938314] [interleaved_to_planar_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.372984616] [interleaved_to_planar_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.373586347] [interleaved_to_planar_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.373608138] [interleaved_to_planar_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] [INFO] [1724421043.380180298] [interleaved_to_planar_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.380385504] [interleaved_to_planar_node]: [NitrosNode] Application termination done
[component_container_mt-1] [INFO] [1724421043.418566748] [resize_left_rt_detr_node]: [NitrosNode] Terminating the running application
[component_container_mt-1] [INFO] [1724421043.418610330] [resize_left_rt_detr_node]: [NitrosContext] Interrupting GXF...
[component_container_mt-1] [INFO] [1724421043.422785167] [resize_left_rt_detr_node]: [NitrosContext] Waiting on GXF...
[component_container_mt-1] [INFO] [1724421043.422830509] [resize_left_rt_detr_node]: [NitrosContext] Deinitializing...
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] [INFO] [1724421043.425448493] [resize_left_rt_detr_node]: [NitrosContext] Destroying context
[component_container_mt-1] [INFO] [1724421043.425770494] [resize_left_rt_detr_node]: [NitrosNode] Application termination done
[component_container_mt-1] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-1] |==================================================================================================================================================================|
[component_container_mt-1] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-1] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-1] |==================================================================================================================================================================|
[INFO] [component_container_mt-1]: process has finished cleanly [pid 2072247]

I have already tried the method described in the NVIDIA Developer Forums post at https://forums.developer.nvidia.com/t/isaac-ros-foundationpose-for-pallets/296926/20 and have successfully trained a YOLOv8s model for object detection. I’ve also created my own .obj and .png files, and the mask image output looks good. However, I’m not receiving any pose outputs. I’m wondering if this could be related to the fact that my input image size is 640x320. Could you help me understand if the image size is causing this issue?

I am grateful for any advice or guidance you can provide. I am eager to resolve this issue and continue working with ISAAC FoundationPose.

Thank you in advance for your help.

Best regards,

2

Hi @user29288

Thank you for your post and your tests.

It seems that you are encountering an error due to an incorrect path for the engine. Have you verified that the files are located in the correct folder?

Let me know

Raffaello

3

hi,

I trained a rt-detr model follow the RT-DETR (Realtime Detection Transformer) - Ultralytics YOLO Docs, and covert it using yolo export model=./weights/rt-detr-l.pt format=onnx

and then covert it using
/usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan admin@ubuntu:/workspaces/isaac_ros-dev$ /usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan

[08/26/2024-10:25:19] [I] === Model Options ===
[08/26/2024-10:25:19] [I] Format: ONNX
[08/26/2024-10:25:19] [I] Model: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
[08/26/2024-10:25:19] [I] Output:
[08/26/2024-10:25:19] [I] === Build Options ===
[08/26/2024-10:25:19] [I] Max batch: explicit batch
[08/26/2024-10:25:19] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[08/26/2024-10:25:19] [I] minTiming: 1
[08/26/2024-10:25:19] [I] avgTiming: 8
[08/26/2024-10:25:19] [I] Precision: FP32
[08/26/2024-10:25:19] [I] LayerPrecisions: 
[08/26/2024-10:25:19] [I] Layer Device Types: 
[08/26/2024-10:25:19] [I] Calibration: 
[08/26/2024-10:25:19] [I] Refit: Disabled
[08/26/2024-10:25:19] [I] Version Compatible: Disabled
[08/26/2024-10:25:19] [I] ONNX Native InstanceNorm: Disabled
[08/26/2024-10:25:19] [I] TensorRT runtime: full
[08/26/2024-10:25:19] [I] Lean DLL Path: 
[08/26/2024-10:25:19] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
[08/26/2024-10:25:19] [I] Exclude Lean Runtime: Disabled
[08/26/2024-10:25:19] [I] Sparsity: Disabled
[08/26/2024-10:25:19] [I] Safe mode: Disabled
[08/26/2024-10:25:19] [I] Build DLA standalone loadable: Disabled
[08/26/2024-10:25:19] [I] Allow GPU fallback for DLA: Disabled
[08/26/2024-10:25:19] [I] DirectIO mode: Disabled
[08/26/2024-10:25:19] [I] Restricted mode: Disabled
[08/26/2024-10:25:19] [I] Skip inference: Disabled
[08/26/2024-10:25:19] [I] Save engine: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.plan
[08/26/2024-10:25:19] [I] Load engine: 
[08/26/2024-10:25:19] [I] Profiling verbosity: 0
[08/26/2024-10:25:19] [I] Tactic sources: Using default tactic sources
[08/26/2024-10:25:19] [I] timingCacheMode: local
[08/26/2024-10:25:19] [I] timingCacheFile: 
[08/26/2024-10:25:19] [I] Heuristic: Disabled
[08/26/2024-10:25:19] [I] Preview Features: Use default preview flags.
[08/26/2024-10:25:19] [I] MaxAuxStreams: -1
[08/26/2024-10:25:19] [I] BuilderOptimizationLevel: -1
[08/26/2024-10:25:19] [I] Input(s)s format: fp32:CHW
[08/26/2024-10:25:19] [I] Output(s)s format: fp32:CHW
[08/26/2024-10:25:19] [I] Input build shapes: model
[08/26/2024-10:25:19] [I] Input calibration shapes: model
[08/26/2024-10:25:19] [I] === System Options ===
[08/26/2024-10:25:19] [I] Device: 0
[08/26/2024-10:25:19] [I] DLACore: 
[08/26/2024-10:25:19] [I] Plugins:
[08/26/2024-10:25:19] [I] setPluginsToSerialize:
[08/26/2024-10:25:19] [I] dynamicPlugins:
[08/26/2024-10:25:19] [I] ignoreParsedPluginLibs: 0
[08/26/2024-10:25:19] [I] 
[08/26/2024-10:25:19] [I] === Inference Options ===
[08/26/2024-10:25:19] [I] Batch: Explicit
[08/26/2024-10:25:19] [I] Input inference shapes: model
[08/26/2024-10:25:19] [I] Iterations: 10
[08/26/2024-10:25:19] [I] Duration: 3s (+ 200ms warm up)
[08/26/2024-10:25:19] [I] Sleep time: 0ms
[08/26/2024-10:25:19] [I] Idle time: 0ms
[08/26/2024-10:25:19] [I] Inference Streams: 1
[08/26/2024-10:25:19] [I] ExposeDMA: Disabled
[08/26/2024-10:25:19] [I] Data transfers: Enabled
[08/26/2024-10:25:19] [I] Spin-wait: Disabled
[08/26/2024-10:25:19] [I] Multithreading: Disabled
[08/26/2024-10:25:19] [I] CUDA Graph: Disabled
[08/26/2024-10:25:19] [I] Separate profiling: Disabled
[08/26/2024-10:25:19] [I] Time Deserialize: Disabled
[08/26/2024-10:25:19] [I] Time Refit: Disabled
[08/26/2024-10:25:19] [I] NVTX verbosity: 0
[08/26/2024-10:25:19] [I] Persistent Cache Ratio: 0
[08/26/2024-10:25:19] [I] Inputs:
[08/26/2024-10:25:19] [I] === Reporting Options ===
[08/26/2024-10:25:19] [I] Verbose: Disabled
[08/26/2024-10:25:19] [I] Averages: 10 inferences
[08/26/2024-10:25:19] [I] Percentiles: 90,95,99
[08/26/2024-10:25:19] [I] Dump refittable layers:Disabled
[08/26/2024-10:25:19] [I] Dump output: Disabled
[08/26/2024-10:25:19] [I] Profile: Disabled
[08/26/2024-10:25:19] [I] Export timing to JSON file: 
[08/26/2024-10:25:19] [I] Export output to JSON file: 
[08/26/2024-10:25:19] [I] Export profile to JSON file: 
[08/26/2024-10:25:19] [I] 
[08/26/2024-10:25:20] [I] === Device Information ===
[08/26/2024-10:25:20] [I] Selected Device: Orin
[08/26/2024-10:25:20] [I] Compute Capability: 8.7
[08/26/2024-10:25:20] [I] SMs: 16
[08/26/2024-10:25:20] [I] Device Global Memory: 62841 MiB
[08/26/2024-10:25:20] [I] Shared Memory per SM: 164 KiB
[08/26/2024-10:25:20] [I] Memory Bus Width: 256 bits (ECC disabled)
[08/26/2024-10:25:20] [I] Application Compute Clock Rate: 1.3 GHz
[08/26/2024-10:25:20] [I] Application Memory Clock Rate: 0.816 GHz
[08/26/2024-10:25:20] [I] 
[08/26/2024-10:25:20] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[08/26/2024-10:25:20] [I] 
[08/26/2024-10:25:20] [I] TensorRT version: 8.6.2
[08/26/2024-10:25:20] [I] Loading standard plugins
[08/26/2024-10:25:20] [I] [TRT] [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 33, GPU 11932 (MiB)
[08/26/2024-10:25:26] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +1154, GPU +1115, now: CPU 1223, GPU 13067 (MiB)
[08/26/2024-10:25:26] [I] Start parsing network model.
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
[08/26/2024-10:25:26] [E] [TRT] ModelImporter.cpp:730: Failed to parse ONNX model from file: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx
[08/26/2024-10:25:26] [E] Failed to parse onnx file
[08/26/2024-10:25:26] [I] Finished parsing network model. Parse time: 0.00179726
[08/26/2024-10:25:26] [E] Parsing model failed
[08/26/2024-10:25:26] [E] Failed to create engine from model or file.
[08/26/2024-10:25:26] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/walker.plan
admin@ubuntu:/workspaces/isaac_ros-dev$ /usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan
&&&& RUNNING TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan
[08/26/2024-10:27:27] [I] === Model Options ===
[08/26/2024-10:27:27] [I] Format: ONNX
[08/26/2024-10:27:27] [I] Model: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
[08/26/2024-10:27:27] [I] Output:
[08/26/2024-10:27:27] [I] === Build Options ===
[08/26/2024-10:27:27] [I] Max batch: explicit batch
[08/26/2024-10:27:27] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[08/26/2024-10:27:27] [I] minTiming: 1
[08/26/2024-10:27:27] [I] avgTiming: 8
[08/26/2024-10:27:27] [I] Precision: FP32
[08/26/2024-10:27:27] [I] LayerPrecisions: 
[08/26/2024-10:27:27] [I] Layer Device Types: 
[08/26/2024-10:27:27] [I] Calibration: 
[08/26/2024-10:27:27] [I] Refit: Disabled
[08/26/2024-10:27:27] [I] Version Compatible: Disabled
[08/26/2024-10:27:27] [I] ONNX Native InstanceNorm: Disabled
[08/26/2024-10:27:27] [I] TensorRT runtime: full
[08/26/2024-10:27:27] [I] Lean DLL Path: 
[08/26/2024-10:27:27] [I] Tempfile Controls: { in_memory: allow, temporary: allow }
[08/26/2024-10:27:27] [I] Exclude Lean Runtime: Disabled
[08/26/2024-10:27:27] [I] Sparsity: Disabled
[08/26/2024-10:27:27] [I] Safe mode: Disabled
[08/26/2024-10:27:27] [I] Build DLA standalone loadable: Disabled
[08/26/2024-10:27:27] [I] Allow GPU fallback for DLA: Disabled
[08/26/2024-10:27:27] [I] DirectIO mode: Disabled
[08/26/2024-10:27:27] [I] Restricted mode: Disabled
[08/26/2024-10:27:27] [I] Skip inference: Disabled
[08/26/2024-10:27:27] [I] Save engine: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan
[08/26/2024-10:27:27] [I] Load engine: 
[08/26/2024-10:27:27] [I] Profiling verbosity: 0
[08/26/2024-10:27:27] [I] Tactic sources: Using default tactic sources
[08/26/2024-10:27:27] [I] timingCacheMode: local
[08/26/2024-10:27:27] [I] timingCacheFile: 
[08/26/2024-10:27:27] [I] Heuristic: Disabled
[08/26/2024-10:27:27] [I] Preview Features: Use default preview flags.
[08/26/2024-10:27:27] [I] MaxAuxStreams: -1
[08/26/2024-10:27:27] [I] BuilderOptimizationLevel: -1
[08/26/2024-10:27:27] [I] Input(s)s format: fp32:CHW
[08/26/2024-10:27:27] [I] Output(s)s format: fp32:CHW
[08/26/2024-10:27:27] [I] Input build shapes: model
[08/26/2024-10:27:27] [I] Input calibration shapes: model
[08/26/2024-10:27:27] [I] === System Options ===
[08/26/2024-10:27:27] [I] Device: 0
[08/26/2024-10:27:27] [I] DLACore: 
[08/26/2024-10:27:27] [I] Plugins:
[08/26/2024-10:27:27] [I] setPluginsToSerialize:
[08/26/2024-10:27:27] [I] dynamicPlugins:
[08/26/2024-10:27:27] [I] ignoreParsedPluginLibs: 0
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] === Inference Options ===
[08/26/2024-10:27:27] [I] Batch: Explicit
[08/26/2024-10:27:27] [I] Input inference shapes: model
[08/26/2024-10:27:27] [I] Iterations: 10
[08/26/2024-10:27:27] [I] Duration: 3s (+ 200ms warm up)
[08/26/2024-10:27:27] [I] Sleep time: 0ms
[08/26/2024-10:27:27] [I] Idle time: 0ms
[08/26/2024-10:27:27] [I] Inference Streams: 1
[08/26/2024-10:27:27] [I] ExposeDMA: Disabled
[08/26/2024-10:27:27] [I] Data transfers: Enabled
[08/26/2024-10:27:27] [I] Spin-wait: Disabled
[08/26/2024-10:27:27] [I] Multithreading: Disabled
[08/26/2024-10:27:27] [I] CUDA Graph: Disabled
[08/26/2024-10:27:27] [I] Separate profiling: Disabled
[08/26/2024-10:27:27] [I] Time Deserialize: Disabled
[08/26/2024-10:27:27] [I] Time Refit: Disabled
[08/26/2024-10:27:27] [I] NVTX verbosity: 0
[08/26/2024-10:27:27] [I] Persistent Cache Ratio: 0
[08/26/2024-10:27:27] [I] Inputs:
[08/26/2024-10:27:27] [I] === Reporting Options ===
[08/26/2024-10:27:27] [I] Verbose: Disabled
[08/26/2024-10:27:27] [I] Averages: 10 inferences
[08/26/2024-10:27:27] [I] Percentiles: 90,95,99
[08/26/2024-10:27:27] [I] Dump refittable layers:Disabled
[08/26/2024-10:27:27] [I] Dump output: Disabled
[08/26/2024-10:27:27] [I] Profile: Disabled
[08/26/2024-10:27:27] [I] Export timing to JSON file: 
[08/26/2024-10:27:27] [I] Export output to JSON file: 
[08/26/2024-10:27:27] [I] Export profile to JSON file: 
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] === Device Information ===
[08/26/2024-10:27:27] [I] Selected Device: Orin
[08/26/2024-10:27:27] [I] Compute Capability: 8.7
[08/26/2024-10:27:27] [I] SMs: 16
[08/26/2024-10:27:27] [I] Device Global Memory: 62841 MiB
[08/26/2024-10:27:27] [I] Shared Memory per SM: 164 KiB
[08/26/2024-10:27:27] [I] Memory Bus Width: 256 bits (ECC disabled)
[08/26/2024-10:27:27] [I] Application Compute Clock Rate: 1.3 GHz
[08/26/2024-10:27:27] [I] Application Memory Clock Rate: 0.816 GHz
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] Note: The application clock rates do not reflect the actual clock rates that the GPU is currently running at.
[08/26/2024-10:27:27] [I] 
[08/26/2024-10:27:27] [I] TensorRT version: 8.6.2
[08/26/2024-10:27:27] [I] Loading standard plugins
[08/26/2024-10:27:27] [I] [TRT] [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 33, GPU 11906 (MiB)
[08/26/2024-10:27:33] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +1154, GPU +1112, now: CPU 1223, GPU 13030 (MiB)
[08/26/2024-10:27:33] [I] Start parsing network model.
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
Could not open file /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
[08/26/2024-10:27:33] [E] [TRT] ModelImporter.cpp:730: Failed to parse ONNX model from file: /workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx
[08/26/2024-10:27:33] [E] Failed to parse onnx file
[08/26/2024-10:27:33] [I] Finished parsing network model. Parse time: 0.00180772
[08/26/2024-10:27:33] [E] Parsing model failed
[08/26/2024-10:27:33] [E] Failed to create engine from model or file.
[08/26/2024-10:27:33] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8602] # /usr/src/tensorrt/bin/trtexec --onnx=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.onnx --saveEngine=/workspaces/isaac_ros-dev/isaac_ros_assets/models/foundationpose/synthetica_detr/rtdetr-l.plan

I have uploaded my custom rt-detr model for you to test on your side, thank you.

rtdetr-l.zip (64.8 MB)

4

Hi @user29288

Have you fixed your issue?

Best,
Raffaello