# Issue with FoundationPose and new object **URL:** \ **Category:** Isaac ROS\ **Created:** [February 20, 2026, 3:37am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112 "2026-02-20T03:37:57Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 20, 2026, 3:37am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/1 "2026-02-20T03:37:57Z")
I am a student learning about computer vision and pose estimation. I have installed FoundationPose on a Jetson Orin NX 16GB with an Intel realsense D435i depth camera I have completed the tutorials for RT-DETR and FoundationPose I am trying to run FoundationPose on a new object, and I have added the mesh .obj file (pinkcat in question in the following command) and the texture file to the folders, and I am trying to run this command: `ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan mesh_file_path:=${ISAAC_ROS_WS}/mesh/pinkcat_scaled.obj texture_path:=${ISAAC_ROS_WS}/mesh/Texture_pinkcat.png launch_rviz:=true launch_bbox_to_mask:=true mask_height:=480mask_width:=640` ![image](https://global.discourse-cdn.com/nvidia/original/4X/f/e/1/fe1af6437c20cc6e4d0a94cd0462ea9f41484978.png) I am experiencing this issue (shown in screenshot) where no bounding box and no camera live feed is present. I am not sure how to begin debugging this, and would appreciate some guidance for this issue. Thank you very much! ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 24, 2026, 4:50am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/2 "2026-02-24T04:50:37Z")
just to add on: even on isaac\_ros\_rtdetr tutorial: ![image](https://global.discourse-cdn.com/nvidia/original/4X/2/d/3/2d3c0feacc2004095895ec267dba79be13a09a79.png) I seem to be getting no image output from rqt\_image\_view I’m not quite sure what might be the problem, as I have followed and double-checked the camera setup using the Isaac ROS Realsense setup. Once again, any guidance would be greatly appreciated. Thank you! ---
**Author:** ![vchuang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vchuang/32/436710_2.png) [@vchuang](https://forums.developer.nvidia.com/u/vchuang)\ **Post date:** [February 24, 2026, 6:38am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/3 "2026-02-24T06:38:24Z")
Hello @e0014667, Welcome to the Isaac ROS forum and thanks for your post! One thing to note is that the command you entered to launch FoundationPose should have two separate arguments (you missed a space): `mask_height:=480 mask_width:=640` Also, `score_engine_file_path` should point to the **score** engine, not the refine engine, e.g.: ```auto score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan ``` You could reference [here](https://nvidia-isaac-ros.github.io/v/release-3.2/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#run-launch-file) for running the command. As for running the isaac\_ros\_rtdetr example, have you tried running with the [rosbag](https://nvidia-isaac-ros.github.io/v/release-3.2/repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index.html#run-launch-file) to see if it works? `ros2 bag play -l ${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_rtdetr/quickstart.bag` ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 25, 2026, 5:03am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/4 "2026-02-25T05:03:43Z")
very good day vchuang, apologies for the late reply and thank you very much for your response I have started the foundationpose setup from scratch and attempted the rosbag example as you suggested: ![image](https://global.discourse-cdn.com/nvidia/original/4X/8/7/5/87583ac0215c106b392bd69165608b5c523ff8f7.jpeg) rosbag seems to work fine, so I suspect the issue is somehow in my image stream from realsense camera to the rtdetr example. I’m not very sure where to begin debugging at the rtdeter side, unfortunately… ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 25, 2026, 5:11am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/5 "2026-02-25T05:11:47Z")
quick update: I found an error in the log when I attempted to rerun the realsense example: [component\_container\_mt-1] [ERROR] [1771996054.503912747] [isaac\_ros\_examples.container]: Could not find requested resource in ament index [ERROR] [launch\_ros.actions.load\_composable\_nodes]: Failed to load node ‘realsense2\_camera’ of type ‘realsense2\_camera::RealSenseNodeFactory’ in container ‘/isaac\_ros\_examples/container’: Could not find requested resource in ament index Edit: I read some github issues from realsense-ros and one of the suggestions was to reverse the usb-c cable orientation and try again, this did not work some other commands I have tried: $ros2 launch realsense2\_camera rs\_camera.launch Package ‘realsense2\_camera’ not found: “package ‘realsense2\_camera’ not found, searching: [‘/opt/ros/humble’]” $ros2 launch realsense2\_camera rs\_camera.launch initial\_reset:=true Package ‘realsense2\_camera’ not found: “package ‘realsense2\_camera’ not found, searching: [‘/opt/ros/humble’]” $lsusb Bus 002 Device 005: ID 8086:0b3a Intel Corp. Intel(R) RealSense™ Depth Camera 435i Bus 002 Device 002: ID 0bda:0489 Realtek Semiconductor Corp. 4-Port USB 3.0 Hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 0bda:c822 Realtek Semiconductor Corp. Bluetooth Radio $realsense-viewer (inside the Docker container) launches realsense viewer and color / stereo both work ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 25, 2026, 6:31am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/6 "2026-02-25T06:31:45Z")
Upon further investigation, it seems the realsense2 package is not installed on my docker container, even though I went through the entire NVIDIA realsense ros setup tutorial on [Isaac ROS RealSense Setup — isaac\_ros\_docs documentation](https://nvidia-isaac-ros.github.io/v/release-3.1/getting_started/hardware_setup/sensors/realsense_setup.html) `$ros2 pkg list | grep realsense` returns empty & `$dpkg -l | grep realsense` also returns empty in the Docker container. I can confirm that this step in the realsense install tutorial: `cd ${ISAAC_ROS_WS}/src/isaac_ros_common/scripts && ` `touch .isaac_ros_common-config && ` `echo CONFIG_IMAGE_KEY=ros2_humble.realsense > .isaac_ros_common-config` was properly followed and so was this step: `cd ${ISAAC_ROS_WS} && ` `colcon build --symlink-install --packages-up-to-regex realsense*` I do not understand why the realsense package just doesnt seem to work. any guidance or resources on how to troubleshoot this would be immensely appreciated… Thank you! ---
**Author:** ![vchuang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vchuang/32/436710_2.png) [@vchuang](https://forums.developer.nvidia.com/u/vchuang)\ **Post date:** [February 25, 2026, 7:42am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/7 "2026-02-25T07:42:02Z")
Hello @e0014667, After you `colcon build` the workspace, did you `source install/setup.bash`? If not, you need to do it after the build completes: ```auto cd ${ISAAC_ROS_WS} source install/setup.bash ``` Then verify that ROS sees the package: ```auto ros2 pkg list | grep realsense ``` You need to do this every time you start a new terminal. ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 26, 2026, 4:29am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/8 "2026-02-26T04:29:51Z")
Good day, I have tried source install/setup.bash after colcon step for realsense, and my rtdetr and foundationpose tutorials work! (this source step was not mentioned in the realsense setup tutorial) I have also fixed the initial command: `ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan mesh_file_path:=${ISAAC_ROS_WS}/mesh/pinkcat_scaled.obj texture_path:=${ISAAC_ROS_WS}/mesh/Texture_pinkcat.png launch_rviz:=true launch_bbox_to_mask:=true mask_height:=480 mask_width:=640` based on your pointers from earlier unfortunately the camera still seems to not show up on rviz ![image](https://global.discourse-cdn.com/nvidia/original/4X/2/d/e/2dec9fe4fed5b4ffa841c9c48f3230d0f0526d28.png) my expectation is that some image would pop up and prompt me to click the bounding box due to `launch_bbox_to_mask:=true` but I do not observe that happening either. is there an issue with my ros2 command? or perhaps some steps I am missing to enable foundationpose for a custom object here is the log file when I run the command mentioned above: [fp\_log.txt](https://forums.developer.nvidia.com/uploads/short-url/j0YIY1hcWOEZc225vowmy3PLxwr.txt) (9.7 KB) once again, much appreciated for your guidance, thank you for taking the time to look at my issue ---
**Author:** ![vchuang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vchuang/32/436710_2.png) [@vchuang](https://forums.developer.nvidia.com/u/vchuang)\ **Post date:** [February 26, 2026, 9:39am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/9 "2026-02-26T09:39:49Z")
Your RViz screenshot shows **Global Status: “Fixed Frame has no tf data”** , which means the frame selected in **Global Options → Fixed Frame** is not present in `/tf` or `/tf_static`. You need to either choose a frame that already exists in TF, or publish one yourself. If you just want to test the feature and don’t have a suitable TF yet, you can simulate one in a separate terminal with: ```auto ros2 run tf2_ros static_transform_publisher \ 0 0 0 0 0 0 1 \ world camera_1_link ``` This will continuously publish a TF from `world` → `camera_1_link`. Replace `camera_1_link` with whatever frame your camera driver is actually using (you can see it in `/tf`). After that, in RViz, set **Fixed Frame = `world`** so RViz can use that frame without errors. ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [February 27, 2026, 5:55am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/10 "2026-02-27T05:55:26Z")
Much appreciated for pointing out the fixed frame issue. I have also noticed that I have made the foolish assumption that foundationpose / Rviz would launch the realsense camera (or at least run the driver) but that was not the case. After launching the realsense driver in another terminal using `ros2 launch realsense2_camera rs_launch.py \` `depth_module.profile:=640x480x30 \` `rgb_camera.profile:=640x480x30` using a 3rd terminal to run `ros2 run tf2_ros static_transform_publisher \` `0 0 0 0 0 0 1 \` `world camera_link` and finally, changing Fixed Frame = world & Camera \> Topic = /camera/color/image\_raw ![image](https://global.discourse-cdn.com/nvidia/original/4X/6/2/7/62765d7f1958eec1b7f9876f261853482d7b5b8c.jpeg) I finally get a camera output on Rviz! However, I am aware there is a mask that has to be fed into foundationpose to enable the pose estimation. my understanding is that launch\_bbox\_to\_mask:=true parameter launches some bounding box to mask converter, but it still seems like the Node is idle once Rviz is launched. the pose estimation still doesnt seem to be working for me, so it seems I will be focusing on debugging the mask issue now ---
**Author:** ![vchuang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vchuang/32/436710_2.png) [@vchuang](https://forums.developer.nvidia.com/u/vchuang)\ **Post date:** [March 2, 2026, 8:24am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/11 "2026-03-02T08:24:03Z")
It would be easier for us to assist if you could share the **exact command** you used to launch FoundationPose, and the **full console log** from that run. Note that `launch_bbox_to_mask:=true` only starts the Detection2D → mask node, which converts **existing 2D detections** into a mono8 mask. It does **not** start a 2D detector by itself. If you only ran the command you posted earlier`ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py ... launch_bbox_to_mask:=true ... `you would still be missing a 2D detector (for example SyntheticaDETR). In our [reference example](https://nvidia-isaac-ros.github.io/v/release-3.2/repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index.html#build-package-name), this is provided via: `rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/sdetr_grasp.plan `which brings up the RT-DETR detector that publishes the bounding boxes needed for the bbox→mask node and FoundationPose to work. ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [March 2, 2026, 3:12pm UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/12 "2026-03-02T15:12:45Z")
Apologies for that confusion, I will list step-by-step the commands I have been running: 1. launch realsense camera `ros2 launch realsense2_camera rs_launch.py \` `depth_module.profile:=640x480x30 \` `rgb_camera.profile:=640x480x30` 2. simulate TF `ros2 run tf2_ros static_transform_publisher ` `0 0 0 0 0 0 1 ` `world camera_link` 3. run foundationpose (I have adjusted the command with the DETR portion you mentioned above, but the bounding box still doesnt seem to work) `ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan mesh_file_path:=${ISAAC_ROS_WS}/mesh/pinkcat_scaled.obj texture_path:=${ISAAC_ROS_WS}/mesh/Texture_pinkcat.png rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/sdetr_grasp.plan launch_rviz:=true launch_bbox_to_mask:=true mask_height:=480 mask_width:=640` here is the log output from the foundationpose terminal: [fp\_log1.txt](https://forums.developer.nvidia.com/uploads/short-url/ibsjhZfcSi2v1Kg8WA5GAqIjSxc.txt) (10.2 KB) and here is a screenshot of my RViz after changing Fixed Frame = world & Camera \> Topic = /camera/color/image\_raw ![Screenshot from 2026-03-02 23-01-38](https://global.discourse-cdn.com/nvidia/original/4X/9/6/4/964e119416a3b97dbe682c72132cf4639be8e02a.jpeg) the little pink cat is what I am attempting to detect / pose-estimate using foundationpose Once again, thank you so much for your guidance (and many apologies for my inadequate knowledge in this space) ---
**Author:** ![vchuang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vchuang/32/436710_2.png) [@vchuang](https://forums.developer.nvidia.com/u/vchuang)\ **Post date:** [March 3, 2026, 9:39am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/13 "2026-03-03T09:39:26Z")
SyntheticaDETR is an exact-instance detector trained on a fixed set of objects, so when you swap in a new object like your pink cat without a matching trained RT-DETR model, the detector won’t be able to detect it. To get correct results on a new object, you need a detector model trained specifically for it. You can follow this [documentation](https://nvidia-isaac-ros.github.io/v/release-3.2/concepts/pose_estimation/foundationpose/tutorial_create_your_own_mesh.html) to create a 3D mesh for your specific object, and refer to this guide [here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/isaac/models/synthetica_detr?version=1.0.0) for model training details. ---
**Author:** ![e0014667](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@e0014667](https://forums.developer.nvidia.com/u/e0014667)\ **Post date:** [March 5, 2026, 7:12am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/14 "2026-03-05T07:12:56Z")
Good day! Apologies for the late response. Many thanks for your guidance regarding this issue. I have begun looking into training a detr model for the 2D detection. much appreciated for your patience and prompt responses to my issues. I will be closing the issue now, as I believe this post has run its course. Once again, my sincerest thanks for your guidance. ---
**Author:** ![2287278570](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@2287278570](https://forums.developer.nvidia.com/u/2287278570)\ **Post date:** [March 9, 2026, 6:07am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/15 "2026-03-09T06:07:54Z")
Hello? have you succeed? I also meet this problem. Can you share your method? ---
**Author:** ![vchuang](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/vchuang/32/436710_2.png) [@vchuang](https://forums.developer.nvidia.com/u/vchuang)\ **Post date:** [March 10, 2026, 4:20am UTC](https://forums.developer.nvidia.com/t/issue-with-foundationpose-and-new-object/361112/16 "2026-03-10T04:20:00Z")