Loading .usdz files from Nurec pipeline into Isaac Sim does not load mesh file

Omniverse Isaac Sim
1

Isaac Sim Version

5.1.0

Operating System

Ubuntu 24.04

GPU Information

  • Model: Gigabyte RTX 5090
  • Driver Version: 580.95.05

Loading Nurec output into Isaac Sim loads only the gaussian but not the mesh file

I am following stereo reconstruction using Nurec for loading a real environment into Isaac sim. For this i recorded a bag file of a room using Zed mini camera and followed the steps as per this tutorial: Reconstruct Scenes from Stereo Camera Data — NVIDIA Omniverse NuRec

I was able to load the 3D recon output from 3dgrut into Isaac Sim and I am able to visualize, however there is no collision mesh for the same available as seen in this video by Nvidia Omniverse (https://www.youtube.com/live/ybtJxQbj2NE?si=WuKTYtpAYHLQk9fF&t=1128) timestamp: 18:25 .

Without the mesh I am unable to simulate collision with environment or add other sim ready assets into the environment.

Please let me know how can the mesh for the Gaussian be generated, if I have missed any step in the execution of Nurec pipeline.

image
image590×240 11.1 KB

As seen in the screenshot, there is no volume mesh loaded when loading the .usdz file into Isaac Sim.

Thank You

2

Hi @samahithsa ,

would you be able to open the nvblox_mesh.ply file (e.g. in CloudCompare) and visualize the mesh?

if that doesn’t exists or it’s empty then that means the nvblox steps has issue, or depth or pose has issue.

3

Hello @dizeng ,

The .usd from Nvblox does exist, however it is a separate file from the .usdz file from 3dgrut.

When I load this nvblox_mesh.usd and the 3dgrut output .usdz file onto Isaac sim, there is a mismatch in the scale(nvblox mesh is nearly 100 times scaled down compared to gaussian) and alignment of the Gaussian and the mesh from Nvblox.

From the video, I understand the output should be a scale accurate mesh for collision physics and the gaussian for visual features onto the mesh. Is this the right expectation from Nurec?

4

Hi @samahithsa ,

If you followed the tutorial here, nvblox step should generate a .ply file instead of .usd file, you should first check if nvblox_mesh.ply content is correct, e.g. having the right scale or not, then debug the 3DGRUT output.

The workflow should generate mesh + 3DGS in the same scale.

Another thing to check is, when you train 3dgrut, you need to make sure you passed the right path for the mesh input:

   initialization.fused_point_cloud_path=/workspace/nvblox_mesh/nvblox_mesh.ply
5

Hello @dizeng ,

I do have a .ply file from nvblox and I did pass it to 3Dgrut however it looks like the tutorial has been updated a lot since I last ran it, so my commands are a bit different.

I will follow the updated tutorial and update here if I face the same issue.

Thank you

6

NuRec use case survey

To better understand your workflow and help the NuRec and Isaac Sim teams improve support, please share a bit more about your project by answering the questions below.

  • Project description
    • Briefly describe what your project is about.
    • What do you use the NuRec scenes for? e.g. model training or navigation testing etc.
  • Input setup
    • What’s your embodiment type? (e.g. AMR or Humanoid, which robot model)
    • What sensors are you using (e.g., ZED Mini, RealSense, other stereo/depth cameras)?
    • How are you recording data (e.g., ROS bag, Isaac ROS pipeline, custom recorder)?
  • NuRec pipeline details
    • Which NuRec workflow are you using (e.g., stereo reconstruction, Gaussian Splatting, other)?
    • Which tutorial or documentation page are you following, if any?

Thank you for sharing these details. This information helps prioritize improvements in the NuRec–Isaac Sim integration and guide next steps for your specific use case.

7

Hello @dizeng ,
Following up on this, I followed the steps in the updated tutorial and the .usdz file does not have volume mesh. The video in section 6 of this tutorial ( Reconstruct Scenes from Stereo Camera Data — NVIDIA Omniverse NuRec ) rig_trajectories, sequence_tracks, volume etc., however when i load the .usdz file from 3dgrut, it only shows ‘gauss’, as seen in the earlier screenshot.

This is the only command that is somewhat different from the instructions, rest just have path of the data is modified.

ros2 run isaac_mapping_ros rosbag_to_mapping_data     
--sensor_data_bag_file 24zedm-night-tilt/     
--pose_bag_file 24zedm-night-tilt     
--output_folder_path rosbag_mapping_data     
--pose_topic_name /zed/zed_node/pose     
--image_extension=.jpeg     
--base_link_name=zed_camera_link     
--generate_edex=True     
--camera_topic_config=stereo_cameras.yaml

Contents of stereo_camera.yaml

stereo_cameras:
  - name: zed
    left: /zed/zed_node/left/color/rect/image
    left_camera_info: /zed/zed_node/left/color/rect/camera_info
    right: /zed/zed_node/right/color/rect/image
    right_camera_info: /zed/zed_node/right/color/rect/camera_info

Thank You

8

Hello @VickNV ,

Project Description:

Building realistic environments for UAV navigation, right we are building on two scenarios, indoor navigation for developing localization and safe navigation pipelines and building outdoor environment for UAV simulation in cluttered environments(building, trees, light poles etc.,)

The testing of UAV navigation at this time will be done using Pegasus plugin for Isaac sim.

NuRec will be used for generation of realistic environments for navigation testing with realistic visual features for sensor inputs.

Input setup:

Currently for indoor, handheld Zed Mini camera with data recording to laptop. Once the pipeline is validated, Jetson Orin NX with Zed Mini on UAV and Zed 2i with Orin AGX on UGV is planned.

Next stage would be for large scale scenes using UAV mounted 3D LiDAR, monocular camera, IMU on gimbal with Jetson Orin NX for onboard recording.

All data recorded are ros2 bag files.

NuRec pipeline details:

Currently I am working with a single Zed Mini stereo camera to reconstruct a room, hence I am following stereo reconstruction work flow.

Currently following: Reconstruct Scenes from Stereo Camera Data — NVIDIA Omniverse NuRec

Issues I am facing:

Currently unable to obtain volume mesh from 3DGRUT’s .usdz output to use in Isaac Sim as shown in documentation ( Reconstruct Scenes from Stereo Camera Data — NVIDIA Omniverse NuRec ). Hence only able to load the Gaussian at this stage.

It would also be great if LiDAR-Monocular camera fused output could be integrated into the workflow, since it is not possible to use stereo for larger scale scenes.

Thank You

9

Hello @dizeng and @VickNV ,

To further add to this, I have also tested the Nvidia Physical AI dataset ( nvidia/PhysicalAI-Autonomous-Vehicles-NuRec · Datasets at Hugging Face ).

The file structure of the usdz file is:

image
image287×492 11.1 KB

Where as when I follow the stereo reconstruction workflow as shown in the tutorial, this is my file structure after the last step(3DGUT):

image
image312×114 3.68 KB

Am I missing something here? Are there additional steps after 3DGUT to be done to obtain the file as per the example dataset?

Would greatly appreciate some guidance in this regard.

Thank You

10

The folder structure of the reference workflow is indeed different from what we published, this is expected.
Have you passed export_usdz.enabled=true when running 3dgrut training?

It would be nice if you can use tools like CloudCompare to visualize the nvblox mesh output to make sure it has the right content.

11

Hi, have you solved this problem? I meet the same problem like it. Every step have passed, but .usdz also has default.usda, export_last.nurec, gauss.usda only….

12

Hello @dizeng ,

Yes export_usdz.enabled=true is passed when I run 3dgrut and the .ply mesh is also passed to it. usdz file does get generated, but contents do not match what is shown in the guide or the physical AI dataset.

I am attaching my stereo reconstruction runs using the physical AI dataset of living room sequence(to make sure that isn’t an issue with my dataset), first video is the .ply mesh in cloud compare, second one is the .usdz file of the same in Isaac sim. As you can see the volume mesh is missing.

I have followed the steps in the guide, but the mesh isn’t within the .usdz file.

Please let me know if I have missed anything.

13

No solution yet from my end.

A make shift solution is to load the mesh inside nvblox_mesh folder (the ply file) into Isaac sim directly and align it manually to align with gaussian and add collision properties to the mesh.

But yeah it is a manual process that takes some time.

14

Hi @samahithsa,
Just checking in to see if you were able to resolve the mesh and Gaussian alignment/scale issues with the NuRec pipeline and Isaac Sim, or if you still need support.

15

Hello @VickNV ,

No, I haven’t resolved the issue yet and have not tried to debug further, as I have not had time to work on this.

Not sure what is missing, I have tired the steps from the guide with the sample dataset and still have not got any mesh co aligned with gaussian yet.

So far at least to my knowledge I have not seen anyone online or my colleagues get co aligned mesh, so not sure if nurec is supposed to output such mesh.

Thank You

16

Hi @samahithsa,

I’ve investigated this thoroughly and can now explain why the mesh is missing from your USDZ file.

The short answer: This is expected behavior, not a bug. The 3DGRUT USDZ export intentionally contains only the Gaussian splat representation. The nvblox mesh must be added as a separate post-processing step.

The solution: Use the add_mesh_to_usdz.py script from the 3DGRUT repository:

python -m threedgrut.export.scripts.add_mesh_to_usdz \                                                                                                                                                                                                                                 
  --input_usdz /workspace/3dgrut/export_last.usdz \                                                                                                                                                                                                                                    
  --output_usdz /workspace/3dgrut/export_with_mesh.usdz \                                                                                                                                                                                                                              
  --mesh_ply /workspace/nvblox_mesh/nvblox_mesh.ply \                                                                                                                                                                                                                                  
  --set_collision

This adds the nvblox mesh to the USDZ and sets up collision properties.

Why the confusion? The NuRec stereo reconstruction tutorial doesn’t mention this post-processing step – it jumps directly from 3DGRUT training (Step 5) to using assets in Isaac Sim (Step 6). The 3DGRUT
README documents this step, but users following only the NuRec tutorial wouldn’t know to look there.

The production-ready assets in the NVIDIA PhysicalAI Robotics NuRec Dataset (which have volume, rig_trajectories, etc.) were post-processed with this additional step – that’s why they differ from raw 3DGRUT
output.

Optional flags:

  • –set_collision – enables collision physics on mesh prims
  • –set_invisible – makes mesh prims invisible (useful if you only want physics, not rendering)

Let me know if this resolves your issue!

17

Hello @VickNV ,

Really appreciate the detailed debugging.

It looks like the script to add mesh to .usdz from 3dgrut was added sometime in March 2026 as per the 3dgrut github commit history and I had stopped looking into it after Jan 2026, hence why the confusion was caused.

The usdz does load mesh now, however at least with my data it looks like the mesh is rotated with respect to gaussian by 180 degrees. Not sure if this issue with my data. I expect the Gaussian and mesh would be perfectly aligned, but looks like that is not the case.

If time permits I will try to update the same on this post after I run the same steps on the sample data from Nvidia and check if this issue remains.

Thank You