How to use my own real-world scenario in a sim simulation?

Omniverse Isaac Sim
1

I used this tutorial: Isaac Sim Examples — isaac_ros_docs documentation .

This scene is the official simulation scene. How can I simulate and verify my own real-world scene in the sim?

Here’s my attempt: I used the command ros2 launch nvblox_examples_bringup realsense_example.launch.py, used the save_ply service to generate a *.ply file, converted the ply file to USD format, imported it into the sim, and tried to make the car navigate and avoid obstacles within the passable area of ​​my scene.

This is an actual small room scene that I reconstructed using NVBLOX.

This is my experiment in Simulator. After enabling physics, the car stays on the edge and cannot enter the passable inner area. Even after I checked “kinematic enable”, the car is still bounced away.

My results:

  1. When trying to get the car to enter the passable area of ​​the scene from the outside using navigation, it got stuck at the edge and couldn’t enter.

  2. When trying to put the car into the passable area of ​​the scene close to the ground and click play, the car was bounced away or the sim reported an illegal error.

  3. Continuing from step 2, I checked “kinematic enable”. the car is still being bounced away.

My questions:

  1. Is the NVBLOX reconstruction result exported by save_ply treated as a whole in the sim, preventing access to the internal components?

  2. If 1 is correct, do I need to use a 3D scanner to scan each individual object in the actual scene, create a USD model, and then place them according to their actual positions?

  3. If 1 is incorrect, how can I replicate my scene into the sim for NVBLOX+VSLAM+NAV2 navigation verification?

3

The issue is how PhysX handles the imported mesh collision. When you convert an nvblox PLY reconstruction to USD and add collision properties, PhysX treats it as a closed solid — so anything placed “inside” the room gets ejected because it’s
technically inside the collision volume.

A few approaches to fix this:

Option 1: Use Triangle Mesh collision (not convex hull)

Select the imported mesh prim, and in the Physics properties make sure the collision approximation is set to Triangle Mesh, not Convex Hull or Convex Decomposition. Also ensure the mesh is set as a static collider (no rigid body):

  from pxr import UsdPhysics, PhysxSchema                                                                                                                                                                                                          
                                                                                                                                                                                                                                                   
  stage = omni.usd.get_context().get_stage ()                                                                                                                                                                                                      
  mesh_prim = stage.GetPrimAtPath("/World/your_nvblox_mesh")                                                                                                                                                                                       
                                                                                                                                                                                                                                                   
  # Add collision                                                                                                                                                                                                                                  
  UsdPhysics.CollisionAPI.Apply(me sh_prim)                                                                                                                                                                                                        
                                                                                                                                                                                                                                                   
  # Set to triangle mesh (not convex)                                                                                                                                                                                                              
  mesh_collision = PhysxSchema.PhysxTriangleMeshCol lisionAPI.Apply(mesh_prim)

With triangle mesh collision on a static body, the robot should be able to navigate inside the room geometry.

Option 2: Separate visual and collision geometry

Use the nvblox mesh for visuals only (no collision), and create simple box/plane colliders for the floor and walls. This gives you more control and is lighter on physics performance.

Option 3: Skip physical simulation of the room entirely

If your goal is to test Nav2 navigation, you don’t necessarily need the room mesh to have physics collision. You can use nvblox’s occupancy grid or ESDF output directly in Nav2’s costmap, and only simulate the robot’s physics (wheels on a
flat ground plane).

For nvblox-specific workflow questions, you may also want to check the Isaac ROS documentation (Isaac ROS Nvblox — Isaac ROS).

Closing this topic due to inactivity. If you’re still working on this, please open a new topic with updated details.