Discrepancy in Cuda Soft Body Simulation: Visual vs. Collision Mesh Alignment

Hi,
I’m trying to simulate soft bodies with Cuda device and torch back-end. I found that collision and simulation tetmeshes aren’t following the visualization meshes after the simulation has started, which is a problem that does not occur when simulating with the default world settings.
In the following images a cube has been generated at a certain height before simulation and then left dropping on the ground. The simulation tetmesh remains in the same position while the visualization mesh is actually following the physics constraints.


I’m noticing the same problem with the standalone deformable examples at:

isaac_sim-2023.1.1/standalone_examples/api/omni.isaac.core/deformable.py

The biggest problem is that the prim position of the mesh is not updating and follows the simulation teth-mesh position as you can see in the images.

Does anyone knows how I can solve this problem?

The following folder contains a simplified version of my project that should replicate the problem:
ProblemProject.zip (4.0 KB)

Thank you very much!

Hello,

I think what you are observing has to do with this mentioned here https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs/blob/main/docs/troubleshoot.md#simulation, to be more precise:

When running with the GPU pipeline, updates to states in the scene will not sync to USD. Therefore, values in the UI may appear wrong when simulation is running. Although objects may be updating in the Viewport, attribute values in the UI will not update along with them. Similarly, during simulation, any updates made through the USD APIs will not be synced with physics.

Related to this is e.g. this post here https://forums.developer.nvidia.com/t/world-position-not-updating-during-simulation-run/245372.

I guess the soft body tet visualization is also not synced.
So, I assume you currently just cannot use the visualization with the gpu pipeline.

1 Like

If I understand correctly, if the visualization does not work, also the USD API fails to work, and I should search for workarounds using the Physx SDK?

In my experience, trying to find workarounds using the PhysX SDK is very difficult (and painful).
One reason for that is mentioned in the second link I sent in my previous comment:

You cant access directly PhysX SDK, since that does have C++ API only,

Instead, I would first try to use the DeformablePrimView. You can import it via
from omni.isaac.core.prims.soft.deformable_prim_view import DeformablePrimView.
There is no documentation for this yet, so you need to look into the source code to know which methods it has.
But with this class you are able to get the nodal positions of the simulation mesh via get_simulation_mesh_nodal_positions. This also works with the gpu pipeline.
Perhaps, this might be a good enough workaround for you for getting the position of the deformable prim.

An example for using it:
https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs/blob/main/omniisaacgymenvs/tasks/franka_deformable.py#L166.

As for the visualization, you can try to create your own by extracting the tet mesh of the surface (like mentioned here https://forums.developer.nvidia.com/t/how-can-i-get-deformable-mesh-information-such-as-tet-positions-or-tet-indices/241785) and using https://docs.omniverse.nvidia.com/py/isaacsim/source/extensions/omni.isaac.debug_draw/docs/index.html to draw the tets (lines between each tet point).

But this won’t look as good as the visualization from Isaac Sim. And is probably slower.

1 Like

Yes, thank you! I was unaware of the debug_draw feature, and it’s proving to be very useful!
Regarding the DeformablePrimView, it’s actually why I began simulations using the CUDA and Torch backend. I’ve attempted to use the get_simulation_nodal_positions, but it’s challenging to pinpoint the exact positions in relation to the object without running the simulation at least once to visually observe it. I understand that the DeformablePrimView.get_simulation_mesh_indicies() correspond to the nodal positions, but each time I use a different tetmesh resolution, the indices shift their position on the body.
I need the position for a pick and place task so knowing the exact position respect to the prim is really important.
Thanks again for the detailed answer, @Duce123!

1 Like