With Fabric Enabled, Particle Cloth Position Reading Causes Reset

[yinhang1]

Isaac Sim Version

4.2.0
4.1.0
4.0.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

• Model: NVIDIA GeForce RTX 4090
• Driver Version: 550.127.05
• CUDA Version: 12.4

Topic Description

We’re encountering an issue with particle cloth behavior when trying to read particle positions while fabric is enabled (with USD updates disabled). The behavior differs significantly between two scenarios:

Scenario 1 (Working):

• Cloth is spawned mid-air
• Only physics stepping is performed
• Result: Cloth falls naturally as expected

Scenario 2 (Problem):

• Cloth is spawned mid-air
• Physics stepping AND particle position reading are performed simultaneously
• Result:
  • Cloth appears to “float” mid-air without falling
  • Visible flickering when zoomed in
  • Appears to reset to initial position after each read/step

Attempted Solutions

We’ve tried reading particle positions through:

1. Reading points attribute from the fabric prim:

fabric_prim = omni.isaac.core.utils.prims.get_prim_at_path(
    prim_path=cloth_prim_path, 
    fabric=True
)

2. Convert this fabric prim into a usdrt geom mesh and read from its point attribute:

usdrt.UsdGeom.Mesh(fabric_prim)

3. Following this guide for reading points attribute as described here: USDRT Documentation - What’s in Fabric

Questions

1. Is this a known issue with reading particle positions from fabric?
2. Is there a recommended way to read particle positions without causing this reset behavior?

We can provide additional details or specific code examples if needed. Any help would be greatly appreciated!

[clow]

I am not able to repro the issue. Would you be able to provide the usd file as well as the code examples?

[yinhang1]

Hi @clow , thanks for getting back to us!

Here’s a one way I found to reproduce this behavior with the Isaac SimParticleClothDemo at extsPhysics/omni.physx.demos/omni/physxdemos/scenes/ParticleClothDemo.py

1. Imports

import numpy as np
from omni.isaac.core.utils.prims import get_prim_at_path

2. At the end of def create, we add:

self.fabric_cloth_prim = get_prim_at_path(prim_path="/World/Cloth", fabric=True)

3. We add an update method and read particle positions from the fabric prim:

    def update(self, stage, dt, viewport, physxIFace):
        super().update(stage, dt, viewport, physxIFace)
        # Read particle positions from fabric particle prim
        points = np.array(self.fabric_cloth_prim.GetAttribute("points").Get())
        mean = points.mean(axis=0)
        print(mean)

4. Enable fabric while you run this.

Here’s what I’m seeing while running it with Isaac 4.2:

Let me know if you can reproduce this on your end! Always happy to provide more details if needed. Thanks again!

[clow]

Thank you for the repro and I can see the issue with the steps you provided. I spoke to a colleague who is more experienced than me on Isaac Sim. He was surprised that

np.array(self.fabric_cloth_prim.GetAttribute("points").Get())

actually works because numpy does not have constructors for fabric USD structures.

Since the fabric prim should be on GPU memory, the preferred way to make modifications is to use a warp kernel as shown in the guide USDRT Documentation - What’s in Fabric.

If you need to use numpy, there is a workaround. You can copy the fabric data to a list and use that list to create the numpy array. I have provided a code snippet for your reference below:

        fabric_points = self.fabric_cloth_prim.GetAttribute("points").Get()
        list = []
        size = len(fabric_points)
        for i in range(size-1):
            list.append(fabric_points[i])
        points = np.array(list)

[cgokmen]

Hi @clow, thanks for the advice.

Indeed your proposed method of iterating through the Vec3fArray works, however is incredibly slow (takes ~100x the equivalent np.array() call on the equivalent usd vec3farray) and is thus unusable.

We did try the described warp approach but for us the Fabric data is not on GPU (we’re using Fabric CPU mode, no GPU pipeline), and so when we try to create a wp.array from the usdrt Vec3fArray it complains about the array being on CPU (it expects it to be on GPU so that it can use the __cuda_array_interface__).

Alternatively we tried an approach using the usdrt SelectPrims API and the wp.fabricarrayarray class, as described on the documentation here. That looks like this in our implementation. This works many times but seems finicky, at least on Windows it seems to occasionally produce segfaults that we have been unable to diagnose, so we are unsure about actually deploying this.

What’s more surprising is that the Vec3fArray object that we get when we call self.fabric_cloth_prim.GetAttribute("points").Get() clearly has an __array_interface__ member that seems to be more or less correctly populated with the right shape etc. so it at least makes a claim to support conversion to numpy by doing np.array(self.fabric_cloth_prim.GetAttribute("points").Get()) . Is there any chance you could reach out to someone on the USDRt team to figure out what’s going wrong here? I believe the issue is that the pointer that’s provided inside the array interface doesn’t actually point at the right location - it seems to actually be a bug.

[clow]

I have filed a bug report and we will be working to find a solution to this problem. Thank you for your patience.

[clow]

Want to provide a quick update. I tried with the latest internal version of Isaac Sim and it is possible to use list as a staging buffer without the need to copy the data. So the code below will work:

    staging_points = list(self.fabric_cloth_prim.GetAttribute("points").Get())
    numpy_points = np.array(staging_points)
    mean = numpy_points.mean(axis=0)

but creating the numpy array directly from the fabric array still does not work

    numpy_points = np.array(self.fabric_cloth_prim.GetAttribute("points").Get())

I think the most optimal solution is to use warp array as a staging buffer but for now, would this workaround address your issue?