Replicator Randomization and Physics Simulation

[christof.schuetzenhoefer]

Continuing the discussion from Combining physics simulation with domain randomization using Replicator:

I have a very similar question to the above mentioned post. In my case, I want to modify the pose of the pallets and wait until the pallet is not moving anymore. BUT, in addition, I want to use the functions of the replicator like randomizing textures, material, lights, and more.

From my experiments, I have noticed that using rep.orchestrator.step() and then doing physics simulation with simulation_app.world.step(render=False) are not really working (the pallets are unchanged and not affected by physical effects).

Is there any way to do a quick simulation, so that the objects can drop and are not overlapping with each other, after applying the randomizations?

In my opinion, it seems that the replicator function and physics simulation are not really cooperative in this scenario.

Please give me advices, this is really important for me!!

Thanks and Regards,
Christof

[ahaidu]

Hi there,

this is currently limiting when using replicator with isaac sim world together, this is planned to be improved in future releases.

Since world.step() will trigger a randomization and a writer data collection. As a workaround one can use get_data directly from the annotators instead of using a writer, this way you will have more control on when to access the data. It will however trigger the randomizer, so you might need to randomize things directly in isaac sim.

Another approach would be to use replicator only, and trigger the data writing at specific time intervals. To make sure you can simulate the prims you should set their physics properties (rep.physics.rigid_body).

Let me know if you need further assistance.

Best,
Andrei

[ahaidu]

@christof.schuetzenhoefer we just added a snippet to the docs doing this, in case it helps:
https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/reference_python_snippets.html#get-synthetic-data-at-custom-events-in-simulations