The context is, for example, when using GridCloner for RL purposes, such as in this orbit example. I start the simulation app from a standalone Python script.
When you move an object in the base environment, the corresponding objects in the cloned environments also move (but not the other way around). This transformation association between the objects of the base and the specific cloned environment breaks when you move the object in the specific cloned environment at least one time manually (e.g., with the gizmos from the GUI), while the association stays the same for the other clones environment objects.
The attached video below demonstrates the issue well.
Is this a bug or a known working principle? I would appreciate some info regarding why this happens and how to prevent it from happening.
1 Like
Hi @ozhanozen
This is a known issue with how the cloner in Isaac Sim behaves. In 2022.2.1 release, it currently “inherits” all the USD attributes/properties from the template/source environment to the clones ones. Due to this, when you are doing the GUI transformation in env_0, the changes are getting propogated to env_N with N=1,…
There has been a fixed made for this for the upcoming 2023.1 release, where the environments “copy” from the source prim and not inherit.
As a quick fix, you might be able to get it working by modifying the omni.isaac.cloner:cloner.py file. Instead of doing the following in L-215 in the class Cloner:
env_spec.inheritPathList.Prepend(source_prim_path)
you can change it to do a copy from source:
Sdf.CopySpec(env_spec.layer, Sdf.Path(source_prim_path), env_spec.layer, Sdf.Path(prim_path))
I hope it helps solve the issue!
1 Like
@mmittal Thanks a lot for the information.
The same issue appears (while running Isaac Sim from a Python script) when I create an XFormPrim for the source prim and apply set set_world_pose on it. Is this due to the API using USD as the middle layer?
Is there actually a way to bypass the USD (e.g., when using flatcache) for the set_world_pose and get_world_pose operations and directly read/write on the cache? This would also be beneficial for performance reasons considering an RL training pipeline.
Yes. Any USD-level changes affect all the clones that are inheriting from each other. Currently (in Isaac Sim 2022.2), it isn’t possible to by-pass the USD API for some things.
For your usecase, if the prim you are manipulating has physics enabled on it, I recommend using the RigidPrimView or ArticulationView classes from Isaac Sim. There you can directly set the poses through Omni-Physics (i.e. via flatcache mechanism).
@mmittal, thanks a lot. As far as I understand, RigidPrimView or ArticulationView should be preferred over XformPrimView as they use flatcache mechanism, which is faster for RL reasons. However, I am a bit confused about how to do this.
In our situation, we have a “module” xform, that represents all the parts inside, including some “bolts” (with meshes). We want, that when we move the module, all the bolts move accordingly, and then we access the positions of these bolts at each step of the training. How would you set the module and bolt, in this case, using RigidPrimView or ArticulationView?
I have tried making the module have an articulation root (and accessing it through ArticulationView), and making its “bolt” (component) have rigid body properties (and accessing it through RigidPrimView). However, I am not sure if it worked as intended, as the training is very slow. It would be nice to have a guideline regarding how to set up access with complex hierarchies using RigidPrim and Articulation.
Hi @ozhanozen
For this case, it probably makes sense that you have a rigid prim view for each asset (module as well as the bolts) and offset all of them together using the physics API (i.e. apply the same offset transformation to each and set them in the world frame using the set_world_poses method).
This doesn’t exploit the scene hierarchy that nicely though. But I am not sure if the PhysX tensor APIs can allow such operations. It might be possible to handle a more generalized version of the tensor APIs called Fabric and USDRT that will be available from Kit 105.
In your case, is it that you don’t want to create a rigid body on the module (i.e. it is a static collider and not really movable)?
Hi @mmittal
The module is not an individual part but rather a container that has maybe 150 small subcomponents inside, including the screws. If I put a rigid body on the module, I cannot put one to the screw as the screw is the child of a module (actually, there are even more layers in between), and PhysicsX complains.
As all these 150 subcomponents are normally fixed to each other, when you move a module, you expect that all the subcomponents move accordingly. This would be very cumbersome to do without using a common parent element such as the “module”, as I would need to iterate for all subcomponents. That is why I thought putting an Articulation on the “module” (and a rigid body on the subcomponents) could be an option. Does this make sense? If so, how should I structure the module/subcomponent assets for optimum performance? (e.g., do I need to construct PhysicsFixedJoints between the module and screws and similar for performance reasons)
Thanks in advance for your support.