Digital twin problem

I am trying to control both real and virtual robot in Issac sim simultaneously and I do that. But the robot gets initialized with the simulated, which makes real robot not safe. How can i used the real robot joint state and update in simulation instead of other way around (simple-joint-control from sdk used for reference). Code attached below:

simple_joint_control.ipynb (15.1 KB)

Thank you

If I understand your objective, you’re trying to run a real robot and then have Isaac Sim mirror the state in the simulated robot, or were you thinking parallel state machines (both sim and real world start in the same initial condition and receive the same commands)? If so, you would want to “rewire” Isaac Sim which is publishing the sim joint state as a CompositeProto (app.connect(simulation_node["output"], "joint_state", lqr_interface, "joint_state")) and listening for command(app.connect(lqr_interface, "joint_command", simulation_node["input"], "joint_position")). You would need to modify this SDK app graph to instead either read this state proto(joint_state) from the real robot and ignore the ones coming from Isaac Sim by changing the channel name to something else. The real robot would be listening to the command channel and react appropriately with its joint state being mirrored in Isaac Sim theoretically. You can change the appropriate channel name from Isaac Sim side as shown in the screenshot below.