Controlling actual robot using isaac sim (python, Ros)

Omniverse Isaac Sim
,
#1

We have ReactorX 150 robotic arm with us and we are wondering how could we control the arm using isaac sim.
We have controlled the arm using Rviz (Interbotix package), Also in Isaac sim we have controlled the virtual model using python
Could you please suggest any tutorial ( which has all the required steps).

Thanks in advance

#2

There’s no actual tutorial for actual robot as far as i know. But you can use the joint state publisher from isaac to publish the simulation joint state and then send them to your arm using whatever way of controll you are using for your robot.

If you are using ros control you can create a forward_position_controller and republish the joint state to them.

#3

Thanks for your reply
It would be great if you provide us the links for the same.

Also we were wondering that if we can detect the arm"s joints position as we are detecting in rviz i.e. initial robotic arms position so that if we enter any value for particular joint the actual arm will move in the same way as simulation.

#4

I’m using a slighty older version of issac sim (2022.1.0) and I understood that the latest version has some changes to this schema but nothing major as the subcribe joint state node output jointstates that you have to pass to another node to apply. I also use ROS2 but the ROS1 node should be the same

image
image757×537 50.4 KB

You can set the targetPrim to be the robot and then just listen on the jointstate outputed by Rviz ( which I assume is moveit ?) and then export the joint stat to your actual robot with another controller.

This is my code for controlling a UR robot

class JointController(Node):

    def __init__(self):
        super().__init__('baxter_joint_controller')
        self.publisher = self.create_publisher(Float64MultiArray, '/forward_position_controller/commands', 10)

        self.subscription = self.create_subscription(
            JointState,
            'joint_states_sim',
            self.listener_callback,
            10)

        self.joint_states = {}
        self.joint_names = [
            "shoulder_pan_joint",
            "shoulder_lift_joint",
            "elbow_joint",
            "wrist_1_joint",
            "wrist_2_joint",
            "wrist_3_joint",
        ]

    def listener_callback(self, msg:JointState):
        for name, pose in zip(msg.name, msg.position):
            self.joint_states[name] = pose
        
        out_msg = Float64MultiArray()
         
        for el in self.joint_names:
            out_msg.data.append(self.joint_states[el])
        
        self.publisher.publish(out_msg)

If you are using moveit as a way of controll isaac has a good documentation concerning it moveit_isaac and the same for ROS2.

If on the other hand you want to use Isaac to find a path to an object and discard moveit you can use any of the 3 motion generation algorithm they propose to have the robot follow a cube and then use the same graph as above to control your real robot.
However with this system i have not found a way to get feedback from the real robot unless you disable the path planning

#5

Thanks Buddy !