Attaching camera to mobile robot | Robot is moving but camera is stationary

Omniverse Isaac Sim
, ,
1

I followed the tutorial 2.5. Add Camera and Sensors — Omniverse IsaacSim latest documentation and added 2 cameras under 2 Xforms in the robot USD which I created and saved the USD for simulation.
Robot I am using is ridgeback_franka robot.

I checked using the viewport 2 to see the camera is moving with the mobile robot Check the video here.

But during Simulation the camera is not moving with the robot. Check the video here.

How to attach the camera to the robot in such a way that the camera moves with the robot’s frame in the simulation or in a RL environment ?
ridgeback_franka_v3.zip (37.3 KB)

from omni.isaac.kit import SimulationApp


simulation_app = SimulationApp({"headless" : False})

from omni.isaac.core.utils import rotations, prims
from omni.isaac.core import World
from omni.isaac.core.objects import VisualCuboid, DynamicCuboid
from omni.isaac.core import SimulationContext
from omni.isaac.core.utils.stage import add_reference_to_stage
from omni.isaac.core.utils.types import ArticulationAction
from omni.isaac.sensor import Camera


from ridgeback_controller import RidgebackController
from ridgeback_panda import Ridgeback_Panda



test_posture = np.array([ 0.0263547, 1.13884234,  0.03276233,  0.74701278,  0.01650456,  1.09152202, -0.04337771])


posture = np.array([0.239, -0.207, -0.191, -1.681, 0, 2.178, -0.048])
finger_posture = np.array([0.004,0.003])

posture2 = np.array([0, -1.00401, -0.95699, 0.087, -1.436, 1.257, -1.675])
vel = 0.5*np.ones((7,))


my_world = World(stage_units_in_meters=1.0)

my_robot = my_world.scene.add(
      Ridgeback_Panda(
            prim_path = "/World/Robot",
            usd_path = "./ridgeback_franka_v3.usd",
            name  = "robot",
            end_effector_name = "endeffector",
            position = np.array([0,0,0]),
            orientation = np.array([1,0,0,0]),   
      )
)

# camera = prims.create_prim(
# "/World/Robot",
# "Camera",
# translation=np.array([0.03721, 0.13557, 1.9807]),
# orientation=np.array([0.6816, 0.154, -0.20498, -0.68519]),
# # attributes={
# #       "focusDistance": FOCUS_DIST,
# #       "focalLength": FOCAL_LEN,
# #       "horizontalAperture": HORIZONTAL_APERTURE,
# #       "verticalAperture": VERTICAL_APERTURE,
# # },
# )



# camera = my_world.scene.add(Camera(
#     prim_path="/World/Robot/workspace_cam/workspace_camera",
#     position=np.array([0.03721, 0.13557, 1.9807]),
#     frequency=20,
#     resolution=(256, 256),
#     orientation=np.array([0.6816, 0.15438, -0.20498, -0.68519])),
# )

base_controller = RidgebackController(name="ridgeback_controller")



# my_target = my_world.scene.add(
#       VisualCuboid(

#       )
# )



my_world.scene.add_default_ground_plane()
my_world.reset()
my_robot.initialize()
# camera.initialize()

# camera.add_motion_vectors_to_frame()

i = 0

while simulation_app.is_running():
      my_world.step(render = True)      
      # my_robot.set_joint_positions(positions = test_posture, joint_indices = my_robot.panda_dof_indices)
    #   my_robot.set_joint_positions(positions = finger_posture, joint_indices = my_robot.panda_finger_dof_indices)
    #   print(my_robot.get_joint_positions(joint_indices = my_robot.panda_dof_indices))
    #   print(my_robot.get_joint_positions(joint_indices = my_robot.panda_finger_dof_indices))
      my_robot.apply_wheel_actions(base_controller.move_robot(
            linear_velocity=0.4,
            linear_command="Y",
            angular_velocity=0,
      ))





simulation_app.close()

This the running python code for the running simulation in the second video.

Thanks and regards,
T.

2

Hi there,

you need to attach the camera xform to a robot part, you currently have it on its own in the workspace. In the first video you are selecting the whole workspace that is why the camera is moving as well. In the simulation the robot moves relative to the workspace, so the camera stays in the same position.

Best

1 Like
3

Hello @ahaidu ,
Thanks for your reply.
So the Xform which holds the camera has to be fixed with any robot Xform by a fixed joint ? Or I can just keep the Camera inside a Robot Xform ?

4

No physics is required (fixed joint), you can use USD Parent/Child relationships

1 Like
5

Dear Andrei,
I did as you told. The camera is now moving with the robot frame.

Thank you very much,
Regards,
Tribikram.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.