I am working on a reinforcement learning project that uses Open AI Gym to train a custom robot in Isaac SIM.
So far I have created my own sample inside
isaac-sim/_build/linux-x86_64/release/exts/omni.isaac.samples/omni/isaac/samples/scripts). Here I have managed to import my robot, control it and make it navigate using a similar logic to the
This is how my step function in my custom gym environment looks like:
def step(self, action): print("Number of steps ", self.numsteps) print("Action ", action) # Check if the action is within the action space assert self.action_space.contains(action), "%r (%s) invalid" % (action, type(action)) left_wheel_rpm, right_wheel_rpm = cmd_vel_to_rpm(action, action) self._rc.control_command(left_wheel_rpm, right_wheel_rpm) angle_to_goal = self._rc.angle_to_goal() distance_to_target = self._rc.distance_to_target() extra_values = np.array([angle_to_goal, distance_to_target]) lidar_scans = np.concatenate((self._rc.rear_lidar_depth, self._rc.front_lidar_depth)) position = np.concatenate(([[distance_to_target]], [[angle_to_goal]])) print(lidar_scans) self.current_state = np.concatenate((lidar_scans, position)).reshape(1, 50, 1) reward, done, info = self.calculate_reward(action) self.numsteps += 1 if self.numsteps > self.max_episode_steps: done = True print("robot stepped 500 times") return self.current_state, reward, done, info
However, when the training starts, even though control commands are sent to the
RobotController (self._rc)the editor is frozen and the robot does not move until the training stops - my guess is that the OmniKit editor is not synchronized with the gym environment python code since when the training stops, the robot reproduces all the actions that were supposed to be performed during training.
I saw in the jetracer reinforcement learning example (
isaac-sim/python-examples/jetracer) that they are using the
OmniKitHelper class to launch OmniKit from a Python environment and also update the OmniKit (
self.omniverse_kit.update(self.dt)) whenever an action command is sent in the step function.
def step(self, action): print("Number of steps ", self.numsteps) transformed_action = self.transform_action(action) self.jetracer.command(transformed_action) frame = 0 total_reward = 0 reward = 0 while frame < 3: self.omniverse_kit.update(self.dt) obs = self.jetracer.observations() ...
I would like to know how to implement an update function that can called whenever a new frame needs to be rendered (i.e. when the
step function is executed). Is there a way to do this without using the
OmniKitHelper and having to launch OmniKit from a Python environment? I would like to maintain my current solution where I launch my gym environment from the Omniverse Isaac Sim Editor menu. Thanks!