Hello everyone, I am using the differential car jetbot. I have two methods. One is through the omnigraph method, and the other is to write the control in the form of python code. But in the end, I couldn’t make the car move. Can you help me to see what’s wrong with the two methods? How to solve it?Any help will be appreciated.
Below is my omnigraph graph
this is python code:
from omni.isaac.examples.base_sample import BaseSample
from omni.isaac.core.utils.types import ArticulationAction
from omni.isaac.core.utils.nucleus import get_assets_root_path
from omni.isaac.core.utils.stage import add_reference_to_stage
from omni.isaac.core.robots import Robot
import numpy as np
import carb
class HelloWorld(BaseSample):
def init(self) → None:
super().init()
return
def setup_scene(self):
world = self.get_world()
world.scene.add_default_ground_plane()
assets_root_path = get_assets_root_path()
if assets_root_path is None:
carb.log_error("Could not find nucleus server with /Isaac folder")
asset_path = assets_root_path + "/Isaac/Robots/Jetbot/jetbot.usd"
add_reference_to_stage(usd_path=asset_path, prim_path="/World/Fancy_Robot")
jetbot_robot = world.scene.add(Robot(prim_path="/World/Fancy_Robot", name="fancy_robot"))
return
async def setup_post_load(self):
self._world = self.get_world()
self._jetbot = self._world.scene.get_object("fancy_robot")
# This is an implicit PD controller of the jetbot/ articulation
# setting PD gains, applying actions, switching control modes..etc.
# can be done through this controller.
# Note: should be only called after the first reset happens to the world
self._jetbot_articulation_controller = self._jetbot.get_articulation_controller()
# Adding a physics callback to send the actions to apply actions with every
# physics step executed.
self._world.add_physics_callback("sending_actions", callback_fn=self.send_robot_actions)
return
def send_robot_actions(self, step_size):
# Every articulation controller has apply_action method
# which takes in ArticulationAction with joint_positions, joint_efforts and joint_velocities
# as optional args. It accepts numpy arrays of floats OR lists of floats and None
# None means that nothing is applied to this dof index in this step
# ALTERNATIVELY, same method is called from self._jetbot.apply_action(...)
self._jetbot_articulation_controller.apply_action(ArticulationAction(joint_positions=None,
joint_efforts=-np.random.rand(9) * 1000,
joint_velocities=None))
return
Now, I can make the car move by applying force to the wheels, but sometimes the car bounces up during the movement, why is that?
Now, I solved the problem of the car bouncing up, but I encountered a new problem. When the car first started running, it walked in a straight line and then started to turn in a circle. I tried to use PID control, but it didn’t seem to work.Can someone help me? @toni.sm can you help me？code show as below.
from omni.isaac.examples.base_sample import BaseSample
from omni.isaac.core.utils.types import ArticulationAction
from omni.isaac.core.utils.nucleus import get_assets_root_path
from omni.isaac.core.utils.stage import add_reference_to_stage
from omni.isaac.core.robots import Robot
from omni.isaac.dynamic_control import _dynamic_control
from omni.isaac.core import SimulationContext
import numpy as np
import carb
class HelloWorld(BaseSample):
def init(self) → None:
super().init()
return
def setup_scene(self):
world = self.get_world()
world.scene.add_default_ground_plane()
assets_root_path = get_assets_root_path()
if assets_root_path is None:
carb.log_error("Could not find nucleus server with /Isaac folder")
asset_path = assets_root_path + "/Isaac/Robots/Jetbot/jetbot.usd"
add_reference_to_stage(usd_path=asset_path, prim_path="/World/Fancy_Robot")
jetbot_robot = world.scene.add(Robot(prim_path="/World/Fancy_Robot", name="fancy_robot"))
return
async def setup_post_load(self):
self._world = self.get_world()
self._jetbot = self._world.scene.get_object("fancy_robot")
# This is an implicit PD controller of the jetbot/ articulation
# setting PD gains, applying actions, switching control modes..etc.
# can be done through this controller.
# Note: should be only called after the first reset happens to the world
# self._jetbot_articulation_controller = self._jetbot.get_articulation_controller()
dci = _dynamic_control.acquire_dynamic_control_interface()
self.dci = dci
art = dci.get_articulation("/World/Fancy_Robot")
self.art = art
dof_left =dci.find_articulation_dof(art, "left_wheel_joint")
dof_right =dci.find_articulation_dof(art, "right_wheel_joint")
dof_state = dci.get_dof_state(dof_left,7)
dof_state2 = dci.get_dof_state(dof_right,7)
self.dof_states = [dof_state.vel,dof_state2.vel]
print(self.dof_states)
# print position for the degree of freedom
self.target = np.array([15.0,15.0], dtype=np.float32)
self.previous_error = np.zeros_like(self.target)
self.integral = 0
# Adding a physics callback to send the actions to apply actions with every
# physics step executed.
self._world.add_physics_callback("sending_actions", callback_fn=self.step_callback)
return
def control(self,current, dt, Kp=0.005, Ki=0.005, Kd=0):
error = self.target - current
self.integral += error * dt
derivative = (error - self.previous_error) / dt
self.previous_error = error
return Kp * error + Ki * self.integral + Kd * derivative
def step_callback(self,step_size):
# print("simulate with step: ", step_size)
effort = self.control(self.dof_states, step_size)
effort = effort.astype(np.float32)
self.dci.wake_up_articulation(self.art)
self.dci.set_articulation_dof_efforts(self.art, effort)
print(effort)
Sorry for the delayed response! I am trying to parse your code. I would recommend against building extensions off of the current BaseSample. Please checkout the extension templater under the Isaac utilities menu, as it’s more clear and easier to modify.
When you run the jetbot does it drive in a tight circle or a broad (large) circle? that is, is the deviation from “forward” severe or minor?
edit: I see where you apply the action now (self.target). I’ll see what it looks like on my side
Thank you for your reply.when I have set the stiffness and damping to zero，and the damping and stiffness in the advanced properties of the revolving joint tab are also zero.Car no longer bounces during simulation.And when I modify the code, the car can also drive at the specified speed. But I have encountered a new problem now. When I imitate the example of Hello Robot, I need to manually modify the damping property in the diver of the car to zero every time before running the simulation. I want to use python.sh to run completely through the code, how to modify it? Below is my later code.
from omni.isaac.examples.base_sample import BaseSample
from omni.isaac.core.utils.types import ArticulationAction
from omni.isaac.core.utils.nucleus import get_assets_root_path
from omni.isaac.core.utils.stage import add_reference_to_stage
from omni.isaac.core.robots import Robot
from omni.isaac.dynamic_control import _dynamic_control
from omni.isaac.core import SimulationContext
from omni.isaac.franka import Franka
import numpy as np
import carb
class HelloWorld(BaseSample):
def init(self) → None:
super().init()
return
def setup_scene(self):
world = self.get_world()
world.scene.add_default_ground_plane()
assets_root_path = get_assets_root_path()
if assets_root_path is None:
carb.log_error("Could not find nucleus server with /Isaac folder")
asset_path = assets_root_path + "/Isaac/Robots/Jetbot/jetbot.usd"
add_reference_to_stage(usd_path=asset_path, prim_path="/World/Fancy_Robot")
jetbot_robot = world.scene.add(Robot(prim_path="/World/Fancy_Robot", name="fancy_robot")
return
async def setup_post_load(self):
self._world = self.get_world()
self._jetbot = self._world.scene.get_object("fancy_robot")
# This is an implicit PD controller of the jetbot/ articulation
# setting PD gains, applying actions, switching control modes..etc.
# can be done through this controller.
# Note: should be only called after the first reset happens to the world
# self._jetbot_articulation_controller = self._jetbot.get_articulation_controller()
dci = _dynamic_control.acquire_dynamic_control_interface()
self.dci = dci
art = dci.get_articulation("/World/Fancy_Robot")
self.art = art
# print position for the degree of freedom
self.target = np.array([15.0,15.0], dtype=np.float32)
self.previous_error = np.zeros_like(self.target)
self.integral = 0
# Adding a physics callback to send the actions to apply actions with every
# physics step executed.
self._world.add_physics_callback("sending_actions", callback_fn=self.step_callback)
return
def control(self,current, dt, Kp=0.005, Ki=0.005, Kd=0):
error = self.target - current
self.integral += error * dt
derivative = (error - self.previous_error) / dt
self.previous_error = error
return Kp * error + Ki * self.integral + Kd * derivative
def step_callback(self,step_size):
# print("simulate with step: ", step_size)
dof_left =self.dci.find_articulation_dof(self.art, "left_wheel_joint")
dof_right =self.dci.find_articulation_dof(self.art, "right_wheel_joint")
dof_state = self.dci.get_dof_state(dof_left,7)
dof_state2 = self.dci.get_dof_state(dof_right,7)
self.dof_states = [dof_state.vel,dof_state2.vel]
print(self.dof_states)
effort = self.control(self.dof_states, step_size)
effort = effort.astype(np.float32)
self.dci.wake_up_articulation(self.art)
self.dci.set_articulation_dof_efforts(self.art, effort)
print(effort)
Another question, I want to add a robotic arm to the car, how to combine them?Thank you very much.@mgussert
I don’t follow. do you mean you have to manually set a damping parameter to zero before pressing play? which parameter in your code? also, I could not load your video, sorry!
Yes, because I want to control the wheels of the car directly through force control, so that the car can move forward. Then I saw someone reply in this post saying “For direct joint effort control, set both drive parameters to zero, and use the Advanced properties of the revolute joint tab to define a stiffness / damping (both values need to be above zero).”Torque Control Model Parameter - #9 by rgasoto. but I want to implement the modification of this parameter directly in the code, so I don’t need to manually modify it every time. And for the video problem, I’m sorry. I’ll try uploading it again. Thank you for your prompt reply @mgussert
Or try to open this link to see if you can see the video, thank you. Isaac sim - YouTube
Why not modify and save the USD directly? Why does it need to be done in code at all? The parameters you want to modify are properties of the joint drive, and are defined as part of the asset.
I think the reason why you need to do it every time you run the sim is because the asset on the stage is configured one way, and that’s what gets simulated when you start. If you modify the parameters before the sim is started, do you still have to set them every time play is called?
I did some tinkering and I found a couple things that we can improve with the asset and the wheeled robot. Consequently, I have a working extension for you that you might find useful!
jetbot_extension.rar (19.7 KB)
To use the extension, unpack it to a directory, and then in Isaac-Sim go to window->Extensions and click on the gear to open settings. Add the path where you unpacked the extension to the list of discoverable paths, and you should be able to “enable” it. click on “jetbot_extension” at the top menu bar and then press the load button. if you press the “run” button you will see the Jetbot drive forward.
some things you should know!
-
this extension was made with the templater tools in the Isaac Utilities menu. There are two files you want to pay attention to, scenario.py and ui_builder.py. In the ui builder we define the function that loads the stage and the assets
_setup_scene. In the scenario.py we define what we DO with the assets, and in this case we set the drive mode to the jetbot to “efforts” and then apply a small action to go forward. -
The call we needed here was the
switch_control_modeof the articulation controller in the robot class. I “cheated” a little bit and referenced a private member of the wheeled robot to get this working. we should probably expose that. -
the jetbot drifts to the right!! why!??!?!? it doesn’t appear to be an asset issue (you can verify this! load the jetbot asset and set the velocity targets on the joint drives to 360 and the jetbot drives forward without drifting). currently investigating
Thank you again for your question and sorry it took so long to get back to you. I hope this helps!
Gus
thanks for your reply.Maybe I don’t know enough about Isaac sim, after I enable it, I can’t open jetbot_extension normally.
For point 3 you mentioned：the jetbot drifts to the right.This is the problem I encountered before. But I also don’t understand why it drifts.Later, the error of the speed is used as the input of the PID control. After the applied force is used as the output, it no longer drifts.
But now when I put an arm on the car and combine the arm with the car and apply force to the wheels, the car doesn’t move at all.I am using the omni.isaac.dynamic_control api.
Thank you for the ideas you gave me on the previous question, that is very useful.@mgussert
I have a local install of ubuntu 20.04 that I installed our latest release on. After extracting the extension and loading it, I am able to run without issue. I also don’t see that viewport error :(
It looks like you extracted and enabled the extension fine. Does the viewport error happen when you enable? I don’t see the jetbot_extension tab anywhere so I’m assuming this error killed it. if you disable autoload, restart isaac sim, and then load the extension (without loading anything else) does this error still occur?
I disable autoload, restart isaac sim, and then load the extension (without loading anything else) this error still occur.Maybe there is something wrong with my previous installation.But this question is not the most important question for me now. I have another problem now, I need to build the manipulator on the car to do a simulation experiment, but when I combine it as mentioned in other posts on the forum,like this Combining UR10 and Smart Transport Robot (STR) to get a mobile manipulator - #3 by Hammad_M result is that the two wheels of the car rotate at the specified speed. However, the car is in the original Man, just like the wheel slipping in real life, I don’t know if I did something wrong combining the two USD files or where, thank you very much and I am very sorry to reply you now. @mgussert
Hi @1365351984 - For any new questions, I would request you to start a new thread instead of combining them into one.
Also, you can close this one if this is no longer an issue.
Ok, thank you
Sorry, there may still be a problem. I uninstalled Isaac sim and reinstalled it, then turned off automatic loading. When it is turned on again, jetbot_extension will be automatically disabled. If I can get your help again, I will be very grateful.@mgussert
Heya!
Sorry for the late reply. I actually have an update for the weird behavior i was seeing in my example extension above. When you are controlling with joint efforts, the distribution of mass matters quite a bit. This is calculated automatically from the mesh geometry and the material properties applied, and as a result, it is slightly off center for the jetbot due to the Jetson card in the mesh. If you go to the physics -> mass section of the properties pane, you can manually set the COM to (0,0,0), and it should drive straight.
you should just need to point at the extension in your new install to enable it. remember, you can do this by going to windows->Extensions->*hamburger settings menu*
please close this topic and make a new one for further questions!