Note: For any Isaac Lab topics, please submit your topic to its GitHub repo (GitHub - isaac-sim/IsaacLab: Unified framework for robot learning built on NVIDIA Isaac Sim) following the instructions provided on Isaac Lab’s Contributing Guidelines (Contribution Guidelines — Isaac Lab Documentation).
Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.
Isaac Sim Version
[v] 4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):
Operating System
[v] Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):
GPU Information
- Model: RTX 4070
- Driver Version: 565.77
Topic Description
Detailed Description
(Describe the issue in detail, including what you were trying to do, what you expected to happen, and what actually happened)
Steps to Reproduce
(Add more steps as needed)
Error Messages
I loaded the TurtleBot3 Burger USD file from Isaac Sim Assets into Isaac Sim, then saved it in my lab workspace for training in Isaac Lab.
To control the two wheels using velocity control, I used DelayedPDActuatorCfg and set the velocity_limit to 6.4. However, as seen in the attached image, the wheel velocity significantly exceeds 6.4.
I tried setting stiffness to 0 and 40.0, but the results remained the same. I am not sure what the issue is.
In fact, to control my own two-wheel robot in Isaac Lab, I converted a URDF file from SolidWorks, then converted it to a USD file using Isaac Sim, and attempted to train it in Isaac Lab, but it did not work properly.
After multiple attempts without success, I tried using the TurtleBot3 Burger instead. However, I discovered that the TurtleBot3 Burger had a similar issue to my robot.
Below is my code.
Screenshots or Videos
Additional Information
What I’ve Tried
I tried changing stiffness to 0 and 40.0, and also adjusted velocity_limit to 0.1, 6.4, and 55.0, but the issue of velocity spikes persisted.
Related Issues
(If you’re aware of any related issues or forum posts, please link them here)
Additional Context
from future import annotations
import torch
import math
from dataclasses import MISSING
import os
import isaaclab.sim as sim_utils
from isaaclab.assets import ArticulationCfg, AssetBaseCfg
from isaaclab.envs import ManagerBasedRLEnvCfg
from isaaclab.managers import CurriculumTermCfg as CurrTerm
from isaaclab.managers import EventTermCfg as EventTerm
from isaaclab.managers import ObservationGroupCfg as ObsGroup
from isaaclab.managers import ObservationTermCfg as ObsTerm
from isaaclab.managers import RewardTermCfg as RewTerm
from isaaclab.managers import SceneEntityCfg
from isaaclab.managers import TerminationTermCfg as DoneTerm
from isaaclab.scene import InteractiveSceneCfg
from isaaclab.sensors import ContactSensorCfg, RayCasterCfg, patterns
from isaaclab.terrains import TerrainImporterCfg
from isaaclab.utils import configclass
from isaaclab.utils.noise import AdditiveUniformNoiseCfg as Unoise
import isaaclab_tasks.manager_based.locomotion.velocity.mdp as mdp
from isaaclab.terrains.config.rough import ROUGH_TERRAINS_CFG
from isaaclab.actuators import (
ImplicitActuatorCfg,
DCMotorCfg,
IdealPDActuatorCfg,
DelayedPDActuatorCfg,
)
TWOWHEEL_CFG = ArticulationCfg(
spawn=sim_utils.UsdFileCfg(
usd_path=os.environ[‘HOME’] + “/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/manager_based/locomotion/velocity/config/two_wheel/two_wheel_turtle.usd”,
activate_contact_sensors=True,
rigid_props=sim_utils.RigidBodyPropertiesCfg(
disable_gravity=False,
retain_accelerations=False,
linear_damping=0.0,
angular_damping=0.0,
max_linear_velocity=1000.0,
max_angular_velocity=1000.0,
max_depenetration_velocity=1.0,
),
articulation_props=sim_utils.ArticulationRootPropertiesCfg(
enabled_self_collisions=False, solver_position_iteration_count=4, solver_velocity_iteration_count=0
),
),
init_state=ArticulationCfg.InitialStateCfg(
pos=(0.0, 0.0, 0.06082), # default: 0.0 0.0 0.2461942 0.33649
# pos=(0.0, 0.0, 0.33649),
joint_pos={
“wheel_left_joint”: 0.0,
“wheel_right_joint”: 0.0,
},
joint_vel={“.“: 0.0},
),
soft_joint_pos_limit_factor=0.8,
actuators={
“joints”: DelayedPDActuatorCfg(
joint_names_expr=[”.”],
effort_limit=1.0,
velocity_limit=6.4,
min_delay=0, # physics time steps (min: 5.0 * 0 = 0.0ms)
max_delay=1, # physics time steps (max: 5.0 * 4 = 20.0ms)
stiffness={
“wheel_left_joint”: 0.0,
“wheel_right_joint”: 0.0,
},
damping={
“wheel_left_joint”: 0.3,
“wheel_right_joint”: 0.3,
},
friction={
“wheel_left_joint”: 0.0,
“wheel_right_joint”: 0.0,
},
armature={
“wheel_left_joint”: 0.0,
“wheel_right_joint”: 0.0,
},
),
# # “joints”: FLAMINGO_JOINT_ACTUATOR_MLP_CFG,
# # “wheels”: FLAMINGO_WHEEL_ACTUATOR_LSTM_CFG,
},
)
@configclass
class MySceneCfg(InteractiveSceneCfg):
“”“Configuration for the terrain scene with a legged robot.”“”
terrain = TerrainImporterCfg(
prim_path="/World/ground",
terrain_type="generator", # terrain_type="generator" -> 자동으로 지형을 생성
terrain_generator=ROUGH_TERRAINS_CFG, # ROUGH_TERRAINS_CFG -> 거친 지형을 사용 (공식 제공하는 설정)
max_init_terrain_level=5, # 초기 지형 레벨
collision_group=-1, # -1은 모든 물체와 충돌 허용
physics_material=sim_utils.RigidBodyMaterialCfg(
friction_combine_mode="multiply", #"multiply": 두 개의 마찰 계수를 곱해서 최종 마찰력을 계산.
restitution_combine_mode="multiply",#충돌 반발력(탄성 계수) 계산 방식.#"multiply": 두 물체의 반발 계수를 곱해서 최종 반발력을 결정.
static_friction=1.0,#정지 마찰 계수 (static friction): 로봇이 멈춰있을 때 최대한 버틸 수 있는 마찰력.
dynamic_friction=1.0,#동적 마찰 계수 (dynamic friction): 움직일 때 적용되는 마찰력.
),
visual_material=sim_utils.MdlFileCfg(
mdl_path="{NVIDIA_NUCLEUS_DIR}/Materials/Base/Architecture/Shingles_01.mdl",
project_uvw=True,
),
debug_vis=False,
)
robot: ArticulationCfg = MISSING ## robot: ArticulationCfg = MISSING → 로봇 모델은 외부에서 지정해야 함.
# RayCastercfg : 시뮬레이션에서 사용 되는 가상의 레이저 센서.
height_scanner = RayCasterCfg(
prim_path="{ENV_REGEX_NS}/Robot/base_footprint",
offset=RayCasterCfg.OffsetCfg(pos=(0.0, 0.0, 20.0)),
attach_yaw_only=True,
pattern_cfg=patterns.GridPatternCfg(resolution=0.1, size=[1.6, 1.0]),
debug_vis=False,
mesh_prim_paths=["/World/ground"],
)
contact_forces = ContactSensorCfg(prim_path="{ENV_REGEX_NS}/Robot/.*", history_length=3, track_air_time=True)
light = AssetBaseCfg(
prim_path="/World/light",
spawn=sim_utils.DistantLightCfg(color=(0.75, 0.75, 0.75), intensity=3000.0),
)
sky_light = AssetBaseCfg(
prim_path="/World/skyLight",
spawn=sim_utils.DomeLightCfg(color=(0.53, 0.81, 0.98), intensity=1500.0),
)
@configclass
class CommandsCfg:
“”“Command specifications for the MDP.”“”
base_velocity = mdp.UniformVelocityCommandCfg(
asset_name="robot",
resampling_time_range=(10.0, 10.0),
rel_standing_envs=0.02,
rel_heading_envs=0.0,
debug_vis=True,
ranges=mdp.UniformVelocityCommandCfg.Ranges(
lin_vel_x=(-0.2, 0.2), lin_vel_y=(0.0, 0.0), ang_vel_z=(-0.5, 0.5)
),
)
@configclass
class ActionsCfg:
“”“Action specifications for the MDP.”“”
wheel_vel = mdp.JointVelocityActionCfg(
asset_name="robot",
joint_names=["wheel_left_joint","wheel_right_joint"],
scale=6.4,
use_default_offset=True,
)
# action은 -1~1의 범위로 들어옴
# 위치제어 같은 경우는 weight가 1이므로 -1~1 rad 범위내로 상대적인 위치제어를 할 것이고
# ( 현재위치에서 action * scale 만큼 이동)
# 바퀴같은경우는 -55~55 rad/s 범위로 절대적인 속도제어. -55~55 rad/s 내에서만 속도값 얻음.
# (최종 속도는 action * scale)
@configclass
class ObservationsCfg:
“”“Observation specifications for the MDP.”“”
# PolicyCfg는 관찰값을 그룹화하는 역할. ObsGroup을 상속받아 Obsterm을 관리
@configclass
class PolicyCfg(ObsGroup):
"""Observations for policy group."""
# Stack 0 : 현재 상태를 나타내는 관찰값
"""
Stack 0
"""
joint_pos = ObsTerm(
func=mdp.joint_pos_rel, # 현재위치 - 기본위치(init dof pos)
noise=Unoise(n_min=-0.04, n_max=0.04), # # 관절 위치에 -0.04 rad ~ 0.04 rad 범위의 작은 노이즈 추가
params={"asset_cfg": SceneEntityCfg("robot", joint_names=[".*"])},
)
# params가 없으니 모든 joint의 속도를 가져옴.
joint_vel = ObsTerm(func=mdp.joint_vel_rel, noise=Unoise(n_min=-1.5, n_max=1.5))
# base의 각속도
base_ang_vel = ObsTerm(func=mdp.base_ang_vel, noise=Unoise(n_min=-0.15, n_max=0.15))
# base의 오일러 각도 x,y,z 즉 roll, pitch, yaw
base_euler = ObsTerm(func=mdp.root_euler_angle, noise=Unoise(n_min=-0.125, n_max=0.125))
# 이전 스텝에서 에이전트가 취한 행동을 기록
actions = ObsTerm(func=mdp.last_action)
# base의 linear velocty
# Stack 1 : 이전 상태를 저장
"""
Stack 1
"""
prev_joint_pos = joint_pos
prev_joint_vel = joint_vel
prev_base_ang_vel = base_ang_vel
prev_base_euler = base_euler
prev_actions = actions
# 로봇이 따라가야 하는 목표 속도(명령)을 관찰값으로 포함
velocity_commands = ObsTerm(func=mdp.generated_commands, params={"command_name": "base_velocity"})
def __post_init__(self):
self.enable_corruption = True # 센서 데이터에 추가적인 노이즈를 넣어 학습을 더욱 일반화
self.concatenate_terms = True # 관찰값들을 하나의 벡터로 합쳐서 사용
policy: PolicyCfg = PolicyCfg()
@configclass
class EventCfg:
“”“Configuration for events.”“”
# startup
physics_material = EventTerm(
func=mdp.randomize_rigid_body_material,
mode="startup",
params={
"asset_cfg": SceneEntityCfg("robot", body_names=".*"),
"static_friction_range": (0.8, 1.0),# 정지 마찰 계수를 0.8 ~ 1.0 범위에서 랜덤하게 결정
"dynamic_friction_range": (0.6, 0.8),# 움직이는 동안의 마찰 계수를 0.6~0.8 범위에서 랜덤하게 설정
"restitution_range": (0.0, 0.0), # 반발 계수(충돌 시 튕기는 정도)를 0으로 설정
"num_buckets": 64,# 랜덤 샘플링을 위해 64개 버킷(bucket)으로 나누어 랜덤 값 생성
},
)
add_base_mass = EventTerm(
func=mdp.randomize_rigid_body_mass,# 로봇의 기본 질량을 랜덤하게 변형하여 학습이 다양한 조건에서도 가능하도록 함.
mode="startup",
params={
"asset_cfg": SceneEntityCfg("robot", body_names="base_footprint"),
"mass_distribution_params": (-0.1, 0.1),# base의 기본 질량에서 -2.5 ~ +2.5 kg 만큼 랜덤하게 변경
"operation": "add",# 현재 질량에 추가하는 방식으로 적용
},
)
# reset
base_external_force_torque = EventTerm(
func=mdp.apply_external_force_torque,# 로봇이 초기화 될 때, 외부에서 힘이나 토크를 적용할 수 있도록 설정
mode="reset",
params={
"asset_cfg": SceneEntityCfg("robot", body_names="base"),
"force_range": (0.0, 0.0),
"torque_range": (-0.0, 0.0),
},
# 이 함수는 "힘(force)과 회전력(torque)을 직접 적용"하는 방식.
# "힘(force)" → 선형 이동을 유발 (로봇이 특정 방향으로 밀려남). 랜덤한 힘이 로봇의 베이스에 작용
# "토크(torque)" → 회전 운동을 유발 (로봇이 회전하려는 힘을 받음).
)
reset_base = EventTerm(
func=mdp.reset_root_state_uniform, # 로봇이 매 에피소드 시작 시 조금씩 다른 위치와 속도를 가지도록 초기화
mode="reset",
params={
"pose_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5), "yaw": (-3.14, 3.14)},
"velocity_range": {
"x": (-0.0, 0.0),
"y": (-0.0, 0.0),
"z": (-0.0, 0.0),
"roll": (-0.0, 0.0),
"pitch": (-0.0, 0.0),
"yaw": (-0.0, 0.0),
},
},
)
reset_robot_joints = EventTerm(
func=mdp.reset_joints_by_offset,# 로봇의 관절을 초기화 할 때, 일정 범위 내에서 랜덤한 값으로 설정
mode="reset",
params={ # # 기본적으로는 정해진 초기 상태에서 시작
"position_range": (0.0, 0.0),
"velocity_range": (0.0, 0.0)},
)
# interval
push_robot = EventTerm(
func=mdp.push_by_setting_velocity,
mode="interval",
interval_range_s=(10.0, 15.0), # 10 ~ 15초 사이의 랜덤한 시점에 이벤트 발생
params={"velocity_range": {"x": (-0.0, 0.0), "y": (-0.0, 0.0)}},
)
# 이 함수는 "속도를 변경해서 로봇을 미는 방식"으로 힘을 적용!
# 즉, 특정 순간에 velocity_range 값만큼 로봇이 순간적으로 이동하도록 함.
@configclass
class RewardsCfg:
“”“Reward terms for the MDP.”“”
# – task
# 목표 속도를 잘 따라가면 보상
track_lin_vel_xy_exp = RewTerm(
func=mdp.track_lin_vel_xy_exp, weight=2.0, params={“command_name”: “base_velocity”, “std”: math.sqrt(0.25)}
)# base_velocity 값과의 차이를 계산하여 보상을 부여
# “std”: math.sqrt(0.25) : 속도 오차가 0.5 m/s 이하일 때 보상이 주어질 확률을 높이는 역할
track_ang_vel_z_exp = RewTerm(
func=mdp.track_ang_vel_z_exp, weight=1.0, params={“command_name”: “base_velocity”, “std”: math.sqrt(0.25)})
# -- penalties
# XY 평면에서 불필요한 회전 방지 : Roll, Pitch 축 회전 제한
# Roll : 좌우로 기울어짐 (구르기), Pitch : 앞뒤로 기울어짐 (고개 끄덕이기), Yaw : 좌우 회전
# ang_vel_xy_l2 = RewTerm(func=mdp.ang_vel_xy_l2, weight=-0.05)
ang_vel_xy_l2 = RewTerm(func=mdp.ang_vel_xy_l2, weight=-3.0)
# 과도한 힘 사용 제한 (토크 패널티)
# dof_torques_l2 = RewTerm(func=mdp.joint_torques_l2, weight=-5.0e-6)
# joint_applied_torque_limits = RewTerm(
# func=mdp.applied_torque_limits,
# weight=0.0, # default: -0.1
# params={"asset_cfg": SceneEntityCfg("robot", joint_names=".*_joint")},
# )
# 관절의 가속도(움직임 변화율)가 너무 크면 패널티를 줌.
dof_acc_l2 = RewTerm(func=mdp.joint_acc_l2, weight=-2.5e-7) # default: -2.5e-7
termination_penalty = RewTerm(func=mdp.is_terminated, weight=-100.0)
# stand_origin_still = RewTerm(
# func=mdp.stand_origin_base,
# weight=-0.1, # default: -0.1
# params={
# "command_name": "base_velocity",
# "asset_cfg": SceneEntityCfg("robot", body_names="base_footprint"),
# },
# )
flat_orientation_l2 = RewTerm(func=mdp.flat_orientation_l2, weight=-5.0)
@configclass
class TerminationsCfg:
“”“Termination terms for the MDP.”“”
time_out = DoneTerm(func=mdp.time_out, time_out=True)
base_tilt = DoneTerm(
func=mdp.bad_orientation,
params={
"limit_angle": 1.0472, # 60도를 라디안으로 변환
"asset_cfg": SceneEntityCfg("robot", body_names="base_footprint"), # 'base_footprint' 대신 'robot' 사용
},
)
@configclass
class CurriculumCfg:
“”“Curriculum terms for the MDP.”“”
# curriculum_dof_torques = CurrTerm(
# func=mdp.modify_reward_weight,
# params={
# "term_name": "joint_applied_torque_limits",
# "weight": -0.01, # 목표 최종 weight
# "num_steps": 500000 # 50만 스텝 이후 적용
# }
# )
@configclass
class TwowheelCfg(ManagerBasedRLEnvCfg):
“”“Configuration for the locomotion velocity-tracking environment.”“”
# Scene settings
scene: MySceneCfg = MySceneCfg(num_envs=4096, env_spacing=1.5)
# Basic settings
observations: ObservationsCfg = ObservationsCfg()
actions: ActionsCfg = ActionsCfg()
commands: CommandsCfg = CommandsCfg()
# MDP settings
rewards: RewardsCfg = RewardsCfg()
terminations: TerminationsCfg = TerminationsCfg()
events: EventCfg = EventCfg()
curriculum: CurriculumCfg = CurriculumCfg()
# 환경이 초기화 될 때 실행, 환경이 생성된 후(__init__이 실행된 후) 추가 설정을 수행하는 역할.
def __post_init__(self):
# general settings 일반 환경설정
self.decimation = 4 # 물리 시뮬레이션을 몇 번 실행한 후 정책이 업데이트될지 설정. 4로 설정했으므로 4번의 물리 스텝 후 한 번 강화학습 업데이트 진행.
self.episode_length_s = 20.0 # 각 에피소드의 길이를 20초로 설정.
# simulation settings 물리 엔진 설정
self.sim.dt = 0.005 # 시뮬레이션 타임스텝을 5ms(0.005초)로 설정.
self.sim.disable_contact_processing = True # 충돌 처리를 비활성화하여 계산 부하 감소.
self.sim.physics_material = self.scene.terrain.physics_material# 지형(terrain)의 물리적 특성(마찰, 반발력 등) 설정.
if self.scene.height_scanner is not None:
self.scene.height_scanner.update_period = self.decimation * self.sim.dt
if self.scene.contact_forces is not None:
self.scene.contact_forces.update_period = self.sim.dt
# check if terrain levels curriculum is enabled - if so, enable curriculum for terrain generator
# this generates terrains with increasing difficulty and is useful for training
# self.curriculum.terrain_levels이 설정되어 있다면, 지형 난이도를 점진적으로 증가시키는 커리큘럼 학습을 활성화.
# 그렇지 않다면 기본적으로 평지에서 학습.
if getattr(self.curriculum, "terrain_levels", None) is not None:
if self.scene.terrain.terrain_generator is not None:
self.scene.terrain.terrain_generator.curriculum = True
else:
if self.scene.terrain.terrain_generator is not None:
self.scene.terrain.terrain_generator.curriculum = False
#-------------------------------------------------------------------------------#
# scene
self.scene.robot = TWOWHEEL_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
self.scene.height_scanner.prim_path = "{ENV_REGEX_NS}/Robot/base_footprint"
self.observations.policy.enable_corruption = True
# reset_robot_joint_zero should be called here
self.events.push_robot = None
# self.events.push_robot.interval_range_s = (12.0, 15.0)
# self.events.push_robot.params = {
# "velocity_range": {"x": (-1.0, 1.0), "y": (-1.0, 1.0)},
# }
# add base mass should be called here
self.events.add_base_mass.params["asset_cfg"].body_names = ["base_footprint"]
self.events.add_base_mass.params["mass_distribution_params"] = (-0.1, 0.1)
# physics material should be called here
self.events.physics_material.params["asset_cfg"].body_names = [".*"]
self.events.physics_material.params["static_friction_range"] = (0.4, 1.0)
self.events.physics_material.params["dynamic_friction_range"] = (0.4, 0.8)
self.events.base_external_force_torque.params["asset_cfg"].body_names = ["base_footprint"]
self.events.reset_base.params = {
"pose_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5), "yaw": (-3.14, 3.14)},
"velocity_range": {
"x": (0.0, 0.0),
"y": (0.0, 0.0),
"z": (0.0, 0.0),
"roll": (0.0, 0.0),
"pitch": (0.0, 0.0),
"yaw": (-0.0, 0.0),
},
}
# rewards
# self.rewards.action_rate_l2.weight *= 2.5 # default: 1.5
# self.rewards.dof_acc_l2.weight *= 2.5 # default: 1.5
# change terrain to flat
self.scene.terrain.terrain_type = "plane"
self.scene.terrain.terrain_generator = None
# Terrain curriculum
self.curriculum.terrain_levels = None
# height scan
self.scene.height_scanner.prim_path = "{ENV_REGEX_NS}/Robot/base_footprint"
self.scene.height_scanner.debug_vis = False
# commands
self.commands.base_velocity.ranges.lin_vel_x = (-0.2, 0.2)
self.commands.base_velocity.ranges.lin_vel_y = (0.0, 0.0)
self.commands.base_velocity.ranges.ang_vel_z = (-0.5, 0.5)