Custom Dynamics (Warp) for 14-DOF Vehicle Model in Large-Scale Parallel RL

Omniverse Isaac Sim
1

6.0.0
5.1.0
5.0.0
4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 24.04
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

GPU Information

  • Model: 4x RTX TITAN
  • Driver Version: 580

Topic Description

Detailed Description

Hello Isaac Sim Community,

I hope you are having a great day! I am currently working on a large-scale parallel Reinforcement Learning (RL) project involving a 14-DOF vehicle model.

Instead of using the built-in PhysX engine for physics simulation, I would like to implement my own custom vehicle dynamics. My goal is to use Isaac Sim strictly for visualization and managing the parallelized environments, while the actual physics calculations are driven by my custom Python code.

I have a few questions regarding the best practices for this architecture:

  1. Kinematic Setup for Custom Dynamics: To achieve this, should I configure the vehicle’s rigid bodies/articulations as kinematic and manually update their positions and velocities at every simulation step? Is this the recommended approach when bypassing PhysX?

  2. NVIDIA Warp Integration: I am considering writing the custom dynamics using NVIDIA Warp to ensure high performance. Is a Warp-based custom dynamics approach well-suited and configurable for large-scale parallel RL training within Isaac Sim/Isaac Lab?

  3. Documentation and Examples: Are there any official documents, tutorials, or reference examples that cover setting up custom dynamics (especially using Warp) for parallel RL environments?

Any guidance, advice, or pointers to relevant resources would be greatly appreciated. Thank you so much for your time and support!Additional Information

I am currently reviewing the documentation on Kinematic state settings and NVIDIA Warp, but I want to ensure this is the right architectural direction for large-scale RL before fully committing to the implementation.

Best,

2

Hi Hoeunsu3,

Great questions – your architectural instincts are solid. This is a well-supported pattern in Isaac Sim/Isaac Lab. Let me address each one:

1. Kinematic Setup for Custom Dynamics

Yes, setting rigid bodies as kinematic (physics:kinematicEnabled = true) and manually writing poses each step is a valid approach when bypassing PhysX. However, there’s now a better option – see point 3 below.

2. NVIDIA Warp Integration – Excellent Fit

Warp is ideal for this:

  • GPU-native parallelism: A single wp.launch(dim=N) processes all N environments in one kernel call
  • JIT compiled: Python → CUDA with no C++ required
  • Differentiable: wp.Tape supports automatic differentiation if you later want gradient-based optimization
  • CUDA graph capture: Eliminates kernel launch overhead at scale

Your 14-DOF state vector maps cleanly to wp.array(dtype=float32, ndim=2) of shape (num_envs, 14), with each kernel thread computing dynamics for one environment.

3. Documentation, Examples & Recommended Path

Here’s the good news: Isaac Lab is actively building first-class Warp environment support. Rather than manually managing kinematic bodies + USD pose writes, I’d recommend looking at the new Warp env infrastructure on the Isaac Lab develop branch:

DirectRLEnvWarp base class (merged into develop)

  • PR: #4905 – Cherry-pick direct warp envs from dev/newton
  • Adds DirectRLEnvWarp with CUDA graph capture, InteractiveSceneWarp with warp-native resets, and reference environments (Cartpole, Ant, Humanoid)
  • This is likely what you want to subclass – implement your 14-DOF vehicle dynamics as @wp.kernel functions, and you get parallel env management + CUDA graph capture for free

Manager-based Warp envs (in progress on develop)

For deeper implementation questions or issues with these Warp env classes, the Isaac Lab GitHub repo is the best place to follow up – the developers working on the Warp infrastructure are active there.

Warp documentation

Recommended Architecture

  1. Subclass DirectRLEnvWarp from isaaclab_experimental (on the develop branch)
  2. Implement your 14-DOF dynamics as Warp kernels (tire model, suspension, steering geometry, etc.)
  3. Let Isaac Lab handle environment cloning, resets, observation/reward dispatch, and RL trainer integration
  4. For training only (no visualization needed): you could even skip Isaac Sim entirely and run pure Warp + a Gymnasium wrapper – much faster iteration

Skeleton Warp kernel for your use case:

import warp as wp

@wp.kernel
def vehicle_dynamics_step(
    state: wp.array(dtype=wp.float32, ndim=2),       # (N, 14)
    action: wp.array(dtype=wp.float32, ndim=2),      # (N, action_dim)
    next_state: wp.array(dtype=wp.float32, ndim=2),  # (N, 14)
    dt: float,
):
    env_id = wp.tid()

    # Read current state for this environment
    x     = state[env_id, 0]
    y     = state[env_id, 1]
    z     = state[env_id, 2]
    roll  = state[env_id, 3]
    pitch = state[env_id, 4]
    yaw   = state[env_id, 5]
    # wheel_angles[4]: state[env_id, 6:10]
    # suspension[4]:   state[env_id, 10:14]

    # Read actions (e.g., steering, throttle)
    steer    = action[env_id, 0]
    throttle = action[env_id, 1]

    # Your 14-DOF dynamics equations here
    # (bicycle model, Pacejka tires, spring-damper suspension, etc.)

    # Write next state
    next_state[env_id, 0] = x + vx * dt
    # ... all 14 DOFs

# Launch for all environments in one call
wp.launch(vehicle_dynamics_step, dim=num_envs,
          inputs=[state_buf, action_buf, next_state_buf, 0.01],
          device="cuda")

With 4x RTX TITANs you should be able to run thousands of parallel environments comfortably.

Hope this helps – feel free to follow up here for Isaac Sim questions, or on the Isaac Lab GitHub for Warp env implementation details!

3

thank you for your response
it helps a lot!!