Autonomous vehicle simulation in Isaac

After a bit of research, it seems that Isaac Sim is designed primarily for simulating indoor robots operating in highly-structured, controlled and relatively small environments on smooth surfaces. In other words, it doesn’t address more interesting vehicle dynamics and real-world effect that are desirable for simulating autonomous vehicles operating on and off road (which is what I’m interested in). Is this an accurate observation? What would you say are the main limitations that would prevent me from using Isaac to simulate an autonomous car with sensors driving around an urban environment? Thanks!
P.S. I know about DriveSim, but have not been able to get into the early access program.

Hi @user3265 - Your observation is partly correct. Isaac Sim is primarily tailor-made for simulating indoor robots operating in structured, controlled environments on smooth surfaces. However, it may not fully cater to the dynamic and real-world effects needed for simulating autonomous vehicles on and off the road.

The main limitations could include, but are not limited to, the following:

  1. Lack of diverse environmental simulation: Isaac Sim might not provide realistic and diverse outdoor scenarios such as traffic conditions, changing weather and lighting, or varied road types which are vital for autonomous vehicle simulation.

  2. Limited Vehicle Dynamics: It possibly lacks the depth for simulating complex vehicle dynamics, an essential aspect for simulating an autonomous car.

  3. Sensor limitations: Isaac Sim might not fully support the wide range of sensors used in an autonomous vehicle, including LiDARs, radars, and multiple cameras.

  4. Scalability: Simulating a large, complex outdoor environment like an urban area might be challenging.

This response sounds auto-generated and doesn’t provide much detail. Are there actual user/developers who could shed more light on this?

1 Like

@user3265 - I am not sure what answer you are looking from developer?

Did you get a chance to review the Isaac Sim documentation? if yes, can you let us know what components you think will be useful for autonomous cars? did you try any of that? please share the issues you are facing in using those components, then I can assign it to the right dev to help you on it.

Looking for a more detailed technical explanation. Your first answer sounds very unsure of itself: “might not provide”, “possibly lacks”, “might not fully support”. These are all straightforward features - they either exist or they don’t. Let me be more specific - what would prevent me from simulating a regular car in Isaac? Can I model the drivetrain? What vehicle-terrain interaction models are in place? What do collision models look like?

1 Like