Overall Isaac Sim is incredibly powerful tool, and has many features that makes it significantly faster to iterate with vs other realtime engines (Unity / Unreal). I would say the ability to quickly iterate with python, and ability to combine it with modern ML libraries (PyTorch) is its biggest value moving away from Unity for synthetic data generation and robot simulation.
Feedback:
- Documentation is kinda all over the place now, especially when it comes to RL. What is Issac Orbit vs Isaac Gym vs Omniverse Isaac Gym? Which one we are supposed to use? Documentation for the Jupyter workflow is outdated, and Jupyter live sessions are broken with 2022.2.1.
- Figuring out if something is documented as part of Omniverse documentation, vs Isaac Sim documentation can be tricky.
- There is no way to do RL from image observations currently, as the rendering engine comes to a halt when having more than 10 cameras. This defies the main advantage of using Isaac in the first place: realistic rendering with raytracing for robots to learn proper representations. Without the ability to collect image observations at scale, then what is the point of all those fancy rendering tricks?
- Some critical bugs remain unfixed for long time. E.g Isaac Sim 2022.2.1 is unusable on Linux with GPU driver 535, and some laptops only support this driver version. This issue has been since the start of the year without any hotfix for such critical issue.
Would be really cool if we can get a GPT 4 finetuned on the latest version of the documentation and the Isaac code base, now that finetuning API is available. This would significantly improve the developer onboarding flow IMO.