Originally published at: https://developer.nvidia.com/blog/training-your-jetbot-in-isaac-sim/
How do you teach your JetBot new tricks? In this post, we highlight NVIDIA Isaac Sim simulation and training capabilities by walking you through how to train the JetBot in Isaac Sim with reinforcement learning (RL) and test this trained RL model on NVIDIA Jetson Nano with the real JetBot. Figure 1. Which one is…
Hi there, for training on omniverse, besides the isaac sim, do we need any other libs? i check your post and the source code in github, the readme.txt in the isaccsim_patch says:
Please copy these files over to your Isaac Sim 2020.2 folder before training.
Specifically, syntheticdata.py goes to source/extensions/omni.isaac/synthetic_utils/python/scripts/
The other 4 files go to source/python_samples/jetbot
so where is the omniverse extension, where can we download the extension?
You can register and download Isaac Sim here, as stated in the blog post.
“First, download Isaac Sim. Running Isaac Sim requires the following resources:”
Hi HaiLocLu,
thanks for the reply, I already downloaded the Isaac Sim, and could not find the extensions in this location: extensions/omni.isaac.
Instead , i found out another directory, looks like it is what I want:
isaac-sim/_build/linux-x86_64/release/exts
I will do the sample training later.
Thanks much. You’re right. I was referring to the internal path, while the path for your public build should be in the _build folder. I updated the instructions on my github.
Thanks
maybe gym version error…
how to solve it?
Looks like a gym version error. Which version of stable_baselines did you use? You can also check the Isaac Sim’s python_samples/requirements.txt.
At the time of the blogpost published, it was
python3 -m pip install stable_baselines3==0.8.0
So the versions in Isaac Sim need to match what’s installed on the real JetBot. Thanks.
I am struggling to train using given material from issac sim patch to isaac simulation windows 11. When I trained its saying omnikithelper error
Hiya Naresh!
Thank you for your interest in our blog post and RL in Omniverse with Isaac-sim! Unfortunately, much of the codebase has changed since this post was released and it is no longer compatible with current versions of Isaac-sim. In short, addressing this issue would not only mean maintaining depreciated code, but also maintaining a demonstration of a workflow that may not even be valid anymore.
That said, we are still developing tools for building and running RL environments in Omniverse with Isaac-sim! Keep your eyes peeled for new developments in the near future! ;)
Sorry I couldn’t be of more help. Good luck in your work!
Gus
1 Like
Thanks for the reply, I am happy to collaborate with you guys to make new blog post based on the latest information on the deployment of Deep Reinforcement learning to robotics kit like jetbot
Thanks for the offer, @ac.nareshkumar1993! While Gus might not be able to work on a new post right now (GTC is coming right up!), we’re always willing to get new posts from other people who have success stories or solutions. Let us know if you’d like to submit a post and we’ll work with you on the process.
I am willing to work with you , and write a post soon
Greetings! I found the blog and started to follow along, but ran into some snags. Is it still currently possible to train the Jetbot in Isaac Sim through the Omniverse, or is this no longer supported?
If it is supported, is it possible to do on Windows? I tried placing the patch files in a few different directories but no luck getting Jetson options to show up in the software.
If it’s not supported, is there a roadmap for when support will return? Or alternatives to use? I am looking to teach AI + robotics and would like to leverage simulation to speed up some elements like model training.
Hiya jbflot! welcome to the forums :D
This is a question I get asked a lot! The bad news first: Unfortunately that code is depreciated.
The good news is we have Omni Isaac Gym Environments (OIGE) GitHub - NVIDIA-Omniverse/OmniIsaacGymEnvs: Reinforcement Learning Environments for Omniverse Isaac Gym
Which leverages the RL games library for training, and might meet your immediate teaching needs.
This might actually make another good topic for a live stream :3
Thanks for the response and the link to the Gym Environments. They’re very cool, but after further investigation they seem out-of-reach for the graphics cards in our computers. The Viewport wasn’t loading in Isaac Sim and the console was throwing an error that said I needed to do a clean install of my drivers - I did, but the problem persisted. I have a Quadro T2000 which seems like it’s not supported. Thanks again.
Edit: Maybe update the blog that this is based on, if possible, to reflect the deprecated support? It’s still quite high in search rankings and makes it seem like it’s something supported until you get into the weeds.
You are not the only person who has pointed this out! Thank you! I want to do an updated version of this RL blog post with updated techniques, but modern techniques are very different. There’s a lot to discuss actually… I will at the very least add a depreciation warning at the top of the post