Deep reinforce learning with JetBot and JetRacer

Robotics & Edge Computing Jetson Systems Jetson Projects
1

I would share my this project related to JetBot and JetRacer. My Project is learning_racer that can AI agent using deep reinforcement learning.

This software can learning policy of driving JetBot or JetRacer without human teaching(unsupervised learning) in 10 to 15 minutes.

This agent can run on the Jetson Nano. Why can run on Jetson Nano and short learning time? because using integrate of SAC[soft actor critic] and VAE. SAC is a state of the art off-policy reinforcement learning method.
In addition VAE train on cloud server beforehand as CNN layer of SAC.(This method called state representation learning) .

Detail of SAC here:

This implementation is based upon greate work of Antonin RAFFIN:
https://towardsdatascience.com/learning-to-drive-smoothly-in-minutes-450a7cdb35f4

you can setup the software to JetBot(JetPack4.2<=) and JetRacer(JetCard based) using install shell easily. likely:

$ cd ~/ && git clone https://github.com/masato-ka/airc-rl-agent.git
$ cd airc-rl-agent
$ sh install_jetpack.sh

My project page in here :

You can see demo in here:

2

Hi @masato-ka, great project and add-on for JetBot/JetRacer! Thanks for sharing.

3

I’m having trouble getting this to work. After adding in all of the packages I was missing, I was finally able to create the vae.torch file.

Now I’m trying to run the “user_interface_without_gamepad.ipynb” file. Under the “Show toggle button”, it says that “Status” is “invalid”.

Screenshot 2023-06-06 at 9.29.44 PM
Screenshot 2023-06-06 at 9.29.44 PM743×262 20.5 KB

The next step says:

Run train.py

$ racer train -robot jetbot # If you use on JetRacer, "-robot jetracer". default is jetbot.

This is the output:

root@nano-4gb-jp45:/workspace/airc-rl-agent# racer train -robot jetbot -vae notebooks/colabo/vae.torch
nvbuf_utils: Could not get EGL display connection
2023-06-07 01:33:54,400 - learning_racer.racer - INFO - Start learning racer :1.7.1
2023-06-07 01:33:54,401 - learning_racer.commands.subcommand - INFO - Start training
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 3 
   Output Stream W = 1640 H = 1232 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
/usr/local/lib/python3.6/dist-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
2023-06-07 01:34:10,327 - learning_racer.commands.subcommand - ERROR - :load_class:module 'learning_racer.teleoperate' has no attribute 'Teleoperation'

Am I missing something?

4

Figured it out. In “/airc-rl-agent/config.yml” there’s a typo:

ENV_CONFIG:
  jetbot:
    robot_name: 'jetbot-v0'
    wrapped_env: 'learning_racer.agent.teleoperation:TeleoperationEnv'
    parts:
      teleoperator: 'learning_racer.teleoperate:Teleoperation'

The last line up there should be:

teleoperator: 'learning_racer.teleoperate:Teleoperator'