Recommendations for Developing a Web-Based Robotic System with Isaac Sim and ROS

Omniverse Isaac Sim
1

Hello NVIDIA Community,

I plan to develop a web-based robotic system control and simulation application. I aim to integrate NVIDIA’s Isaac Sim with either ROS 1 or ROS 2 to create an interactive and accessible platform for robotic operations. I have CAD files of my own developed robot. I would appreciate any guidance or recommendations on the following:

  1. Are there specific web frameworks or tools you recommend for interfacing Isaac Sim and ROS with a web application? I am looking for robust solutions that can support real-time interactions and simulations.
  2. Can anyone share examples or resources where Isaac Sim and ROS have been successfully used in web-based applications? Learning from existing projects could greatly accelerate our development process.
    I found this on tech blog : Develop a Multi-Robot Environment with NVIDIA Isaac Sim, ROS, and Nimbus | NVIDIA Technical Blog
    in this, they use Nimbus.

I am excited to embark on this project and greatly value the community’s expertise and recommendations. Thank you in advance for your help and support!

2

Hello jjjenny!

I don’t know about specific web frameworks, and whatever you end up doing depends on how integrated this application is going to be with Omniverse and Isaac-sim in the end, but I can offer my thoughts. Please keep in mind that I have very little experience with ROS, and practically no experience with real robots, so take my ideas with a grain of salt!

not knowing more about your specific application, you may find the following tools useful

Isaac-sim comes equipped with “bridges” for ROS and ROS2, allowing the simulator to interface with those messaging systems https://docs.omniverse.nvidia.com/isaacsim/latest/installation/install_ros.html. For a list of specific nodes we provide, checkout ROS2 Bridge [omni.isaac.ros2_bridge] — isaac_sim 2023.1.1-rc.8 documentation.

Isaac Mission Dispactch is a cloud service that facilitates the communication between and management of a fleet of robots GitHub - nvidia-isaac/isaac_mission_dispatch: VDA5050-compatible cloud service for fleet mission dispatch, GitHub - NVIDIA-ISAAC-ROS/isaac_ros_mission_client: VDA5050-compatible mission controller

Isaac ROS NITROS lets you keep certain data on the GPU for processing while allowing the rest of the ROS network to proceed ahead, only fetching the data from the device when it is needed on the host. NITROS — isaac_ros_docs documentation GitHub - NVIDIA-ISAAC-ROS/isaac_ros_nitros: NVIDIA Isaac Transport for ROS package for hardware-acceleration friendly movement of messages

I hope this helps! I’m also interested to see what web frameworks other people suggest :3

Gus

1 Like
4

I appreciate your insights and suggestions a lot!! Maybe the Isaac Missin Dispatch is what I’m looking for, and other sources are beneficial.

I also have a few follow-up questions about your answer.

  1. The web application which I want to make is the GUI to control the robots. I checked the Mission Dispatch briefly, and as far as I understand, it’s a cloud API service to connect robots and (other) cloud services. So, if I want to control robots via GUI, can I use the general cloud service+general web GUI (made with JavaScript, etc.) and then connect it with robots via Mission Dispatch?

  2. If you are familiar with cloud robotics, May I ask about some cloud services that are generally used for robotics? (mentioned above)

Thanks again for your kind answer! Hope you have a nice day

5

to be clear, you can ask whatever you like! :D

Maybe the solution here is to build a stand alone app or an extension of isaac sim to exploit the GUI functionality of omniverse kit, and then stream that result through the web browser to create the “web app”? Livestream Clients — Omniverse IsaacSim latest documentation

https://docs.omniverse.nvidia.com/streaming-client/latest/user-manual.html

6

Sounds like an exciting project! You might want to explore using ROS Bridge Suite with WebSockets for real-time communication and tools like Node.js or Flask for the web backend—Nimbus is a good reference too. Good luck!