We have a usecase for interactive RL (user input in the training loop). At the moment we currently use ML Agents in Unity3D for the same, but we cannot really scale up the simulations using GPU.
I was just wondering if Gym would be right tool for the job? I was thinking about something on the lines of Unity (as a front end for user interaction, connect to Gym via Python API), and IsaacGym to run the simulations in GPU and send the data back? Any thoughts on this would be really appreciated. I am fairly new to the Nvidia eco system.
Or does Omniverse eco system has a better toolset suited for such usecases? (We aim for the interaction in VR. That means the solution has some compaitlilbyty with OpenVR/XR)