Discovered a major issue with the Virtual Gamepad/Keypad (VGP/VKP). If there is a connectivity issue and the front-end loses connection with the backend while the VGP/VKP is in use, the last pressed key value is retained and keeps executing this last command in a loop even when disconnected from backend. I connected the
request from the virtual gamepad bridge to our joystick codelet. And printed out the Json messages being received from the VGP/VKP in Isaac Sight.
fig 1: Channel initialization to receive data from Nvidia Sight VGP/VKG
fig 2: Receiving data from VGP/VKG into codelet
fig 3: source and target channels of the data being received from sight
fig 4: Messages before connecting to VGP/VKP
fig 5 : Messages after connecting to Sight. On disconnecting from sight, the messages remain the same.
fig 7: No activity initialization of VKP on connection being established with backend for the first time.
fig 8: VKP Failsafe is present only if the connection is established. If the connection is lost the return causes the last keypress to be retained.
This poses a huge safety hazard for our robots. While testing our robot, we were moving it forward by pressing ‘w’ on the VKP during which we lost connection with the backend. The robot kept moving forward instead of stopping when the connection was lost. We cannot solve this issue without the assistance of NVIDIA especially since we don’t have edit permissions to these scripts unless we fork them and even then, we cannot get the Sight front-end built. Kindly assist us in solving this issue.
For your information, we have created our entire navigation stack using the NVIDIA Isaac SDK. This problem in the Virtual Gamepad/Keypad (VGP/VKP) feature is blocked for us on our project. Any assistance that we could get will be greatly appreciated. To virtually control our robot using VKP, we followed this page in the NVIDIA Isaac(2020.2) documentation.
Please advise. Thanks!