Stable Baselines example doesn't work as intended

Hello everyone,

I am currently trying out the Stable Baselines example by simply running But even after 500k steps, the mean reward is somewhere around 0. Inspecting the policy by running shows, that the robot simply rotates on the spot. It feels like he doesn’t get the visual input. Looking back at the logs, I recognized the following warnings

[Warning] [] Different GPUs have different availalbe VRAM. The texture streaming system will be limited to the lower VRAM number
[Warning] [] Requesting texture to use texture streaming, but the service isn't available.The texture will be created as a regular resource.

Has anyone encountered the same problem or warnings? I am using a RTX 2070 super as the only GPU in my setup and my driver version is 470.86. Thanks in advance :)

Hi user119144,

Are you still facing this problem? If so, can you elaborate on what you mean with “not getting the visual input”. Additionally, if you can post a video of the inference it will be helpful in identifying the problem.

I had the same problem.

It was not the problem of not getting correct visual input. I had the same warning as you showed, but I printed the input to the robot frame-by-frame and they were all good.

I solved the issue by tuning the parameter “gamma” in PPO to a smaller value (0.9 in my case), although the value is still 0.995 in the tutorial and the official version of example script. A greater “gamma” makes the agent afraid of exploring long distance, that’s why the policy converged to a status of not going far but rotating around the origin.

I had the same problem and not solved the issue by tuning the parameter “gamma” in PPO to a smaller value. It feels like the visual input is poor.
From the video of Stable Baselines example, the cube size in the viewpoint is big.

When I run the sample code, the cube in the viewpoint seems too far and poor to make the jetbot go round in circles.

My isaac sim version is 2022.1.0. Does it have something to do with the camera parameters?