When I set arm angles, which is an action for a robot in the control loop using set_jointposition target , I get the above error. When num_envs are increased training quits before 5 epochs.
I checked the action ranges and it was not exceeding joint limits but still this error happens.
Debugging it with a constant angle however never raised an error i.e. arm angle is 0 or some constant value. The obs space at this time gives an NaN value.