"rewords" are different under the same conditions

Hi
I repeated the training of the task I created 8 times under the same conditions, but I got a different rewords curve(light blue curve, Task_0) only once as shown in the graph below.
“seed” is fixed to the value of ‘42’.
Is this normal?

Also, in the above, the activation function is set to “elu”, but the result of setting it to “selu” is the graph below. It will be a different curve each time.

In the case of “FrankaCabinet”, there seems to be no difference.

Hi @DDPG7,

Indeed, this is a not-uncommon problem that will be very dependent on the particulars of your network and training, as well as any runtime changes you are making in the environment.

See more info here: IsaacGymEnvs/reproducibility.md at main · NVIDIA-Omniverse/IsaacGymEnvs · GitHub and here: Reproducibility — PyTorch 1.10.0 documentation

We’ve reviewed all of the networks and environments that we ship in IsaacGymEnvs and they should all now be fully deterministic. Changing activation functions or some kinds of network structures or training could easily have determinism issues if you are not using the torch_deterministic=True argument. But that mechanism appears to be broken in PyTorch > 1.8.1 from our testing.

Dynamic runtime changes to the environment outside of simulation also have the potential to cause issues. Please see the notes on Reproducability I pointed to or my GTC Isaac Gym talk for more info.

Take care,
-Gav

Hi

Sure, it looks like a pytorch bug.

though in our testing we did not notice determinacy issues arising from not setting this flag.

The task I created is not reproducible, but the samples (eg “FrankaCabinet”) seem to be reproducible. Do you know any good reason?

Hi @DDPG7,

The biggest reasons for non-determinacy are dynamic runtime changes to the environment (ie: certain kinds of domain randomizations, or other simulation changes that come outside the tensorized APIs).

If you are seeing non-determinacy with PyTorch 1.8.1 and the torch_deterministic=True flag on (ruling out NN side non-determinism), and are not changing any simulation parameters at runtime, please let us know if you can share some sample code.

Take care,
-Gav