I am currently trying to use the Pose CNN Decoder with isaac-sim, but while training on the generated data the decoder just turns all white. With reducing the learning rate to e-6 and add_noise_to_image=true the edges of the decoder stay black but are also slowly turning white.
The training object is the armadillo figure from the Stanford 3D Scanning Repository:
The data is generated by placing multiple of these figures on a table and using the offline dataset generation with isaac-sdk. How can I stop the decoder from saturating ?
Is it a problem to have multiple objects on the table because they sometimes overlap? Any help is greatly appreciated.
Update: I tried to train on the default Isaac-Sim pose estimation example in python_samples/isaac_sdk/pose_estimation.py still no convergence after ~30000 steps.