Training the Isaac SDK Pose CNN Decoder with Isaac Sim Omniverse

[jacob.loth]

Hi,
I am currently trying to use the Pose CNN Decoder with isaac-sim, but while training on the generated data the decoder just turns all white. With reducing the learning rate to e-6 and add_noise_to_image=true the edges of the decoder stay black but are also slowly turning white.

The training object is the armadillo figure from the Stanford 3D Scanning Repository:

The data is generated by placing multiple of these figures on a table and using the offline dataset generation with isaac-sdk. How can I stop the decoder from saturating ?
Is it a problem to have multiple objects on the table because they sometimes overlap? Any help is greatly appreciated.

Update: I tried to train on the default Isaac-Sim pose estimation example in python_samples/isaac_sdk/pose_estimation.py still no convergence after ~30000 steps.

[jacob.loth]

Solved: When using isaac-sim with pose cnn encoder the there has to be a uniform mesh background for the decoder training data that is always visible otherwise the background of the decoder training image is 255 not 0 like it should be, maybe the example python_samples/isaac_sdk/pose_estimation.py should be corrected.