So my group is trying to get class-wise and instance-wise segmentation masks from images in a simulated environment of our own creation.
In this environment we have a bunch of objects which already had a class label in their object label (e.g. “chair”) and we gave a printer in the image a class label by setting IsaacSimTag for the object to “printer”.
We captured the class-wise and instance-wise segmentation masks across several images and found some odd results. The attached file seg_class.png shows the class segmentation image attained for a given moment in time. Note the yellow printer in the background and the chair to the far right of the image.
Now this all looks correct but when we go to the instance-wise segmentation mask we get the following image (see instance.png file attached).
Here, the printer and the chair to the far right of the image have been given the same instance id (colour) as the wall. They have not been segmented correctly.
However, at a different frame, we managed to get an instance segmentation for the chair on the right when looking at it from a different angle. (see instance_2.png file attached)
Is this a bug or can someone explain what is going on here and how to fix it?