I have trained a custom UNET model to perform semantic segmentation. Using the TAO pipeline I have exported the model to an INT8 etlt + calibration file.
There’s no detections and the pixel mask is all the same class.
When inspecting the UNET output layer configuration, the DeepStream repo lists the output layer as softmax_1. However, the TAO documentation states that the softmax layer is replaced with argmax_1 during the model export optimization phase.
Is this mismatch the issue with the lack of detection from my custom model? If yes, so I retrain a previous version of UNET that does not replace softmax OR should I reconfigure the DeepStream app to accept argmax (and how does that process look??) ?
I was able to train the same custom UNET model using an older TAO/TLT version that did not replace the softmax output with argmax output. This model when run though the sample ds-tao-segmentation app was able to generate the different class pixel masks (note training was short to test the theory so output quality is not great):
I would prefer to use the up-to-date TAO pipeline but it seems it is not compatible with the currently supported DeepStream apps. How can I modify the DeepStream app source code to accommodate the replaced argmax output layer?
Using the current version of TAO is what caused this issue in the first place
I’ve already tried that fix and it did not have any impact when trying to run the newest version TAO model (see above). However, the older version TAO model does generate mask output using the config in my most recent reply.
The main difference between the two versions seems to be that in the older TAO model the softmax layer is not replaced with an argmax layer. All of the deepstream documentation and examples seem to look for softmax.
Outputs as listed by the Github ds-tao-segmentation repo:
9~10. UNET/PeopleSemSegNet softmax_1: A [batchSize, H, W, C] tensor containing the scores for each class
Is there a way to check whether this layer mismatch is the root cause of the lack of mask output in the newest TAO version models?