Problems encountered in training unet and inference unet

Could you please check all the images? Is there any detection for the fire?

Using the same images and masks, the accuracy of training the model under the pytorch framework is good, but the training results using tlt or tao are bad. I don’t know what went wrong.

I check all the images, there is no detection for the fire.

Thanks for the info. Will check further.

Thanks, is there any progress now?

We still focus on this fire dataset and trigger experiments. With above spec file, actually one of our internal engineers can run successfully and get the correct inference result. But strangely another guy cannot. So, we’re still checking.

Please try below solution which is working on my side.
Change the training image from jpg to png.

$ for i in *.jpg ; do convert "$i" "${i%.*}.png" ; done

The fire can be detected during inference. And there is no Nan issue in evaluation.

More, after changing training .jpg files to .png files, you can also use below loss parameter.

  • loss: “cross_entropy”
  • weight: 2e-06
  • crop_and_resize_prob : 0.01
1 Like

Thanks, it works now.