The postprocessing algorithms of classifier and detector are different. Please check with your postprocessing implementation.
Please refer to Gst-nvinfer — DeepStream documentation and DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums