Get different results between .onnx python and .engine nvinfer

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): NVIDIA Quadro P620
• DeepStream Version: 6.3
• TensorRT Version: 8.5.3.1
• Docker container: http://nvcr.io/nvidia/deepstream:6.3-gc-triton-devel
• NVIDIA GPU Driver Version (valid for GPU only): 535.54.03
• Issue Type( questions, new requirements, bugs): questions

Hi,
I get the different output (pred cls, conf) between nvinfer (using .engine file parsed from yolov5-cls.onnx) and python infer (using yolov5-cls.onnx).
Postprocessing of custom-parser function and python code are similar, just do sigmoid(vector_output) and get value having max_score .
So I think the difference come from preprocessing. I do the input preprocessing in python code based on “net-scale-factor”, “offsets” for normalization; “model-color-format” for color’s mode; and “infer-dims” for resize. Which properties could I miss, which lead to the difference in output? Pls help me, thank a lot.
Note: Input video is sample_720p.h264 and I compare these two outputs in a specific frame.

Yes. nvinfer preprocessing is done by hardware which is not bit by bit precision as python(software). And there are other elements before gst-nvinfer which may introduce more extra video processing.

From python point of view, there is also extra video processing which is not compatible. The H264 video is encoded I420 data, but python decoder will output BGR data, that means there is extra format conversion algorithm.

It is no meaning to compare python to DeepStream bit by bit.

For DeepStream, the only suggestion is to avoid extra processing before gst-nvinfer. E.G. nvstreammux will scale the video if the “width” and “height” are set to he different values of the original video resolution. You need to set “width” and “height” as the original video resolution to avoid extra video scaling.

1 Like

Hi @Fiona.Chen, Thank you for your reply. I have 2 relative questions:

  1. Does pgie’s preprocessing affect sgie’s ?
  2. I get the frames from osd_sink_probe(), input it in python inference, and compare with the output from Deepstream’s nvinfer (compare label, conf). Are frames from osd_sink_probe() same with the input frames of pgie ? Because only if they are the same, my comparison will make sense.

Pipeline’s flow: streammux → pgie → sgie → nvvidconv → filter → nvosd

No.

No. At least YUV to RGB format conversion is done.

1 Like

Ok. Thank you!

Hi @Fiona.Chen ,
I just get the different classification results between with and without tracker when I use my custom classification model.
I also test deepstream_test2. Without tracker, it doesn’t display output labels, it also means secondary-gies can not predict anything.
Do trackers change the pixels’ value or affect the frames ? I also try using IOU and NvSORT trackers, because they no need video frame buffers, but the output results are different, too.
Pipeline’s flow: streammux → pgie → tracker → sgie → nvvidconv → filter → nvosd.

We have PGIE+tracker+SGIE sample NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream (github.com). The tracker will not impact SGIE if it tracks the objects correctly. NvTracker may change the bboxes a little bit, it never changes any pixel of the video.

Thank you.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.