Hi. I have a binary classification model in ONNX format, which I converted to the .engine format using FP32 precision. To verify that the conversion was successful, I wrote a script that runs inference on the .engine file and returns the results. I tested the model on a set of images with known ground truth labels, and the outputs from the .engine model closely matched those from the original ONNX model.
Next, I wanted to validate that the model behaves the same when integrated into a DeepStream pipeline. However, I noticed that the outputs from the DeepStream pipeline differ significantly from those obtained with the script.
For example, for an image with a ground truth label of 0, the script produced a prediction of 0.1309, while the DeepStream pipeline output was 0.7418. For other images, the differences are similarly large. Predictions for obvious true positive samples are consistent (close to 1, e.g., ~0.999), but for other cases, the discrepancy is significant.
The model expects an input size of 1x3x384x384, and I have ensured that images are preprocessed correctly in both the script and the DeepStream pipeline. I’m trying to understand what could be causing this discrepancy since both nvinfer and script use the same
The script for.engine model inference :
engine_classification_inference .py.txt (8.0 KB)
Nvinfer config:
[property]
gpu-id=0
net-scale-factor=0.007843137255
offsets=127.5;127.5;127.5
model-color-format=0 # 0=RGB, 1=BGR
model-engine-file=binary_model.engine
labelfile-path=labels.txt
network-input-order=0
batch-size=1
network-type=1 # classifier
process-mode=1
gie-unique-id=1
classifier-threshold=0.1
scaling-compute-hw=2
interval=0
maintain-aspect-ratio=1
The Deepstream pipeline schema:
