Classification model with the gpu setup and docker

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi — I wanted to share the investigation results and confirm the approach we’re taking for the SGIE classifiers in DeepStream.

Summary

We identified that the ResNet18 classifiers (color/lid/load) were trained using ImageNet normalization:

(pixel/255 − mean) / std

but in DeepStream only the scaling step was applied:

pixel/255

This caused an input distribution mismatch and resulted in the classifiers always predicting class 0.

Training Setup

  • Dataset format: ImageFolder

  • Classes

    • Color: 3

    • Lid: 2

    • Load: 4

  • Model: ResNet18 pretrained (ImageNet1K)

  • Preprocessing:

    • Resize → 224×224

    • ToTensor

    • Normalize
      mean = [0.485, 0.456, 0.406]
      std = [0.229, 0.224, 0.225]

  • Loss: CrossEntropy

  • Optimizer: Adam (lr=0.001)

Export + TensorRT

We exported ONNX models (opset 18) and generated corrected versions embedding normalization layers:

*_with_std.onnx

Engine build:

trtexec --onnx=.onnx
–saveEngine=.engine
–fp32

DeepStream Pipeline

  • PGIE

    • YOLOv7 detector

    • gie-unique-id = 1

  • SGIE

    • Color → id=2

    • Lid → id=3

    • Load → id=4

Original preprocessing:

net-scale-factor=1/255
model-color-format=RGB
infer-dims=3;224;224

This lacked mean/std normalization.

Config Changes Applied

Updated SGIE configs to use corrected ONNX:

ds_sgie1_c2g_config.txt
→ resnet18_color_v2_fixed_with_std.onnx

ds_sgie2_c2g_config.txt
→ resnet18_lid_v4_fixed_with_std.onnx

ds_sgie3_c2g_config.txt
→ resnet18_load_fixed_with_std.onnx

No other pipeline changes made.

Expected Result

  • Previously → all predictions class 0

  • Now → varied predictions across classes

  • Metadata flow unchanged:
    obj_meta → classifier_meta_list → label extraction

Questions

Could you please confirm:

1️⃣ Is embedding normalization inside the ONNX model the recommended approach for our deployment setup, or should we instead replicate mean/std normalization via DeepStream preprocessing/config?

2️⃣ Are there any additional DeepStream parameters you would like us to validate before final deployment?

Thanks!

4

Sorry for the long delay because a holiday.

Since your model has different mean/std values ​​for different channels, you need to insert nvdspreprocess into the pipeline.

We provide a GPU-accelerated preprocess custom-impl library. please refer to this repository.

5

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.