How To Link SGIEs Together

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1
  • Hardware Platform: GPU
  • DeepStream Version: 6.2
  • TensorRT Version: 10.3.0.26
  • NVIDIA GPU Driver Version: 535.230.02
  • Issue Type: Question, New requirement

Hello,

I’m building a DeepStream Python application where the goal is to process video streams through a sequence of inference

  1. PGIE: Detects vehicles.
  2. SGIE1: Detects license plates from the vehicle bounding boxes.
  3. SGIE2: Performs OCR on the detected license plate regions.

The first two stages (PGIE and SGIE1) are working as expected. SGIE1 correctly operates on the PGIE output and identifies license plates. The challenge is in the third stage: SGIE2 needs to run OCR on the outputs from SGIE1, but I’m running into limitations with how DeepStream handles SGIE-to-SGIE metadata propagation.

As far as I understand, SGIEs are designed to operate off PGIE metadata, and while the operate-on-gie-id field theoretically allows SGIE chaining, the framework doesn’t seem to support using the output of one SGIE as input for another in a straightforward way.

Appreciate any guidance or best practices for handling this kind of multi-stage inference workflow.

Thanks.

3

please refer to NV LPR recognition sample, which includes three models(detection->detection->classification). You can set operate-on-gie-id of sgie2 to the ID of sige1.

4

When adding SGIE2 to the pipeline, I get the following error:

Traceback (most recent call last):
  File "/opt/nvidia/deepstream/deepstream-7.1/sources/deepstream_python_apps/apps/deepstream-yolo/deepstream_yolo.py", line 539, in <module>
    main(stream_paths, pgie, car_config, plate_config, ocr_config, disable_probe)
  File "/opt/nvidia/deepstream/deepstream-7.1/sources/deepstream_python_apps/apps/deepstream-yolo/deepstream_yolo.py", line 438, in main
    pipeline.add(sgie2)
  File "/usr/lib/python3/dist-packages/gi/overrides/Gst.py", line 73, in add
    raise AddError(arg)
gi.overrides.Gst.AddError: <__gi__.GstNvInfer object at 0x73e3f8364700 (GstNvInfer at 0x5b8c775e6fc0)>

SGIE1 Config (License Plate Detection):

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=../models/plate_detection.pt.onnx
labelfile-path=../models/plate_detection.txt
batch-size=30
network-mode=2
num-detected-classes=1
interval=0
gie-unique-id=2
operate-on-gie-id=1
network-type=0
cluster-mode=2
# workspace-size=2000
parse-bbox-func-name=NvDsInferParseYolo
# parse-bbox-func-name=NvDsInferParseYoloCuda
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

process-mode=2
maintain-aspect-ratio=1
symmetric-padding=1
network-input-shape=1;3;256;256

[class-attrs-all]
nms-iou-threshold=0.7
pre-cluster-threshold=0.45
topk=100

SGIE2 Config (OCR Detection):

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=../models/ocr_detection.pt.onnx
labelfile-path=../models/ocr_detection.txt
batch-size=30
network-mode=2
num-detected-classes=37
interval=0
gie-unique-id=3
operate-on-gie-id=2
network-type=0
cluster-mode=2
# workspace-size=2000
parse-bbox-func-name=NvDsInferParseYolo
# parse-bbox-func-name=NvDsInferParseYoloCuda
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

process-mode=2
maintain-aspect-ratio=1
symmetric-padding=1
network-input-shape=1;3;256;256

[class-attrs-all]
nms-iou-threshold=0.7
pre-cluster-threshold=0.45
topk=100

All three models are YOLO11 and have been successfully converted using the DeepStream-YOLO repository. The pipeline works fine with just PGIE + SGIE1, but fails when adding SGIE2.

I’ve verified that operate-on-gie-id=2 in SGIE2 config matches SGIE1’s gie-unique-id=2, and all model files exist and are accessible.

The Python DeepStream error messages are quite vague. Are there better ways to debug these issues?

5

The issue was that I had created both SGIE elements with the same name:

sgie1 = Gst.ElementFactory.make("nvinfer", "secondary-inference")
sgie2 = Gst.ElementFactory.make("nvinfer", "secondary-inference")

Changing the second element to use a unique name fixed the problem:

sgie1 = Gst.ElementFactory.make("nvinfer", "secondary-inference")
sgie2 = Gst.ElementFactory.make("nvinfer", "tertiary-inference")

The pipeline now works perfectly with all three YOLO11 models chained together.

While I solved this particular issue, it highlights a broader problem: DeepStream Python error messages are often quite vague and unhelpful.

The gi.overrides.Gst.AddError gave no indication that the real issue was duplicate element names. This made debugging unnecessarily difficult.

Are there better ways to debug DeepStream Python applications?

6

you can use gst-lauach command-line to debug the pipeline first. Here is a sample.

  gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! mux.sink_0 nvstreammux name=mux batch-size=1 width=1280 height=720 ! nvinfer  config-file-path=./dstest2_pgie_config.txt ! nvtracker tracker-width=640 tracker-height=384 ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so ll-config-file=../../../../samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml !  nvvideoconvert ! 'video/x-raw(memory:NVMM),format=RGBA' ! dsexample full-frame=FALSE blur-objects=TRUE ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvinfer  config-file-path=./dstest2_sgie1_config.txt  ! nvinfer  config-file-path=./dstest2_sgie2_config.txt  !  nvvideoconvert ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvdsosd !   nvvideoconvert ! nveglglessink
  1. after the modified command-line works, you can port the pipeline to the Python code. Please refer to this faq for how to dump pipeline in Python.