Two detector nvinfer show label wrong

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU) jetson orin nx
**• DeepStream Version 7.0
**• JetPack Version (valid for Jetson only) 6.0
**• TensorRT Version 8.6
• NVIDIA GPU Driver Version (valid for GPU only)
**• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I use PGIE(yolov7) and SGIE(yolov7) by this pipline:

gst-launch-1.0 rtspsrc latency=200 location=rtsp://***************/h265/ch1/main/av_stream drop-on-latency=1 ! rtph265depay ! nvv4l2decoder ! videorate ! “video/x-raw(memory:NVMM), framerate=(fraction)18/1”
! mux.sink_0 nvstreammux name=mux batch_size=1 width=1920 height=1080 enable-padding=0 nvbuf-memory-type=0 batched-push-timeout=100 live-source=0 ! queue ! nvvideoconvert ! queue
! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-7.0/sources/objectDetector_Yolo/config_infer_primary_TX.txt interval=10 batch-size=8 unique-id=1
! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-7.0/sources/objectDetector_Yolo/config_infer_secondary_TX.txt unique-id=2 process-mode=2
! nvtracker name=tracker1 ll-config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so tracker-width=640 tracker-height=384 user-meta-pool-size=128
! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=RGBA’ ! nvdsosd process-mode=1 ! nv3dsink

it is like pgie->sgie->tracker

my pgie is a detector for only one class:person and my sgie is a detector for 7 classes

I find that the show on my window, the label of pgie is sgie’s label

then I print the object_meta in plugin nvosd, it’s show that all the unique_component_id, obj_label, text_params.display_text are sgie’s result but the rect_params is pgie’s

How could I use two detector nvinfe in my pipeline?

1 Like
3
  1. if the second model is dependent to the first, please refer ro this sample deepstream_lpr_app. the first model detects vehicles, the second model detects the car license plate.
  2. if the two models have no dependency relation, process-mode of the second model should be set 1. nvinfer plugin is opensource. “process-mode=1” means process_full_frame.
4

yes my second model is dependent to the first

I have already see this project , but in this project the plugin nvtracker is only for the first model

But I use interval=10 in PGIE and secondary-reinfer-interval=10 in SGIE , so I must use one nvtracker for both two detector to get the box it missed , in this situation the result is wrong .

I also tried that use two nvtracker either for the PGIE and SGIE，and the result is also wrong, the detection box of SGIE is replaced by the PGIE detection box.

1 Like
5

if the second model is dependent to the first, please use the pipeline “…pgie + tracker+ sgie…”. you don’t need to track the sgie objects because the sgie object is a part of pgie object. if you want to the trackid of sgie objects. you can use the parent pointer of NvDsObjectMeta to get the parent object first, then get the trackid of parent object.

6

I have try the pipelne like “…pgie + tracker+ sgie…”, it works good, but the gpu runs 100% , and the fps can’t get 20

So I must use secondary-reinfer-interval parameter in the SGIE config file, if I use this param I must ust nvtracker to generate the box I have intervaled in SGIE.

If U mean that I must generate the trackid for sgie by myself?

7

please refer to this link for performance improvement.
nvinfer plugin is opensource. the second is detection model. if you set secondary-reinfer-interval=n, , every consecutive n frames will not be inferenced by sgie. there will sgie box in these consecutive frames.

8

If there will sgie box in these consecutive frames, how can I get them?
My purpose is to get every box in sgie of every frame

1 Like
9

if there are sgie object meta, please refer to osd_sink_pad_buffer_probe of deepstream-test1 for how to access obj_meta. the bboxes are in rect_params of obj_meta.

10

ok I will see deepstream-test1
and my question is that if I use secondary-reinfer-interval=n in my sgie, how can I track the boxes of sgie

11

please refer to my comment on Oct 29. you can use the trackid of parent objects as the trackid of sgie objects because sgie objects are a part of parent object(pgie object).