Adding tracker to the secondary GIE

Hardware Platform:

  • GPU: GeForce RTX 3060 Mobile
  • CUDA Version: 11.8
  • DeepStream Version: 6.2
  • NVIDIA GPU Driver Version: 520.56.06

Issue Description:
I have set up two YOLO models in my application using Primary and Secondary GIE . The Secondary GIE has been modified to receive the entire frame. However, by default, the tracker is enabled only for the primary GIE. I’m seeking assistance on how to enable the tracker for the Secondary GIE as well.

By going through the DeepStream configuration, I couldn’t find an option to enable the tracker for the Secondary GIE. Could you please provide guidance on how to enable the tracker for both the primary and secondary GIE models?

Thank you in advance for your help!

Suppose nvtracker will track all objects. Why it only tracker PGIE? Can you share your configure file of GIE and tracker?

test5_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt (8.8 KB)

I am sharing the config file

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
#uri=rtsp://foo.com/stream1.mp4
#uri=file:/opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_1080p_h265.mp4
#uri=file:/opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_720p.mp4
uri = file:/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/videos/stream_piece_20_163.mp4
num-sources=1
gpu-id=0
nvbuf-memory-type=0

smart record specific fields, valid only for source type=4

0 = disable, 1 = through cloud events, 2 = through cloud + local events

#smart-record=1

0 = mp4, 1 = mkv

#smart-rec-container=0
#smart-rec-file-prefix
#smart-rec-dir-path

smart record cache size in seconds

#smart-rec-cache

default duration of recording in seconds.

#smart-rec-default-duration

duration of recording in seconds.

this will override default value.

#smart-rec-duration

seconds before the current time to start recording.

#smart-rec-start-time

value in seconds to dump video stream.

#smart-rec-interval

[source1]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
#uri=rtsp://foo.com/stream2.mp4
uri=file:/opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_720p.mp4
num-sources=1
gpu-id=0
nvbuf-memory-type=0

smart record specific fields, valid only for source type=4

0 = disable, 1 = through cloud events, 2 = through cloud + local events

#smart-record=1

0 = mp4, 1 = mkv

#smart-rec-container=0
#smart-rec-file-prefix
#smart-rec-dir-path

smart record cache size in seconds

#smart-rec-cache

[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=UDPSink 5=nvdrmvideosink 6=MsgConvBroker
type=6
msg-conv-config=configs/dstest5_msgconv_sample_config.txt
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(2): PAYLOAD_DEEPSTREAM_PROTOBUF - Deepstream schema protobuf encoded payload
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM - Custom schema payload
msg-conv-payload-type=1
#(0): Create payload using NvdsEventMsgMeta
#(1): New Api to create payload using NvDsFrameMeta
#msg-conv-msg2p-new-api=0
#Frame interval at which payload is generated
msg-conv-frame-interval=30
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
#msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream-6.2/lib/libnvds_kafka_proto.so
#Provide your msg-broker-conn-str here
#msg-broker-conn-str=;;
#topic=
msg-broker-conn-str=192.168.29.17;9092;
topic=quickstart-events
#Optional:
#msg-broker-config=/opt/nvidia/deepstream/deepstream/sources/libs/kafka_protocol_adaptor/cfg_kafka.txt
#new-api=0
#(0) Use message adapter library api’s
#(1) Use new msgbroker library api’s

[sink2]
enable=1
type=4
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4

only SW mpeg4 is supported right now.

codec=1
sync=0
bitrate=2000000
output-file=out.mp4
source-id=0

sink type = 6 by default creates msg converter + broker.

To use multiple brokers use this group for converter and use

sink type = 6 with disable-msgconv = 1

[message-converter]
enable=0
msg-conv-config=dstest5_msgconv_sample_config.txt
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(2): PAYLOAD_DEEPSTREAM_PROTOBUF - Deepstream schema protobuf payload
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM - Custom schema payload
msg-conv-payload-type=0

Name of library having custom implementation.

#msg-conv-msg2p-lib=

Id of component in case only selected message to parse.

#msg-conv-comp-id=

Configure this group to enable cloud message consumer.

[message-consumer0]
enable=0
proto-lib=/opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
conn-str=;
config-file=/opt/nvidia/deepstream/deepstream/sources/libs/kafka_protocol_adaptor/cfg_kafka.txt
subscribe-topic-list=;;

Use this option if message has sensor name as id instead of index (0,1,2 etc.).

#sensor-list-file=dstest5_msgconv_sample_config.txt

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=2
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1920
height=1080
#enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

If set to TRUE, system timestamp will be attached as ntp timestamp

If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached

attach-sys-ts-as-ntp=1

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=2
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
#Required by the app for SGIE, when used along with config-file property
gie-unique-id=1
nvbuf-memory-type=0
model-engine-file=/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/Primarydetector/model_b2_gpu0_fp16.engine
labelfile-path=/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/Primarydetector/rac_primary_labels.txt
config-file=/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/Primarydetector/config_infer_primary_rac.txt
#infer-raw-output-dir=…/…/…/…/…/samples/primary_detector_raw_output/

[tracker]
enable=1

For NvDCF and NvDeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively

tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so

ll-config-file required to set different tracker types

ll-config-file=…/…/…/…/…/samples/configs/deepstream-app/config_tracker_IOU.yml

ll-config-file=…/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvSORT.yml

ll-config-file=…/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml

ll-config-file=…/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvDCF_accuracy.yml

ll-config-file=…/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvDeepSORT.yml

gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

[secondary-gie0]
enable=1
gpu-id=0
gie-unique-id=4
operate-on-gie-id=1
#operate-on-class-ids=2;
batch-size=16

model-engine-file=/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/Secondarydetector/model_b16_gpu0_fp16.engine
labelfile-path=/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/Secondarydetector/rac_secondary_labels.txt
config-file=/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/deepstream-test5/Secondarydetector/config_infer_secondary_rac.txt

[tests]
file-loop=1

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Can you see the rectangle which produced by SGIE? Can you share the result video to show the issue?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.