Unable to get track ids for secondary gie in deepstream test5 application

muzair1 · August 29, 2024, 6:25am

Please provide complete information as applicable to your setup.

±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 2224 G /usr/lib/xorg/Xorg 53MiB |
±----------------------------------------------------------------------------------------+

• DeepStream Version 7.0-triton-multiarch

We are encountering an issue where, when both the primary and secondary are configured as detectors, the tracking by the secondary is not accurate and does not align correctly with the primary detector.

I’m trying to implement yoloface as my secondary gie, I’m getting random track ids for it. Below is the config file

################################################################################

Copyright (c) 2018-2022, NVIDIA CORPORATION. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a

copy of this software and associated documentation files (the “Software”),

to deal in the Software without restriction, including without limitation

the rights to use, copy, modify, merge, publish, distribute, sublicense,

and/or sell copies of the Software, and to permit persons to whom the

Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER

DEALINGS IN THE SOFTWARE.

################################################################################

application:
enable-perf-measurement: 1
perf-measurement-interval-sec: 5
#gie-kitti-output-dir: streamscl

tiled-display:
enable: 1
rows: 1
columns: 2
width: 1280
height: 720
gpu-id: 0
nvbuf-memory-type: 0

source:
#csv-file-path: sources_rtsp.csv
csv-file-path: sources_4.csv

sink0:
enable: 0
#Type - 1=FakeSink 2=EglSink 3=File
type: 1
sync: 0
source-id: 0
gpu-id: 0
nvbuf-memory-type: 0

sink1:
enable: 1
#Type - 1=FakeSink 2=EglSink 3=File 4=UDPSink 5=nvdrmvideosink 6=MsgConvBroker
type: 6
msg-conv-config: dstest5_msgconv_sample_config.yml
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(2): PAYLOAD_DEEPSTREAM_PROTOBUF - Deepstream schema protobuf encoded payload
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM - Custom schema payload
msg-conv-payload-type: 1
#(0): Create payload using NvdsEventMsgMeta
#(1): New Api to create payload using NvDsFrameMeta
msg-conv-msg2p-new-api: 1
#Frame interval at which payload is generated
msg-conv-frame-interval: 1
msg-broker-proto-lib: /opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
#Provide your msg-broker-conn-str here
#msg-broker-conn-str: ;;
#topic:
msg-broker-conn-str: localhost;9092
topic: quickstart-events

#Optional:
#msg-broker-config: /opt/nvidia/deepstream/deepstream/sources/libs/kafka_protocol_adaptor/cfg_kafka.txt
#new-api: 0
#(0) Use message adapter library api’s
#(1) Use new msgbroker library api’s

sink2:
enable: 1
type: 3
#1=mp4 2=mkv
container: 1
#1=h264 2=h265 3=mpeg4

only SW mpeg4 is supported right now.

codec: 3
sync: 1
bitrate: 2000000
output-file: out.mp4
source-id: 0

sink type = 6 by default creates msg converter + broker.

To use multiple brokers use this group for converter and use

sink type = 6 with disable-msgconv : 1

message-converter:
enable: 0
msg-conv-config: dstest5_msgconv_sample_config.yml
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(2): PAYLOAD_DEEPSTREAM_PROTOBUF - Deepstream schema protobuf encoded payload
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM - Custom schema payload
msg-conv-payload-type: 0

Name of library having custom implementation.

#msg-conv-msg2p-lib:

Id of component in case only selected message to parse.

#msg-conv-comp-id:

Configure this group to enable cloud message consumer.

message-consumer0:
enable: 0
proto-lib: /opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
conn-str: ;
config-file: /opt/nvidia/deepstream/deepstream/sources/libs/kafka_protocol_adaptor/cfg_kafka.txt
subscribe-topic-list: ;;

Use this option if message has sensor name as id instead of index (0,1,2 etc.).

#sensor-list-file: dstest5_msgconv_sample_config.txt

osd:
enable: 1
gpu-id: 0
border-width: 1
text-size: 15
text-color: 1;1;1;1
text-bg-color: 0.3;0.3;0.3;1
font: Arial
show-clock: 0
clock-x-offset: 800
clock-y-offset: 820
clock-text-size: 12
clock-color: 1;0;0;0
nvbuf-memory-type: 0

streammux:
gpu-id: 0
##Boolean property to inform muxer that sources are live
live-source: 0
batch-size: 2
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout: 40000

Set muxer output width and height

width: 1920
height: 1080
#enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding: 0
nvbuf-memory-type: 0

If set to TRUE, system timestamp will be attached as ntp timestamp

If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached

attach-sys-ts-as-ntp: 1

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

primary-gie:
enable: 1
gpu-id: 0
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size: 1
#Required by the app for OSD, not a plugin property
bbox-border-color0: 1;0;0;1
bbox-border-color1: 0;1;1;1
bbox-border-color2: 0;0;1;1
bbox-border-color3: 0;1;0;1
interval: 0
#Required by the app for SGIE, when used along with config-file property
gie-unique-id: 1
nvbuf-memory-type: 0

model-engine-file: …/…/…/…/…/samples/models/Primary_Detector/resnet10.caffemodel_b2_gpu0_int8.engine

labelfile-path: …/…/…/…/…/samples/models/Primary_Detector/labels.txt

config-file: …/…/…/…/…/samples/configs/deepstream-app/config_infer_primary.yml

#infer-raw-output-dir: …/…/…/…/…/samples/primary_detector_raw_output/
config-file: /opt/nvidia/deepstream/deepstream-7.0/sources/apps/sample_apps/kadada_deepstream-test5-V1/models/config_infer_primary_yoloV8_face.txt

config-file: /opt/nvidia/deepstream/deepstream-7.0/sources/apps/sample_apps/kadada_deepstream-test5-V1/models/config_infer_primary.yml

tracker:
enable: 1

For NvDCF and NvDeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively

tracker-width: 640
tracker-height: 384
ll-lib-file: /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so

ll-config-file required to set different tracker types

ll-config-file: …/…/…/…/…/samples/configs/deepstream-app/config_tracker_IOU.yml

ll-config-file: …/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvSORT.yml

ll-config-file: …/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml

ll-config-file: …/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvDCF_accuracy.yml

ll-config-file: …/…/…/…/…/samples/configs/deepstream-app/config_tracker_NvDeepSORT.yml

gpu-id: 0
enable-batch-process: 1
enable-past-frame: 1
display-tracking-id: 1

secondary-gie0:
enable: 0
gpu-id: 0
gie-unique-id: 2
operate-on-gie-id: 1
operate-on-class-ids: 0
batch-size: 1

config-file: …/…/…/…/…/samples/configs/deepstream-app/config_infer_secondary_vehicletypes.yml

labelfile-path: …/…/…/…/…/samples/models/Secondary_VehicleTypes/labels.txt

model-engine-file: …/…/…/…/…/samples/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine

config-file: /opt/nvidia/deepstream/deepstream-7.0/sources/apps/sample_apps/kadada_deepstream-test5-V1/models/config_infer_primary_yoloV8_face.txt

secondary-gie1:
enable: 0
gpu-id: 0
gie-unique-id: 5
operate-on-gie-id: 1
operate-on-class-ids: 0
batch-size: 1
config-file: /opt/nvidia/deepstream/deepstream-7.0/sources/apps/sample_apps/kadada_deepstream-test5-V1/models/arcface_sgie3_config.txt

config-file: …/…/…/…/…/samples/configs/deepstream-app/config_infer_secondary_carcolor.yml

labelfile-path: …/…/…/…/…/samples/models/Secondary_CarColor/labels.txt

model-engine-file: …/…/…/…/…/samples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine

secondary-gie2:
enable: 0
gpu-id: 0
gie-unique-id: 6
operate-on-gie-id: 1
operate-on-class-ids: 0
batch-size: 16
config-file: …/…/…/…/…/samples/configs/deepstream-app/config_infer_secondary_carmake.yml
labelfile-path: …/…/…/…/…/samples/models/Secondary_CarMake/labels.txt
model-engine-file: …/…/…/…/…/samples/models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine

tests:
file-loop: 0

kesong · August 29, 2024, 9:32am

What is the mean of random track id? Can you share some video to show it?

muzair1 · August 29, 2024, 9:35am

muzair1 · September 2, 2024, 9:17am

I’m using a custom YOLO model as my primary GIE, and the tracker is generating correct track IDs. However, when I use YOLOFace as my secondary GIE (sgie 0), I’m encountering an issue where it’s assigning a random track ID of 18446744073709551615. This is causing problems in my pipeline, and I urgently need help resolving this. Any guidance or solutions would be greatly appreciated!

kesong · September 5, 2024, 6:09am

nvtracker is located after PGIE in the pipeline. So the bbox of PGIE is tracked. But SGIE is located after nvtracker, So the bbox of SGIE isn’t tracked. Suppose the bbox of SGIE has “parent” bbox. Can you get the object_id from the 'parent" bbox if the object_id is untracked object ID?

  52 /** Defines an untracked object ID. */
  53 #define UNTRACKED_OBJECT_ID 0xFFFFFFFFFFFFFFFF


 342 typedef struct _NvDsObjectMeta {
 343   NvDsBaseMeta base_meta;
 344   /** Holds a pointer to the parent @ref NvDsObjectMeta. Set to NULL if
 345    no parent exists. */
 346   struct _NvDsObjectMeta *parent;

yingliu · September 30, 2024, 7:00am

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

system · October 14, 2024, 7:01am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Adding tracker to the secondary GIE DeepStream SDK ubuntu	4	430	July 7, 2023
Try to get the result of SGIE result, failed DeepStream SDK deepstream61	9	595	July 24, 2022
Two back to back PGIEs not working using config txt DeepStream SDK	12	761	October 12, 2021
Draw tracking id on deepstream-test-5 DeepStream 5.0 DeepStream SDK	5	993	October 12, 2021
Secondary-gie malfunctionality DeepStream SDK	5	1523	October 12, 2021
Can only see SGIE inferences inside the PGIE inferences. Not sure if SGIE is acting on full frame DeepStream SDK	12	511	September 26, 2023
Deepstream5.0.1+yolov5+resnet50 No output classifier result DeepStream SDK gstreamer	4	926	August 4, 2021
No sgie metadata for some pgie detections using pyds DeepStream SDK gstreamer , python	17	1439	October 12, 2021
Deepstream_parappel_app issue with kafka messaging DeepStream SDK deepstream	8	34	July 27, 2024
Integrate back to back detectors with deep stream test5 application DeepStream SDK	11	440	July 18, 2023