Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU): Thor Dev Kit
• DeepStream Version: 8.0
• JetPack Version (valid for Jetson only): 7.0
• TensorRT Version: 10.13.2
• NVIDIA GPU Driver Version (valid for GPU only): 580.0
Issue Type( questions, new requirements, bugs)
Unable to enable the NvTracker for objects detected from an RT-DETR model.
** INFO: <bus_callback:277>: Pipeline running
GPUassert_VPI: VPI_ERROR_INTERNAL (:EGL_NOT_INITIALIZED) /dvs/git/dirty/git-master_linux/deepstream/sdk/src/utils/nvmultiobjecttracker/src/modules/VisualTracker/DCFTracker.cpp 269
!![Exception] GPUassert_VPI failed
An exception occurred. GPUassert_VPI failed
gstnvtracker: Low-level tracker lib returned error 1
0:00:01.172834381 54 0xaaaaf1acbe60 WARN nvinfer gstnvinfer.cpp:2435:gst_nvinfer_output_loop:<primary_gie> error: Internal data stream error.
ERROR from tracking_tracker: Failed to submit input to tracker
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvtracker2/gstnvtracker.cpp(841): gst_nv_tracker_submit_input_buffer (): /GstPipeline:pipeline/GstBin:tracking_bin/GstNvTracker:tracking_tracker
0:00:01.172947307 54 0xaaaaf1acbe60 WARN nvinfer gstnvinfer.cpp:2435:gst_nvinfer_output_loop:<primary_gie> error: streaming stopped, reason error (-5)
ERROR from primary_gie: Internal data stream error.
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(2435): gst_nvinfer_output_loop (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
streaming stopped, reason error (-5)
ERROR from tracking_tracker: Failed to submit input to tracker
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvtracker2/gstnvtracker.cpp(841): gst_nv_tracker_submit_input_buffer (): /GstPipeline:pipeline/GstBin:tracking_bin/GstNvTracker:tracking_tracker
ERROR from tracking_tracker: Failed to submit input to tracker
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvtracker2/gstnvtracker.cpp(841): gst_nv_tracker_submit_input_buffer (): /GstPipeline:pipeline/GstBin:tracking_bin/GstNvTracker:tracking_tracker
ERROR from qtdemux0: Internal data stream error.
Debug info: ../gst/isomp4/qtdemux.c(7439): gst_qtdemux_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason error (-5)
Quitting
[NvMultiObjectTracker] De-initialized
App run failed
How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Take a trained model, configure deepstream-app to use a tracker like this one:
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0
[source0]
enable=1
# Type 3 = URI/File
type=3
uri=file:///opt/nvidia/deepstream/deepstream-8.0/samples/streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
cudadec-memtype=0
[streammux]
gpu-id=0
## Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
## Resolution of the pipeline (adjust to match your video aspect ratio)
width=1920
height=1080
enable-padding=1
nvbuf-memory-type=0
[primary-gie]
enable=1
gpu-id=0
batch-size=1
config-file=/model/config_infer_rtdetr.txt
nvbuf-memory-type=0
[tracker]
enable=1
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
gpu-id=0
[osd]
enable=1
gpu-id=0
border-width=3
text-size=15
text-color=1;1;1;1
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=0
clock-y-offset=0
clock-text-size=12
clock-color=1;0;0;0
# CRITICAL: display-mask=1 is required to see segmentation outputs
display-mask=0
display-bbox=1
display-text=1
process-mode=1
[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink/nv3dsink(Jetson only) 3=File 4=RTSPStreaming 5=nvdrmvideosink
type=2
sync=0
conn-id=0
width=0
height=0
plane-id=1
source-id=0
[sink1]
#source0 output as filesink
enable=1
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
#bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
# set profile only for hw encoder, sw encoder selects profile based on sw-preset
profile=0
#output-file=out_source0.mp4
output-file=/data/S1000030-output.mp4
source-id=0
msg-conv-dummy-payload=1
Configure the Primary GIE for inference:
[property]
gpu-id=0
offsets=123.675;116.28;103.53
net-scale-factor=0.0173520735728
onnx-file=/model/rtdetr_model.onnx
model-engine-file=/model/rtdetr_model_fp16.engine
labelfile-path=/model/labels.txt
# int8-calib-file=../../models/Primary_Detector/cal_trt.bin
batch-size=1
process-mode=1
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=9
interval=0
gie-unique-id=1
#parse-bbox-func-name=NvDsInferParseCustomResnet
#custom-lib-path=/path/to/this/directory/libnvds_infercustomparser.so
output-blob-names=pred_boxes;pred_logits
custom-lib-path=/model/libnvds_infercustomparser_tao.so
parse-bbox-func-name=NvDsInferParseCustomDDETRTAO
## 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)
cluster-mode=2
#scaling-filter=0
#scaling-compute-hw=0
#Use the config params below for dbscan clustering mode
#[class-attrs-all]
#detected-min-w=4
#detected-min-h=4
#minBoxes=3
#Use the config params below for NMS clustering mode
[class-attrs-all]
topk=20
nms-iou-threshold=0.2
pre-cluster-threshold=0.5
NOTE: I’m using the customparser_tao library
The RT-DETR doesn’t have an official post-processor from Nvidia, but the Deformable DETR post-processor produces bboxes that appear to be valid.
output-blob-names=pred_boxes;pred_logits
custom-lib-path=/model/libnvds_infercustomparser_tao.so
parse-bbox-func-name=NvDsInferParseCustomDDETRTAO
Then attempt to launch the deepstream-app:
#!/bin/bash
docker run -it --rm \
-v $(pwd)/model:/model \
-v $(pwd)/data:/data \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e DEBUG2=true \
-e DISPLAY=$DISPLAY \
-e GST_DEBUG2=2,GST_SCHEDULING:5,v4l2videoenc:9,nvv4l2h265enc:9 \
nvcr.io/nvidia/deepstream:8.0-triton-multiarch \
deepstream-app -c /model/ds_config_rtdetr.txt
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
Additional Notes:
- I was able to run inference on the entire video, produce an encoded video output, and see the annotations/bounding boxes.
- When I enable the tracker, everything fails.
- Everything works until at least 1 detection occurs, and then when there are no detections, the error happens. I was able to verify this using the
-e DEBUG=trueenvironment variable to see from the custom post-processor that there were no detections right before the error happens, but there were in the previous frames.
Seems this configuration file works (NvSORT), so the issue is in the use of NvDCF_perf.yml
ll-config-file=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvSORT.yml`