Cant Track Object Using Deepstream Gstreamer Plugin NvTracker with NvDCF

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,

I have a problem with using NvDCF for tracking as the NVOSD wont be able to show the tracking bbox along with object ID.
FYI, the tracking works if I swapped NvDCF with IOU or KLT.

Besides, I have also used the same tracker_config.yml from deepstream samples.(copied it out instead)
I have changed abit of the config after reading tru NvDCF guides.

NvDCF:
  minTrackerConfidence: 0.6  # If the confidence of an object tracker is lower than this on the fly, then it will be tracked in shadow mode. Valid Range: [0.0, 1.0]
  ......
  # [Instance-awareness]
  useInstanceAwareness: 1 # Use instance-awareness for multi-object tracking

gst-launch-1.0 -v \
nvstreammux name=m batch-size=4 width=1920 height=1080 ! \
nvinfer config-file-path= $CONFIG_FILE_PATH batch-size=4 unique-id=1 ! \
nvmultistreamtiler rows=2 columns=2 width=3840 height=2160 ! \
nvtracker tracker-width=320 tracker-height=320 ll-lib-file=/opt/nvidia/deepstream/deepstream-5.1/lib/libnvds_nvdcf.so ll-config-file=tracker_config.yml ! \
nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! queue ! nvdsosd process-mode=1 ! nvegltransform ! fpsdisplaysink video-sink=nveglglessink sync=false text-overlay=false \
filesrc location=$VIDEO ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_1 \
filesrc location=$VIDEO ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_2 \
filesrc location=$VIDEO ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_3 \
filesrc location=$VIDEO ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 \

Please help.

Thanks,
Vincent

5

put tiler after tracker, then it works.
gst-launch-1.0 -v nvstreammux name=m batch-size=4 width=1920 height=1080 ! nvinfer config-file-path= samples/configs/deepstream-app/config_infer_primary.txt model-engine-file= samples/models/Primary_Detector/resnet10.caffemodel_b4_gpu0_int8.engine batch-size=4 unique-id=1 ! nvtracker tracker-width=320 tracker-height=320 ll-lib-file=/opt/nvidia/deepstream/deepstream-5.1/lib/libnvds_nvdcf.so ll-config-file= samples/configs/deepstream-app/tracker_config.yml ! nvmultistreamtiler rows=2 columns=2 width=1920 height=1080 ! nvvideoconvert ! “video/x-raw(memory:NVMM), format=RGBA” ! queue ! nvdsosd process-mode=1 ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=script.mp4 filesrc location= samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_1 filesrc location=samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_2 filesrc location=samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_3 filesrc location=samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0

6

Hi Amy,

Thanks for your solution.
Can I know is putting tracker after nvinfer a good practice or it is only applicable for NvDCF?

Thanks,
Vincent

7

It’s only for NvDCF tracker. it needs stream source id, frame meta, frame buffer for low level NvMOT_Process, while iou, klt does not need source id. after nvtiler, all frames metadata transformed to one frame metadata, individual frames source id all lost.

8

Hi Amy,

Thanks for your clarification.

Thanks,
Vincent