I’m building a DeepStream people tracking app based on deepstream_test_app3.c and deepstream_infer_tensor_meta_test.cpp. This app uses a multi-object tracker, ByteTrack, which is different from other DeepStream apps that the output detections are processed in a stateful fashion; for one input source, ByteTrack maintains a data structure that keeps track of the detections from the previous N frames and performs some post-processing logic. For this reason, my app maintains a BYTETracker object associated with one input source and depending on which input source the frame is coming from, the corresponding BYTETracker object is used.
Please have a look at
https://github.com/cocoza4/ByteTrack/blob/main/deploy/TensorRT/cpp/src/ds_rtsp_bytetrack.cpp, it contains a CustomData struct which is hard-coded with 2 BYTETracker’s for debugging purposes.
To reproduce the issue, please follow the steps below
cd ByteTrack/deploy/TensorRT/cpp/ mkdir build && cd build cmake .. make
Run 1 input source
in this case, only 1 BYTETracker is used because only 1 input source is instantiated. Notice the tracker_id associated with each bounding box, everything works as expected.
Run 2 input sources
./ds_rtsp_bytetrack file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
this time, 2 input sources are instantiated. Notice the tracker_id associated with each bounding box, they are different from the previous example. For some reason, the BYTETracker for each input source is not processed independently. So the question is, how to make BYTETracker process each one independently so that the output for each stream in the nv-tiler, matches the previous example?
I ran this experiment on docker image nvcr.io/nvidia/deepstream:6.0-devel, TensorRT v8001, and NVIDIA GeForce GTX 1650 Ti with Max-Q Design.