Tracker Lib and Tracker Confidence

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

• Hardware Platform (Jetson / GPU) : Jetson
• DeepStream Version : 7.1
• JetPack Version (valid for Jetson only) : 6.2
• TensorRT Version : 10.3.0.30

Hi, I am using deepstream 7.1 with python apps, where i altered the pipeline to add tracking algorithm

    tracker = Gst.ElementFactory.make("nvtracker", "tracker")
    if not tracker:
        sys.stderr.write(" Unable to create tracker \n")
    #Set properties of tracker
    config = configparser.ConfigParser()
    config.read('dstest2_tracker_config.txt')
    config.sections()

    for key in config['tracker']:
        if key == 'tracker-width' :
            tracker_width = config.getint('tracker', key)
            tracker.set_property('tracker-width', tracker_width)
        if key == 'tracker-height' :
            tracker_height = config.getint('tracker', key)
            tracker.set_property('tracker-height', tracker_height)
        if key == 'gpu-id' :
            tracker_gpu_id = config.getint('tracker', key)
            tracker.set_property('gpu_id', tracker_gpu_id)
        if key == 'll-lib-file' :
            tracker_ll_lib_file = config.get('tracker', key)
            tracker.set_property('ll-lib-file', tracker_ll_lib_file)
        if key == 'll-config-file' :
            tracker_ll_config_file = config.get('tracker', key)
            tracker.set_property('ll-config-file', tracker_ll_config_file)

   #for linking the tracker in the pipeline - 
    streammux.link(pgie)
    pgie.link(tracker) 
    tracker.link(nvvidconv1)

In the tracker config file, it is mentioned that the NvDCF tracker is using, I am trying to get a flag for all the objects if it is moved out of the frame or not, for that, deepstream defined a term called “shadowTracking”, based on the confidence value generated from the tracking algorithm. I will assume, that if the tracker_confidence is less than a threshold, then i will consider that the object is moved out of the frame.

But when I accessed the tracker confidence from the obj_meta.tracker_confidence from class pyds.NvDsObjectMeta, I get 0 for all the objects.

Could you please help me on how to get the tracker confidence score.

And also, In the config file, the lib file is used /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so for NvDCF tracker,
I planned to use NvSORT, NvDeepSORT, KLT and IOU for that, when I checked the lib files in /opt/nvidia/deepstream/deepstream/lib/ folder, it is not found, could you please help me on where to find these tracker lib files.

thanks

3

I can get the tracker confidence with below sample:

ubuntu@ubuntu:/opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/apps/deepstream-test2$ python3 deepstream_test_2.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264

diff --git a/apps/deepstream-test2/deepstream_test_2.py b/apps/deepstream-test2/deepstream_test_2.py
index f787a7f..a3ce979 100755
--- a/apps/deepstream-test2/deepstream_test_2.py
+++ b/apps/deepstream-test2/deepstream_test_2.py
@@ -77,6 +77,7 @@ def osd_sink_pad_buffer_probe(pad,info,u_data):
             except StopIteration:
                 break
             obj_counter[obj_meta.class_id] += 1
+            print(f"tracker confidence {obj_meta.tracker_confidence}")
             try:
                 l_obj=l_obj.next
             except StopIteration:

The output is below:
tracker confidence 0.7318087220191956
tracker confidence 0.601154088973999
tracker confidence 0.7488856315612793
tracker confidence 0.4647119343280792
tracker confidence 0.75792396068573
tracker confidence 0.5954587459564209
tracker confidence 0.40935754776000977
tracker confidence 0.721225917339325
tracker confidence 0.7556650638580322
tracker confidence 0.6902937889099121
tracker confidence 0.4697114825248718
tracker confidence 0.346706748008728

Please use different tracker config for different type of tracker:
/opt/nvidia/deepstream/deepstream-7.1/samples/configs/deepstream-app/config_tracker_NvSORT.yml
/opt/nvidia/deepstream/deepstream-7.1/samples/configs/deepstream-app/config_tracker_IOU.yml
/opt/nvidia/deepstream/deepstream-7.1/samples/configs/deepstream-app/config_tracker_NvDCF_accuracy.yml
/opt/nvidia/deepstream/deepstream-7.1/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
/opt/nvidia/deepstream/deepstream-7.1/samples/configs/deepstream-app/config_tracker_NvDeepSORT.yml
/opt/nvidia/deepstream/deepstream-7.1/samples/configs/deepstream-app/config_tracker_NvDCF_max_perf.yml

4

@kesong, thanks for writing back, I will check it out,
I have one another thing to clarify, on my yesterday testing, I used deepstream-imagedata-multistream app, in that when I observed the obj_meta.object_id which is the track_id, it shows 18446744073709551615 for all the objects.

In deepstream_test_2 app, the tracker is defined in the pipeline, but in deepstream-imagedata-multistream, the tracker is not defined in the pipeling, now, I dont understand why the track_id is showing some number.

And, when testing the deepstream-imagedata-multistream without tracker algorithm, it took 25fps in jetson nano module, also when i have added the tracking in the pipeline as mentioned in the deepstream_test_2, it shows the same 25 fps and it gives the track_id for each object starting from 0,1,… - which means adding the tracking algorithm in pipeline is not affected the fps that much, even though it uses GPU with more computation - is it the regular/usual scenario or when we are adding tracking algorithm the fps will reduce ?

And, if the FPS is same for with/without adding tracker, then this mean - deepstream-imagedata-multistream app has doing tracking but the value is not correct as it is shows 18446744073709551615 for all the objects

Could you please justify this, so that I can understand the flow, thanks

5

Can you share how you test the FPS? Regarding performance test, please refer: Performance — DeepStream documentation

6

Hi @kesong,

I have tested fps from the app, in tiler_sink_pad_buffer_probe function, after reading each frames, the perf_data is updated for fps -

global perf_data
perf_data.update_fps(stream_index)

by this I get to know the fps.

Do you have any suggestion on this tracking_number is same for each objects

thanks for writing back

7

for eg., to change the tracker algorithm from NvDCT to IOU, then in dstest2_tracker_config.txt file, is it enough to change the ll-config-file parameter to the according yaml file?

Because, when I gothrough the parameters, i didnt see a specific parameter for specificing tracker_algorithm to use,

In dstest2_tracker_config.txt, the authors have commented that ll-config-file: required for NvDCF, optional for KLT and IOU

If I choose IOU, then we need to change the yaml file to config_tracker_IOU.yml but it conflicts with the comments mentioned in the dstest2_tracker_config.txt

Could you please explain these