DeepStream 5, request to carry confidence value from GIE in Tracker?

• Hardware Platform (Jetson / GPU)
Jetson
• DeepStream Version
5.0
• JetPack Version (valid for Jetson only)
4.4
• TensorRT Version
7.1

From nvdsmeta.h where it states "For objects found by a tracker and not inference component, confidence will be set to -0.1"

Would it be possible to change this such that the parameter contains the value from the last component to process confidence?.

If an IOU or KTL Tracker comes after the inference, to somehow carry the confidence value between frames that were inferred on? Perhaps this is a IOU and KTL update?

The challenge with the current solution is… if a client is making decisions in a Pad Probe callback based on confidence levels – i.e. shade the background of the object with the highest confidence – , then the client has to carry this information forward between inferred on frames.

It seems that it would be a lot easier if the Plugin did this instead

A feature request perhaps?

Thanks
Robert.

Hey, Could you explain more details?

Hey @baco, sorry for being unclear. Hopefully this helps.

For the case, as shown in a number of your examples, where you have a Pipeline, PGIE, Tracker., Tiler and/or OSD… with the PGIE interval set to 4.

If you have a Pad Probe Handler on one of the pads downstream of the Tracker, the handler processing the object metadata will see

frame number  |   1  |   2  |   3  |   4  |   5   |   6  | ...
confidence    | 0.92 | -0.1 | -0.1 | -0.1 |  0.94 | -0.1 | ...
inferred on   |   1  |   0  |   0  |   0  |   1   |   0  | ...

If the handler is making decisions based on the confidence value – shading the background of the object with highest confidence as a simple example – how does the handler deal with the missing data?

The only way that I can think of solving this is to have the handler track each of the objects by tracker id, replacing the -0.1 with the last know confidence value… which is not a trivial task.

It just seems like the KTL and IOU Tracker, already tracking the objects, could do the above instead of adding the -0.1. The inferred on flag lets the client know if it came from the GIE or Tracker,

Thanks,
Robert.

User can do this with 5.0 GA release. Section: Adding new metadata fields in DeepStream-5.0 GA release:
https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_metadata.html#wwpID0E3HA:

@baco, sorry for the tardy response…

using user metadata will help with a custom implementation, but I’d be surprised if others haven’t run into this issue.

Even the deepstream_imagedata-multistream.py example breaks down if someone was to set the interval on the GIE.