Confidence values from pgie like -0.1?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

In deepstream 4.02 we had to patch the nvinfer library and use dbscan for bounding boxes to be able to get confidence values.

We never really trusted them however as I would get a lot of false positives with the standard resnet10 with very high confidence values like .997.

Now with deepstream 5.0 preview I expected the confidence intervals would work “out of the box”. They seem to but every second or third object detection has a confidence of -0.1.

How can we have negative confidence values?
What’s going on and how do we get the real confidence values?

Currently using : obj_meta->confidence

• Hardware Platform (Jetson / GPU) Jetson Nano
• DeepStream Version DS5.0

3

Hi,

Thanks for testing our latest deepstream SDK.
We are going to reproduce this issue and will update more information with you later.

Thanks.

2 Likes
4

Any news on this one? Here is a sample of detection when I walk in front of an rtsp camera:

Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.909228
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: -0.100000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 1.000000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 1.000000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.919295
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.972799
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.981579
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 1.000000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 1.000000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 1.000000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.956577
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.925140
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.943734
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: -0.100000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: -0.100000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: -0.100000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: -0.100000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: -0.100000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 1.000000
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.951104
Class 2  on Source Id: 1, Obj.Id: 3, Confidence: 0.971247

The negative values are weird -but then so are the 1.000000 values…

The NvDCF tracker has a parameter minDetectorConfidence - how can this be used if the detector confidence values are unreliable?

I’m also noticing that the -0.1 values only seem to occur on class 2 (person).

5

Hi,

Sorry for the late update.

The confidence in Deepstream 5.0 is not set, similar to the v4.0.2.
Have you applied the patch you used before?
If not, the value should be meaningless.

Thanks.

1 Like
6

Ok thank you that is good to know.

Does this mean that without nvinfer patch that the confidence parameter in the nvinfer config file does nothing. And likewise the detector confidence config value in the DCF config file does nothing?

7

In DS 5.0.2, confidence value is set by default at the primary detector. However, you are seeing -0.1 because of the group rectangles type of clustering method. This method does not output the confidence values for the clustered boxes and hence all objects will have -0.1 conf at the pgie. However you are not seeing -0.1 for all objects because nvdcf also updates the tracker confidence values of the object. This issue will be fixed in the next release so pgie and tracker confidence values will be explicitly stated.

Would it be possible for you to share the stream with which you observed this behavior ?

1 Like
8

The stream is a live rtsp stream. I could save it to a file and send to you if that will help?

For now, before a fix comes out - is the threshold (under class-attrs-?) from the nvinfer config file working properly even though the confidence metadata is invalid ?

Also the minDetectorConfidence: 0.0 # If the confidence of a detector bbox is lower than this, then it won't be considered for tracking parameter in the dcf tracker config ?

If those two are accurately working off the real pgie confidence values than I am happy and can live with not having the value accessible in the metadata.

The reason I have been trying to access the confidence value in the metadata is to try and remove false positive detections from the standard reset10 model.

Thankyou for your help.

9

If you can record a video stream that reproduces the same behaviour, that would be helpful for us to debug.

class-attrs- config property are all working as expected. But keep in mind that ‘post-clustering-threshold’ is not applicable to group rectangles type of clustering because of the reason i mentioned above.

We will get back to regarding the question about ‘mindetectorConfidence’

10

What would be the best way to record the raw stream? I’ve tried simply:

gst-launch-1.0 -e rtspsrc location=rtsp://1.1.1.1/Streaming/Channels/201 ! rtph265depay ! filesink location=camera201.h265

But it doesn’t seem to create a file that I can feed into my app instead of the rtsp source.

Any recommendations?

11

Any detector bboxes that have confidence values lower than mindetectorConfidence would be discarded and not be tracked by the tracker. So if you expect the detector confidence values of -0.1, then it would be better to set mindetectorConfidence: -1

12

But -0.1 is a garbage confidence value. So minDetectorConfidence should probably NOT be used?

13

By setting mindetectorConfidence: -1, you don’t filter out any bboxes. So I recommend you set it that way if you don’t use any clustering mode in detector.

14

ok so -1 means use ALL detector bounding boxes.

I want to try and filter out false detections. So was thinking of using mindetectorConfidence: 0.8 for example. But my question is - will this even work since the detector is not properly setting the confidence values?

Sorry for all the replies but its very hard to work this out from your responses.

15

If you use DBSCAN or NMS in PGIE, the confidence values of detector bboxes will be >=0. Then you can use mindetectorConfidence in tracker to filter out some false positives.

16

ok. Now I’m understanding. And if using DBSCAN and I patch nvinfer I can get accurate confidence values too? What about NMS?

17

Hi Jason,

You dont need to apply any patch for nvinfer plugin in DS 5.0. The confidence values from PGIE will only be added to the metadata if you choose NMS or DBSCAN as the clustering type. For group rectangles mode of clustering, confidence values would be -0.1 for all objects.

1 Like
18

Hi Jason,
As mentioned, all confidence from primary detector are -0.1 in your case. For boxes with values near 1.0, they are not detected by primary detector, but discovered by tracker only. So currently their confidence values are given by tracker. In GA release coming soon, we will have separate confidence fields for detector and tracker so they will not be mixed together.

19

Thankyou @fangyul - makes sense and I’ll wait for 5.0 GA.

21

Any idea if this is fixed in the new release 5.0. I still get neg numbers

22

GA means general availability, for now we only have access to developer preview (5.0 dp).