False line-crossing count for persons who do not pass the line in deepstream-occupancy-analytics

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson
• DeepStream Version: 6.0.1
• JetPack Version (valid for Jetson only): 4.6.2
• TensorRT Version: 8.2.1-1+cuda10.2
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs): questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I encountered an issue with a sample app deepstream-occupancy-analytics (https://github.com/NVIDIA-AI-IOT/deepstream-occupancy-analytics) that I used to count line crossing of people.

I found that a person whose bounding box has not intersected with the configured line is sometimes regarded to cross the line.
The count increases when the bounding box is away from the line.
Here is a part of nvdsanalytics config related to line-crossing:

[line-crossing-stream-0]
enable=1
line-crossing-Exit=1050;600;1050;490;950;490;1200;490;
line-crossing-Entry=700;400;700;510;600;510;850;510
class-id=0
extended=0
mode=loose

The false counts persist even when setting “mode” to “strict”.

I am wondering how the nvdsanalytics plugin, which deepstream-occupancy-analytics depends on, counts line crossing.
My understanding is that the count for the configured line increases by one when a straight line segment between bottom centers of bounding boxes with the same tracking-id in consecutive two frames intersects the configured line and the angle between them satisfies a certain condition that is determined by “mode” in nvdsanalytics config.

Is this kind of false line-crossing count a known issue?
And, if my understanding is not correct, could you tell me how nvdsanalytics counts line crossing?
Thank you.

3

Could you share the input video by forum prviate email？thanks，we will have a try. And Please point out the timespan.

4

Thank you.
But sorry, for now the original input video cannot be shared because it contains information of my customer.
If some video reproducing the same issue becomes available, I will let you know.

6

I checked the video you shared.

  1. about “For line “Exit”, 00:15 (ID=22), 00:18 (ID=1), 00:38 (maybe ID=43), and 01:01 (ID=47).”, after checking again, there are some counting errors.
    at 00:08 (ID=22), 00:30 (ID=43) , the person passed the line, but nvdsanalytics did not count.
    at 00:15 (ID=1), the person did not pass the line, but nvdsanalytics counted.
  2. about “For line “Entry”, 00:16 (ID=12), 00:26 (ID=11), 01:16 (ID=70), and 01:28 (ID=31).”. it is expected because the moving direction is not the same with the line’s.
7

image
image394×546 27.2 KB

about “at 00:08 (ID=22), 00:30 (ID=43)” 's issue, please refer to nvdsanalytics , nvdsanalytics plugin will check the bottom center coordinate(x_left + width/2, y_top + height). when the person 's bottom center coordinate passed the crossing line, the bbox disappeared because the person’s body left the frame partially, as a result, nvdsanalytics did not count.

8

Thank you very much for checking.

  1. about “For line “Exit”, 00:15 (ID=22), 00:18 (ID=1), 00:38 (maybe ID=43), and 01:01 (ID=47).”, after checking again, there are some counting errors.
    at 00:08 (ID=22), 00:30 (ID=43) , the person passed the line, but nvdsanalytics did not count.
    at 00:15 (ID=1), the person did not pass the line, but nvdsanalytics counted.
  2. about “For line “Entry”, 00:16 (ID=12), 00:26 (ID=11), 01:16 (ID=70), and 01:28 (ID=31).”. it is expected because the moving direction is not the same with the line’s.

Could you tell me whether these comments are for your output video or my output video?
If your output video is different from mine, could you share it via forum private message?

about “at 00:08 (ID=22), 00:30 (ID=43)” 's issue, please refer to nvdsanalytics , nvdsanalytics plugin will check the bottom center coordinate(x_left + width/2, y_top + height). when the person 's bottom center coordinate passed the crossing line, the bbox disappeared because the person’s body left the frame partially, as a result, nvdsanalytics did not count.

Thanks. I have the same understanding.

9

I tested restnet10 and peoplenet models based on the configuration and video you shared. I can’t see that “00:15 (ID=22),…” timestamp and trackerid in my output video. " 00:08 (ID=22), 00:30 (ID=43)" is my test video result.

ok, will share.

10

Thank you for sharing.

PeopleNet model I use is https://api.ngc.nvidia.com/v2/models/org/nvidia/team/tao/peoplenet/deployable_quantized_v2.6.1/files?redirect=true&path=resnet34_peoplenet_int8.etlt.

As I mentioned in my personal message, I used peoplenet with resnet34 backbone. I think this makes the results different.
Could you please try with the model I used?

I also noticed that in your video the color of a counted bounding box does not change.
Is there any other difference between your and my setups? For example, I’m running DeepStream with version 6.0.1 on Jetson.

11

yes, I tested restnet10 and resnet34 you shared. both get the same result, at 00:08 (ID=22), the person passed the line, but nvdsanalytics did not count.

I tested on deesptream6.2 xavier and deepstram6.3 T4. As I said in the email, you can change the crossing line’s positon to improve.

12

Thanks, I understand the situation! I will put your advise to good use.

Now I suspect false counting is caused due to my DeepStream version.
I am afraid that it is difficult to immediately update DeepStream in my Jetson.
Would it be possible for you to test on DeepStream6.0.1 and Jetson Xavier?

13

image
image487×601 37.4 KB

it is related to the detector model. if you don’t want to change line’s position, you can use other models with high accuracy.
Take timestamp “00:08” for example, when the person’s bottom center coordinate passed the crossing line, the body was covered. both restnet10 and resnet34 can’t give bbox, but yolov7 can give the bbox. with the bbox, the count added one finally .

14

Thank you.
But I think this does not account for a false count for person ID=1 at “00:18” in my output video, for example. He seems to be detected and tracked successfully.
This false count does not occur in your output video, so I guess the issue is related to DeepStream version.

15

please refer to nvdsanalytics for the meaning of “extended”, the person with id 1 did not pass the line if extended is 0.

16

I set “extended=0” in nvdsanalytics config, which I shared in personal message. In spite of that, the false count occurs.

17

image
image1254×742 39.8 KB

About the “false count at 00:18” issue. we can see the person with ID1 bypassed the crossing line and the count was added incorrectly. I can reproduce this issue. it is because nvdsanalytics was using the start object and current object to get the moving line when fps is low. the video you shared only has about 2 fps. we will continue to check.

18

Thank you for checking.

it is because nvdsanalytics was using the start object and current object to get the moving line when fps is low.

I have two questions:

  • Do you mean that the way to get moving lines changes depending on fps?
  • Are the start and current objects always in consecutive two frames? Or can the frame of the current object be far after the frame of the start object?

we will continue to check.

Thanks. Please let me know if there is any progress.

19

yes, low fps means low objects count. if fps is low, the start objects will be used to get moving line, if fps is high, the recent consecutive objects will be used.

20

Thank you for your quick response.

I have further questions:

  • Is there any parameter in config files to specify the fps threshold or something related to it?
  • In the case of low fps, what is the start object? the object at the beginning of the tracking?
22

[property]
…
obj-cnt-win-in-ms=2

gst-nvdsanalytics is opensource in ds6.3, the code path is /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdsanalytics, you can set “obj-cnt-win-in-ms=2” in nvdsanalytics 's configuration file. but please apply this patch first because of the bug.
1.diff (817 Bytes)
here are the detailed steps:

  1. apply the patch, then rebuild according to the readme.
  2. replace -rw-r–r-- 1 root root 93048 Aug 20 15:33 /opt/nvidia/deepstream/deepstream/lib/gst-plugins/libnvds_dsanalytics.so with new libnvds_dsanalytics.so.

yes.

1 Like
23

Thank you very much.

gst-nvdsanalytics is opensource in ds6.3

This is good news!

I will try to check using the patch, and let you know if there is any progress.