Gst-nvdsanalytics roi filtering objects presence inconsistent

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

I have been testing the nvdsanalytics element to determine the presence of persons in a region, however I am having inconsistent results. It was my understanding that the analytics will determine that an object is in the ROI if the bottom center coordinate is inside the defined region, however I get object bounding boxes classified in the ROI when the bottom line is outside the region and bounding boxes not detected in the ROI when the bottom line is inside the region.

Following example shows how in some occasions the man is detected inside the RT ROI when the bottom bounding box line is outside the region:

On the other hand next video shows the man is not detected in ROI even when the bottom line of the bounding box is inside the ROI-RF:

Also I am getting different results depending on the detector that I use, I tested with peoplenet and rtmdet, with the same video and pipeline, even tough I get similar bounding boxes, the analytics ROI detections are different, see below:

  • rtmdet

  • peoplenet

I tested with the following gstreamer pipeline:
gst-launch-1.0 filesrc location= test-walking.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! .sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 live-source=1 ! nvinfer config-file-path= peoplenet/config_infer_primary_peoplenet.txt ! nvdsanalytics config-file=config.txt ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! avimux ! filesink location=/tmp/test.avi

Using the following analytics configuration file:

[property]
enable=1
config-width=1920
config-height=1080
osd-mode=2
display-font-size=12

[roi-filtering-stream-0]
enable=1
roi-RF=0;643;1920;643;1920;890;0;890
inverse-roi=0
class-id=-1

Is my understanding of how the analytics element should work wrong? Do I have a configuration error?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson AGX Orin
• DeepStream Version 7.0
• JetPack Version (valid for Jetson only) 6.0
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Melissa Montero
www.ridgerun.ai

3

Thanks for the sharing! I am checking.

4

could you share the test videos? Thanks! let’s test the same videos. which peoplenet version are you testing?

5

Hi fanzh,

You can find the sample videos here (actually it happens mostly with any video):

Video 1

Video 2

The version is

Latest Version
pruned_quantized_decrypted_v2.3.4

In any case, after testing with RTM and peoplenet we determined that it does not matter the detector used (or version) these issues happen anyway.

Best Regards,
Marco Madrigal

6

Using your configuration file and peoplenet v2.3.4 model, I tested deepstream-nvdsanalytics-test on Orin with DS7.1. my test results are correct. Here are the log log1028.txt (32.1 KB) and screenshots. when he bottom center coordinate (x_left + width/2, y_top + height) entered the ROI, the count will increase.

Capture1
Capture11153×1264 118 KB
Capture2
Capture21445×1147 150 KB

7

Hi fanzh,

Thanks for looking at this. We are using DS7.0 and a GStreamer pipeline, would this make any difference? Anyway, we will look at this example but just wondering if there might be any different with the DS version and/or using a GStreamer pipeline instead of the sample application.

Best Regards,
Marco Madrigal

8

Thanks for the sharing! nvdsanalytics plugin is opensource. there is no difference between DS7.0 and DS7.1 in nvdsanalytics low-level lib.
could you use DS7.1 to reproduce this accuracy issue? Thanks! please refer to my last comment. I used your test video and configuration file without modifying the code.

9

Hi @fanzh ,

We completed the tests. Here is what we found:

  1. We tested the sample application you suggested with the videos and we reproduced your results. Using the application the ROI detects the bounding box correctly.

  2. When checking the application, we realized it uses the tracker after the detector.

  3. When we add the tracker to the original GStreamer pipeline (as shown below) the results are the same than when using the application i.e. the detection within the ROI is correct.

gst-launch-1.0 filesrc location= test-walking1.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! .sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 ! nvinfer config-file-path= peoplenet/config_infer_primary_peoplenet.txt ! nvtracker ll-lib-file=“/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so” ll-config-file=“/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml” tracker-width=640 tracker-height=480 ! nvdsanalytics config-file=config.txt ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! avimux ! filesink location=/tmp/test.avi

  1. The following videos show the difference between using the tracker and not using it with the sample videos:

For our use case, using the tracker is not necessary, we only need to detect whether the detection occurs within the ROI or not. That said, our questions is:

Is it required to use the tracker to get the analytics detecting the bounding box in the ROI correctly?

You can easily test this by removing the tracker in your application or using the GStreamer pipeline without tracker.

Best Regards,
Marco

10

without tracker, using the same video and cfg, I can reproduce this issue. that is, when the bottom center coordinate (x_left + width/2, y_top + height) did not enter the ROI, the ROI count increased. we are checking then get back to you. Thanks!