How to properly use the Gst-nvdsAnalytics plug-in in deepstream-yolo and transfer the inspection result data externally?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson)
• DeepStream Version 6.3
• JetPack Version (valid for Jetson only) 5.1.2

How to correctly use the roi-filtering function in the Gst-nvdsanalytics plugin in deepstream-yolo and transfer the result data of the detection to the outside?

First question

I’m using the Gst-nvdsanalytics feature in deepstream and I’ve successfully implemented it Overcrowding Detection, Direction Detection, Line Crossing, but it seems that roi-filtering doesn’t work, why is this?

Could it be that there is a problem with my configuration parameters? I’ve tried it many times and it doesn’t show up in the output, why is this?

This is my config_nvdsanalytics, configuration parameters, and I set enable=1, but it doesn’t seem to work.

[property]
enable=1
config-width=1920
config-height=1080
osd-mode=2
display-font-size=12

[roi-filtering-stream-0]
#enable or disable following feature
enable=1
#ROI to filter select objects, and remove from meta data
roi-RF=444;357;995;357;1225;424;1860;882;4;882;
#remove objects in the ROI
inverse-roi=0
class-id=-1

[overcrowding-stream-0]
enable=0
roi-OC=444;357;995;357;1225;424;1860;882;4;882;
object-threshold=15
class-id=-1

[line-crossing-stream-0]
enable=0
line-crossing-Entry=566;611;428;914;158;738;854;732
line-crossing-Exit=1407;880;1227;620;895;732;1672;732
class-id=-1
extended=0
mode=loose

[direction-detection-stream-0]
enable=0
#Label;direction;
direction-South=566;611;428;914;
direction-North=1407;880;1227;620;
class-id=-1

This is my deepstream_app_config parameter configuration! I guess they should be fine?

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=2
uri=file:///home/cvai/lintao/ultralytics/main/video/LT_cloudy_morning_sub_output.mp4
#uri=file:///home/cvai/lintao/ultralytics/main/video/traffic.mp4
num-sources=1
num-extra-surfaces=10
drop-frame-interval=0
nvbuf-memory-type=0
gpu-id=0

[sink0]
enable=1
type=2
sync=0
gpu-id=0
nvbuf-memory-type=0

[tracker]
enable=1
tracker-width=640
tracker-height=640
ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=/home/cvai/lintao/ultralytics/DeepStream-Yolo-master/config_tracker_NvDCF_perf.yml
gpu-id=0
display-tracking-id=1
user-meta-pool-size=256

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=1
clock-x-offset=1000
clock-y-offset=30
clock-text-size=15
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1280
height=720
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV8.txt

[tests]
file-loop=0

[nvds-analytics]
enable=1
config-file=config_nvdsanalytics.txt

But the reality is that when I only set enable=1 in [roi-filtering-stream-0], there is no output, and even the ROI area is not displayed, as shown below:

If I set [overcrowding-stream-0] and [line-crossing-stream-0] to enable=1, it works!

So how should I achieve this? I refer to the example in the example as well as the official docs, my setup should be fine, but it doesn’t work?

If there is a problem, can you tell me how it should be set up correctly?

Second question

Can I get the detection data in it,
For example, the detection data of crowd detection in the figure above, as well as the data obtained by cross-line counting.

It just shows OC OverCrowding=False Count=10, Entry=10, etc
Can I get this data?

Because I want to show them on the web, but I can’t figure out how to set it up.
Does deepstream have a plug-in or function switch setting to get this data for me to process further.

Actually, I was thinking about whether I could get this data in real time for display, or write it into a log so that I could read it in python, or some other way to get it

  1. the coordinates of ROI and overcrowding are the same.
  2. these analytics results are saved in meta data. please refer to parse_nvdsanalytics_meta_data of \opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-nvdsanalytics-test\deepstream_nvdsanalytics_meta.cpp for how to get these information from NvDsAnalyticsFrameMeta and NvDsUserMeta.

I modified them to make sure the coordinates were different, but it still didn’t work, I guess it shouldn’t be the problem.

Using your configuration file, I can’t reporduce this issue on DS6.3. nvdsanalytics is opensource. You can add logs in attach_framemeta_analytics_metadata of \opt\nvidia\deepstream\deepstream\sources\gst-plugins\gst-nvdsanalytics\gstnvdsanalytics.cpp to check if the ROI display meta is added.

Thank you for your patient reply!

I followed your instructions and checked the file. Please take a look; I believe this feature exists!

deepstream_nvdsanalytics_meta.txt (4.8 KB)

But I don’t seem to fully understand it, where is the output information in the code?

I don’t seem to see these outputs in the terminal, is it using this data to draw to the output screen?

Can I send this data out with WebSocket for further analysis, and if so, what should I do?

Thank you very much again!

parse_nvdsanalytics_meta_data is called by Gstreamer probe function. please refer to nvdsanalytics_src_pad_buffer_probe of opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-nvdsanalytics-test\deepstream_app_main.c. the analytics results are saved in meta. in parse_nvdsanalytics_meta_data, the code get information from usermeta of frames and objects. these information will be printed on the terminal, As “OC OverCrowding=False Count=10, Entry=10” shown.

I understand what you said, thank you very much! I’m trying to implement this roi-filtering feature right now because he’s not working at all

I just set [roi-filtering-stream-0] enable to 1 doesn’t work, why is that?

can you use deepstream-nvdsanalytics-test to reproduce this issue? To narrow down this issue, you may only try the following cfg.

[property]
enable=1
config-width=1920
config-height=1080
osd-mode=2
display-font-size=12

[roi-filtering-stream-0]
#enable or disable following feature
enable=1
#ROI to filter select objects, and remove from meta data
roi-RF=444;357;995;357;1225;424;1860;882;4;882;
#remove objects in the ROI
inverse-roi=0
class-id=-1

I found a big problem,
That is, I use opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt to configure, and it works fine,
config_nvdsanalytics.txt (1.7 KB)

[nvds-analytics]
#unique-id=0
enable=1
#config-file=config_nvdsanalytics_yolov8.txt
config-file=config_nvdsanalytics.txt

I only turned on one feature and it worked

[property]
enable=1
#Width height used for configuration to which below configs are configured
config-width=1920
config-height=1080
#osd-mode 0: Dont display any lines, rois and text
#         1: Display only lines, rois and static text i.e. labels
#         2: Display all info from 1 plus information about counts
osd-mode=2
#Set OSD font size that has to be displayed
display-font-size=12

## Per stream configuration
[roi-filtering-stream-0]
#enable or disable following feature
enable=1
#ROI to filter select objects, and remove from meta data
roi-RF=295;643;579;634;642;913;56;828
#remove objects in the ROI
inverse-roi=0
class-id=-1

## Per stream configuration
[roi-filtering-stream-2]
#enable or disable following feature
enable=0
#ROI to filter select objects, and remove from meta data
roi-RF=295;643;579;634;642;913;56;828
#remove objects in the ROI
inverse-roi=1
class-id=0

[overcrowding-stream-1]
enable=0
roi-OC=295;643;579;634;642;913;56;828
#no of objects that will trigger OC
object-threshold=3
class-id=-1

[line-crossing-stream-0]
enable=0
#Label;direction;lc
#line-crossing-Entry=1072;911;1143;1058;944;1020;1297;1020;
line-crossing-Exit=789;672;1084;900;851;773;1203;732
class-id=0
#extended when 0- only counts crossing on the configured Line
#              1- assumes extended Line crossing counts all the crossing
extended=0
#LC modes supported:
#loose   : counts all crossing without strong adherence to direction
#balanced: Strict direction adherence expected compared to mode=loose
#strict  : Strict direction adherence expected compared to mode=balanced
mode=loose

[direction-detection-stream-0]
enable=0
#Label;direction;
direction-South=284;840;360;662;
direction-North=1106;622;1312;701;
class-id=0

And if I copy its parameter settings, it doesn’t work properly, why?

[property]
enable=1
config-width=1920
config-height=1080
osd-mode=2
display-font-size=12

## Per stream configuration
[roi-filtering-stream-0]
#enable or disable following feature
enable=1
#ROI to filter select objects, and remove from meta data
roi-RF=295;643;579;634;642;913;56;828
#remove objects in the ROI
inverse-roi=0
class-id=-1

[roi-filtering-stream-0]
enable=0
roi-RF=450;384;1102;370;1908;995;9;1001;
inverse-roi=1
class-id=0

[overcrowding-stream-0]
enable=0
roi-OC=444;357;995;357;1225;424;1860;882;4;882;
object-threshold=15
class-id=-1

[line-crossing-stream-0]
enable=0
line-crossing-Entry=566;611;428;914;158;738;854;732
line-crossing-Exit=1407;880;1227;620;895;732;1672;732
class-id=-1
extended=0
mode=loose

[direction-detection-stream-0]
enable=0
#Label;direction;
direction-South=566;611;428;914;
direction-North=1407;880;1227;620;
class-id=-1

config_nvdsanalytics_yolov8.txt (866 Bytes)

Shouldn’t they be the same?

Thank you again for your patience, I already know that the problem is in my file configuration
[roi-filtering-stream-0]、[roi-filtering-stream-0]
I delete one and it works!

Thanks for helping me solve the first problem, it seems pretty stupid. Now I think I’m going to solve my second problem!

I’d like to ask if I want to get this test data, if I modify it\opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-nvdsanalytics-test\deepstream_nvdsanalytics_meta.cpp

Should I recompile it? Is this compiling?
/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-app/

If not, how exactly should it be achieved?

please refer to \opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-nvdsanalytics-test\README for how to build.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.