Yolov4_deepstream is not working with nvdanaytics

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)agx xavier
**• DeepStream Version 6.0
**• TensorRT Version 8.2.1.8

I am running the pipeline with commandine

sudo gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-dsexample-huz/Multitarget-tracker/data/Wallnut_Creek/cam_one/test1.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=640 height=480 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvtracker display-tracking-id=1 enable-past-frame=1 ll-lib-file = /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so ll-config-file = /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! nvdsanalytics config-file= /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=RGBA’ ! dsexample full-frame=0 blur-objects=1 ! nvdsosd ! nvegltransform ! nveglglessink

Preformatted text when I replace config_infer_primary.txt with yolov4 Preformatted text deepstream config_infer_primary_yoloV4.txt. The nvdsanalytics stop counting the objects which cross the line

config_nvdsanalytics.txt

[property]
enable=1
#Width height used for configuration to which below configs are configured
config-width=640
config-height=480
osd-mode 0: Dont display any lines, rois and text
1: Display only lines, rois and static text i.e. labels
2: Display all info from 1 plus information about counts
osd-mode=2
Set OSD font size that has to be displayed
display-font-size=12

#Per stream configuration
[roi-filtering-stream-0]
#enable or disable following feature
enable=1
#ROI to filter select objects, and remove from meta data
roi-RF=199;124;397;115;415;156;216;169
#remove objects in the ROI
inverse-roi=0
class-id=0

#Per stream configuration
[roi-filtering-stream-2]
#enable or disable following feature
enable=0
#ROI to filter select objects, and remove from meta data
roi-RF=295;143;279;334;342;213;56;428
#remove objects in the ROI
inverse-roi=0
class-id=0

[overcrowding-stream-1]
enable=0
roi-OC=295;143;279;334;342;213;56;428
#no of objects that will trigger OC
object-threshold=3
class-id=-1

[line-crossing-stream-0]
enable=1
#Label;direction;lc
#line-crossing-Str=789;672;1084;900;851;773;1203 ;732
line-crossing-Str_lane_2 =284;143;330;202;280;124;189;129
line-crossing-Str_lane_3 =361;123;489;212;394;124;314;137
line-crossing-Trigger =271;51;322;77;271;64;328;62
line-crossing-Right =521;118;567;116;553;98;556;117
line-crossing-Left =182;75;115;78;136;38;136;80
line-crossing-Trigger_2 =461;86;430;106;383;86;446;111
line-crossing-Right_lane_4 =277;80;306;75;315;86;355;79
line-crossing-Left_lane_flag =184;61;153;66;162;53;183;69

class-id=0
#extended when 0- only counts crossing on the configured Line
#1- assumes extended Line crossing counts all the crossing
extended=0
#LC modes supported:
#loose : counts all crossing without strong adherence to direction
#balanced: Strict direction adherence expected compared to mode=loose
#strict : Strict direction adherence expected compared to mode=balanced
mode=loose

[direction-detection-stream-0]
enable=1
#Label;direction;
direction-South=284;840;360;662;
#direction-North=1106;622;1312;701;
class-id=0

how i would be able to count objects with nvdsanalytics and inference with yolov4.

#3

Hi @user158435
Sorry for delay! Could you share us this video to align our repo?

Thanks!

#4

Here is example for caffe model
#and how i am running it

sudo gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvtracker display-tracking-id=1 enable-past-frame=1 ll-lib-file = /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so ll-config-file = /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! nvdsanalytics config-file= /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=RGBA’ ! nvdsosd ! nvegltransform ! nveglglessink

here is same video but with yolov4
#command for running pipeline

sudo gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_yolov4_with_nvdspreprocess/config_infer_primary_yoloV4.txt ! nvtracker display-tracking-id=1 enable-past-frame=1 ll-lib-file = /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so ll-config-file = /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! nvdsanalytics config-file= /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=RGBA’ ! nvdsosd ! nvegltransform ! nveglglessink

config_infer_primary_yoloV4.txt

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=yolov4_-1_3_416_416_dynamic.onnx_b16_gpu0_fp16.engine
onnx-file=./yolov4_-1_3_416_416_dynamic.onnx
labelfile-path=labels.txt
batch-size=16

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=80
gie-unique-id=1
network-type=0
is-classifier=0

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV4
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
#scaling-filter=0
#scaling-compute-hw=0

[class-attrs-all]
nms-iou-threshold=0.6
pre-cluster-threshold=0.4

#5

so, the difference is:
the workable pipeline uses
“nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt”

the not-workable pipeline uses:
“nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_yolov4_with_nvdspreprocess/config_infer_primary_yoloV4.txt”

right?

did you modify /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt or /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_yolov4_with_nvdspreprocess/config_infer_primary_yoloV4.txt ?

#6

Yeah workable pipeline uses
“nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt”

but what changes should I made in config_infer_primary_yoloV4.txt to make it work with workable pipeline . Because there is no guidance available online.Please guide me.
Thanks your reply

#7

Both use the same config_nvdsanalytics.txt, right?

Can you also /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt so that we can do some comparison beween your /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt and your /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_yolov4_with_nvdspreprocess/config_infer_primary_yoloV4.txt?

#8

Yes I am using same config_nvdsanalytics.txt for running both pipelines.

here is config_nvdsanalutics.txt

[property]
enable=1
config-width=1920
config-height=1080
osd-mode=0
display-font-size=12
[roi-filtering-stream-0]
enable=1
#roi-RF=295;643;579;634;642;913;56;828
roi-RF=295;643;579;634;642;913;56;828
inverse-roi=0
class-id=0
[overcrowding-stream-1]
enable=0
roi-OC=295;643;579;634;642;913;56;828
#no of objects that will trigger OC
object-threshold=3
class-id=-1
[line-crossing-stream-0]
enable=1
line-crossing-straight_count = 636;497;974;613;1006;510;589;578;
class-id=0
extended=0
mode=loose
[direction-detection-stream-0]
enable=1
direction-South=284;840;360;662;
class-id=0
[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=1
sync=0
bitrate=2000000
output-file=out.mp4
source-id=0

#9

Sorry! I mean share with me these two files:

/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt
/opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_yolov4_with_nvdspreprocess/config_infer_primary_yoloV4.txt

#10

config_infer_primary.txt

config_infer_primary.txt (4.0 KB)

config_infer_primary_yoloV4.txt

config_infer_primary_yoloV4.txt (2.9 KB)

#11

The reason of this issue should be you set “class-id=0” filter in analytics config file.
For /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt, “class-id=0” is car, but for YoloV4, Car is “class-id=2” as yolov4_deepstream/labels.txt at master · NVIDIA-AI-IOT/yolov4_deepstream · GitHub

#12

I will try it than update

#13

Thanks for your help!
one thing I want to ask what if i want to calculate multiple class ids crossing the line is it possible

#14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.