Output of deepstream app model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, , ,
1

• Hardware Platform: Jetson AGX Xavier
• DeepStream Version: 6.2
• JetPack Version: 5.0.2
• TensorRT Version: 8.4.1
• Issue Type: Informations

Hello,

I’d like to create a detection and classification model similar to the deepstream-app1 example but I don’t understand what the output form of this model is. This will allow me to avoid using a Triton server to carry out custom parsing in C.

Could you tell me the output format of the model used (an example would be really helpful)?

Thank you.

5

What do you mean?
Which model do you talking about? Do you know about the model itself?
Do you want to know what the model output looks like in DeepStream?

6

Yes because I tried to implement my model which have the next dimensions:
Output ‘boxes’ shape: [1, 8400, 4]
Output ‘scores’ shape: [1, 8400, 1]
Output ‘classes’ shape: [1, 8400, 1]

Where 8400 is the maximum number of elements in the model’s prediction.

I developped the custom parser in C for the bounding box but I have the next warning:
WARNING: Num classes mismatch. Configured:1, detected by network: 8400

7

How did you implement your custom parser? Can you provide the code and configurations?

We have provide many customization output parser samples: E.G. /opt/nvidia/deepstream/deepstream/sources/objectDetector_Yolo

8

I implement with the example here: /opt/nvidia/deepstream/deepstream-6.2/sources/libs/customparser_detector_classifier
nvdsinfer_custombboxparser.cpp (28.0 KB)
nvdsinfer_customclassifierparser.cpp (4.3 KB)

My model use YOLOv8 but I didn’t find a parser for that kind of model. The custom resnet parser should work no ?

9
  1. According to the output layers, the model is a detector but not a classifier
  2. The output dimension is CHW, c=1, h=8400, w=4 or 1. Why did you use “c” as the class number? You implementation is wrong.
  3. You can refer to https://github.com/NVIDIA-AI-IOT/yolo_deepstream/blob/main/deepstream_yolo/nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp which is the yolov7 & yolov4 post-processing for bbox parsing.
10

I only have one class actually.

11

The outputs are bboxs and the scores

12

The output is that

13

The yolovx output like this. It is OK

14

Detectors output class ids too. NVIDIA DeepStream SDK API Reference: NvDsInferObjectDetectionInfo Struct Reference | NVIDIA Docs

15

I updated the code from your page and got this:
nvdsinfer_custombboxparser.cpp (5.9 KB)

When I’m running my code, I have nothing from the osd_sink_pad_buffer_probe function. The content of the osd_sink_pad_buffer_probe function is the same as osd_sink_pad_buffer_probe function from deepstream app test1.

I tried to check if something is detected from the custom parser by printing the bbox coordinates, the score and the identifier and I got something like that:
272.723 543.192 349.794 168.54 0 0
317.451 548.294 320.962 157.911 0 0
355.035 556.413 339.684 142.038 0 0
388.881 565.243 357.286 124.902 0 0
434.759 572.433 385.868 112.666 0 0
452.356 576.059 339.327 108.43 0 0
475.126 574.797 309.198 115.935 5.96046e-08 0
496 570.575 274.378 129.183 2.23517e-06 0
519.355 562.291 239.708 150.788 2.46763e-05 0
545.718 555.918 192.082 166.719 0.000124365 0
561.117 551.262 155.976 170.549 0.000211596 0
49.1543 548.063 95.0536 184.445 0.000128806 0
71.7683 541.537 139.457 196.301 0.000426561 0
99.8329 546.757 197.885 185.804 0.000261277 0
120.235 581.546 230.948 115.809 3.72827e-05 0
145.793 600.763 256.244 77.8271 1.72853e-06 0
170.909 603.068 281.922 72.2293 1.49012e-07 0
193.645 600.278 301.613 76.8993 2.98023e-08 0
212.528 598.74 321.755 79.9933 2.98023e-08 0
232.508 601.394 334.665 74.9086 2.98023e-08 0
273.474 608.596 300.942 60.1772 0 0
312.039 611.04 285.198 54.7383 0 0
346.74 611.475 296.673 53.7321 0 0
381.238 611.444 309.542 54.0078 2.98023e-08 0
426.073 610.521 333.835 55.7668 8.9407e-08 0
450.674 610.212 307.931 56.952 1.49012e-07 0
475.57 607.923 289.107 61.6656 2.68221e-07 0
498.764 601.86 259.078 73.8761 1.3113e-06 0
524.851 586.943 227.829 104.701 1.12057e-05 0
552.134 576.179 181.468 125.865 3.60906e-05 0
566.756 568.394 146.125 143.317 1.19805e-05 0

My problem is that there is bbox found from the custom parser but there are not draw on the screen. I didn’t put threshold value in the pgie config file:
detector_classifier_nemesis_pgie_config.txt (624 Bytes)

16

Apparently the values of “scores” are very low. Please consult the guy who provided the model to you whether such output is correct.

17

Shouldn’t bboxes be displayed despite a low score?

18

There is clustering algorithm after bbox parsing. Please check your settings of the clustering.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.