Get classification output from msgconv

satyamjay030 · April 24, 2024, 6:36am

• Hardware Platform (GPU)
• DeepStream 6.4
• NVIDIA GPU Driver Version (550.54.14)
• Issue Type( questions )

I have developed the LPR pipeline following the GitHub - NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream .
I have also added an additional component for sending messages to a kafka-topic. However I only get object bounding box information and not number plate in the output messages. Kindly nudge in the right direction.

Pipeline.yaml:-

################################################################################
# Copyright (c) 2021-2023, NVIDIA CORPORATION. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
#
################################################################################
 
dependencies:
  - extension: NvDsBaseExt
    uuid: 56d7e3ec-62c6-4652-bcc8-4f1c3b00df03
    version: 1.4.1
  - extension: NvDsCloudMsgExt
    uuid: d16b1366-dba2-47f6-95fc-b09b39f885de
    version: 1.4.0
  - extension: NvDsInferenceExt
    uuid: 0b02963e-c24e-4c13-ace0-c4cdf36c7c71
    version: 1.4.0
  - extension: NvDsInferenceUtilsExt
    uuid: 27856a43-5ad4-4d8e-be36-0ec2cf9bbb58
    version: 1.4.0
  - extension: NvDsSourceExt
    uuid: a632d022-3425-4848-9074-e6483ef74366
    version: 1.4.0
  - extension: NvDsMuxDemuxExt
    uuid: 89b8398c-5820-4051-835c-a91f2d49766b
    version: 1.4.0
  - extension: NvDsOutputSinkExt
    uuid: 3fc9ad87-03e7-47a8-bbfc-8501c3f7ff2f
    version: 1.4.0
  - extension: NvDsVisualizationExt
    version: 1.4.0
    uuid: 25903cd4-fc5c-4139-987b-47bb27e8b424
  - extension: NvDsSampleExt
    uuid: 44a711e4-8519-4a68-81e8-be7ee4af3ff0
    version: 1.4.0
  - extension: NvDsSampleModelsExt
    uuid: 909dc1ea-c654-44c1-97a3-0b8fec12141a
    version: 1.4.0
---
components:
- name: single_source_input
  parameters:
    audio-out-%u: audio_output
    # uri: file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
    # uri: file:///opt/nvidia/deepstream/deepstream-6.4/reference_graphs/deepstream-test1/test_1.mp4
    uri: file:///opt/nvidia/deepstream/deepstream-6.4/in.mp4
    video-out-%u: video_output
  type: nvidia::deepstream::NvDsSingleSrcInput
- name: audio_output
  type: nvidia::deepstream::NvDsDynamicOutput
- name: video_output
  type: nvidia::deepstream::NvDsDynamicOutput
name: single_source_input
ui_property:
  position:
    x: -86.51969146728516
    y: -6.062990188598633
  state: 0
---
components:
- name: streammux
  parameters:
    batch-size: 1
    # height: 1080
    height: 480
    video-in-%u: video_input
    video-out: video_output
    # width: 1920
    width: 848
  type: nvidia::deepstream::NvDsStreamMux
- name: video_input
  type: nvidia::deepstream::NvDsOnRequestInput
- name: video_output
  type: nvidia::deepstream::NvDsStaticOutput
name: streammux
ui_property:
  position:
    x: 153.81895446777344
    y: 33.13385772705078
  state: 0
---
components:
- name: nv_ds_connection45..d320
  parameters:
    source: single_source_input/video_output
    target: streammux/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..9208
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
  - name: object_detector
    parameters:
      config-file-path: '/opt/nvidia/deepstream/deepstream-6.4/deepstream_lpr_app/deepstream-lpr-app/lpd_yolov4-tiny_ch.txt'
      video-in: video_input
      video-out: video_output
    type: nvidia::deepstream::NvDsInferVideo
  - name: video_input
    type: nvidia::deepstream::NvDsStaticInput
  - name: video_output
    type: nvidia::deepstream::NvDsStaticOutput
name: object_detector
ui_property:
  position:
    x: 370.43316650390625
    y: 4.251962661743164
  state: 0
---
components:
- name: nv_ds_connection49..9550
  parameters:
    source: streammux/video_output
    target: object_detector/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..2c50
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
  - name: lpr
    parameters:
      config-file-path: '/opt/nvidia/deepstream/deepstream-6.4/deepstream_lpr_app/deepstream-lpr-app/lpr_config_sgie_ch.txt'
      # unique-id: 2
      video-in: video_input
      video-out: video_output
    type: nvidia::deepstream::NvDsInferVideo
  - name: video_input
    type: nvidia::deepstream::NvDsStaticInput
  - name: video_output
    type: nvidia::deepstream::NvDsStaticOutput
name: lpr
ui_property:
  position:
    x: 831.5119018554688
    y: 37.25985336303711
  state: 0
---
components:
  - name: nv_ds_connection17..ac88
    parameters:
      source: object_detector/video_output
      target: lpr/video_input
    type: nvidia::deepstream::NvDsConnection
name: node..aeb8
ui_property: {}
---
components:
- name: onscreen_display
  parameters:
    video-in: video_input
    video-out: video_output
  type: nvidia::deepstream::NvDsOSD
- name: video_input
  type: nvidia::deepstream::NvDsStaticInput
- name: video_output
  type: nvidia::deepstream::NvDsStaticOutput
name: onscreen_display
ui_property:
  position:
    x: 586.0159912109375
    y: 22.818899154663086
  state: 0
---
components:
- name: nv_ds_connection53..8b00
  parameters:
    source: lpr/video_output
    target: onscreen_display/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..8cf8
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: tee_1
  parameters:
    in: input
    out-%u: output
  type: nvidia::deepstream::NvDsTee
- name: input
  type: nvidia::deepstream::NvDsStaticInput
- name: output
  type: nvidia::deepstream::NvDsOnRequestOutput
name: tee_1
ui_property:
  position:
    x: 809.850341796875
    y: 42.417572021484375
  state: 0
---
components:
- name: nv_ds_connection57..47f0
  parameters:
    source: onscreen_display/video_output
    target: tee_1/input
  type: nvidia::deepstream::NvDsConnection
name: node..8cf8
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: video_renderer
  parameters:
    video-in: video_input
    output-file: output.mp4
  type: nvidia::deepstream::NvDsFileOut
- name: video_input
  type: nvidia::deepstream::NvDsStaticInput
name: video_renderer
ui_property:
  position:
    x: 1083.1968994140625
    y: -44.2282600402832
  state: 0
---
components:
- name: nv_ds_connection60..3e48
  parameters:
    source: tee_1/output
    target: video_renderer/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..4550
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: message_converter_broker
  parameters:
    in: input
    debug-payload-dir: /tmp/payload-dir
    config: /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test4/dstest4_msgconv_config.txt
    msg2p-newapi: True
    multiple-payloads: True
    # topic: test-topic
  type: nvidia::deepstream::NvDsMsgConverter
- name: input
  type: nvidia::deepstream::NvDsStaticInput
name: message_converter_broker
ui_property:
  position:
    x: 1085.259765625
    y: 145.56710815429688
  state: 0
---
components:
- name: nv_ds_connection63..0198
  parameters:
    source: tee_1/output
    target: message_converter_broker/input
  type: nvidia::deepstream::NvDsConnection
name: node..4b70
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: scheduler
  type: nvidia::deepstream::NvDsScheduler
name: scheduler
ui_property:
  position:
    x: 178.57481384277344
    y: 316.79534912109375
  state: 0
---
components:
- name: sample_probe_message_meta_creation
  parameters:
    # generate-dummy-data: true
    probe-connector: probe_connector
  type: nvidia::deepstream::NvDsSampleProbeMessageMetaCreation
- name: probe_connector
  type: nvidia::deepstream::NvDsProbeConnector
name: sample_probe_message_meta_creation
ui_property:
  position:
    x: 697.4172973632812
    y: 431.29132080078125
  state: 0
---
components:
- name: nv_ds_probe2..cc88
  parameters:
    io: lpr/video_output
    probe-connector: sample_probe_message_meta_creation/probe_connector
  type: nvidia::deepstream::NvDsProbe
name: node..13c8
ui_property: {}

Output that I am getting:-

{
 "messageid" : "0defcf33-b1c8-4f94-afe6-75fa7228022f",
 "mdsversion" : "1.0",
 "@timestamp" : "2024-04-24T06:20:43.078Z",
 "place" : null,
 "sensor" : null,
 "analyticsModule" : null,
 "object" : {
   "id" : "18446744073709551615",
   "speed" : 0.0,
   "direction" : 0.0,
   "orientation" : 0.0,
   "" : {
     "confidence" : 0.33419784903526306
   },
   "bbox" : {
     "topleftx" : 241,
     "toplefty" : 185,
     "bottomrightx" : 540,
     "bottomrighty" : 228
   },
   "location" : {    },
   "coordinate" : {    }
 },
 "event" : {
   "id" : "8709b57c-4e54-440d-babe-d9342a0ccca4",
   "type" : ""
 },
 "videoPath" : ""
}

yuweiw · April 24, 2024, 8:30am

You need to customize your own NVDS_EVENT_MSG_META and add the number plate to the metadata.

satyamjay030 · April 24, 2024, 9:56am

The OSD component displays the license plate number. So, I assumed that the metadata of classifier must be attached.

yuweiw · April 24, 2024, 10:55am

Yes. It has attached to the display metadata. But you want to send that through the msgbroker. So you need to attach that to the NVDS_EVENT_MSG_META type. You can refer to our source code deepstream-test5 to learn how to attach that to the NVDS_EVENT_MSG_META type.

satyamjay030 · April 27, 2024, 6:49am

I decided to modify /opt/nvidia/deepstream/sources/libs/nvmsgconv and rebuild it. However I am not able to build it(inside DeepStream-6.4 container). To be sure, I tried to build unmodified source, that is failing as well.
Steps I performed:-

sudo apt-get install libglib2.0-dev libjson-glib-dev uuid-dev libyaml-cpp-dev protobuf-compiler
make

PFA the error I am getting
err.txt (622.1 KB)

yuweiw · April 28, 2024, 1:25am

Could you use the command below to change the env path and try?

export PATH="/opt/proto/bin:$PATH"

yingliu · May 21, 2024, 5:30am

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

system · June 4, 2024, 5:31am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.