Get classification output from msgconv

• Hardware Platform (GPU)
• DeepStream 6.4
• NVIDIA GPU Driver Version (550.54.14)
• Issue Type( questions )

I have developed the LPR pipeline following the GitHub - NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream .
I have also added an additional component for sending messages to a kafka-topic. However I only get object bounding box information and not number plate in the output messages. Kindly nudge in the right direction.

Pipeline.yaml:-

################################################################################
# Copyright (c) 2021-2023, NVIDIA CORPORATION. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
#
################################################################################
 
dependencies:
  - extension: NvDsBaseExt
    uuid: 56d7e3ec-62c6-4652-bcc8-4f1c3b00df03
    version: 1.4.1
  - extension: NvDsCloudMsgExt
    uuid: d16b1366-dba2-47f6-95fc-b09b39f885de
    version: 1.4.0
  - extension: NvDsInferenceExt
    uuid: 0b02963e-c24e-4c13-ace0-c4cdf36c7c71
    version: 1.4.0
  - extension: NvDsInferenceUtilsExt
    uuid: 27856a43-5ad4-4d8e-be36-0ec2cf9bbb58
    version: 1.4.0
  - extension: NvDsSourceExt
    uuid: a632d022-3425-4848-9074-e6483ef74366
    version: 1.4.0
  - extension: NvDsMuxDemuxExt
    uuid: 89b8398c-5820-4051-835c-a91f2d49766b
    version: 1.4.0
  - extension: NvDsOutputSinkExt
    uuid: 3fc9ad87-03e7-47a8-bbfc-8501c3f7ff2f
    version: 1.4.0
  - extension: NvDsVisualizationExt
    version: 1.4.0
    uuid: 25903cd4-fc5c-4139-987b-47bb27e8b424
  - extension: NvDsSampleExt
    uuid: 44a711e4-8519-4a68-81e8-be7ee4af3ff0
    version: 1.4.0
  - extension: NvDsSampleModelsExt
    uuid: 909dc1ea-c654-44c1-97a3-0b8fec12141a
    version: 1.4.0
---
components:
- name: single_source_input
  parameters:
    audio-out-%u: audio_output
    # uri: file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
    # uri: file:///opt/nvidia/deepstream/deepstream-6.4/reference_graphs/deepstream-test1/test_1.mp4
    uri: file:///opt/nvidia/deepstream/deepstream-6.4/in.mp4
    video-out-%u: video_output
  type: nvidia::deepstream::NvDsSingleSrcInput
- name: audio_output
  type: nvidia::deepstream::NvDsDynamicOutput
- name: video_output
  type: nvidia::deepstream::NvDsDynamicOutput
name: single_source_input
ui_property:
  position:
    x: -86.51969146728516
    y: -6.062990188598633
  state: 0
---
components:
- name: streammux
  parameters:
    batch-size: 1
    # height: 1080
    height: 480
    video-in-%u: video_input
    video-out: video_output
    # width: 1920
    width: 848
  type: nvidia::deepstream::NvDsStreamMux
- name: video_input
  type: nvidia::deepstream::NvDsOnRequestInput
- name: video_output
  type: nvidia::deepstream::NvDsStaticOutput
name: streammux
ui_property:
  position:
    x: 153.81895446777344
    y: 33.13385772705078
  state: 0
---
components:
- name: nv_ds_connection45..d320
  parameters:
    source: single_source_input/video_output
    target: streammux/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..9208
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
  - name: object_detector
    parameters:
      config-file-path: '/opt/nvidia/deepstream/deepstream-6.4/deepstream_lpr_app/deepstream-lpr-app/lpd_yolov4-tiny_ch.txt'
      video-in: video_input
      video-out: video_output
    type: nvidia::deepstream::NvDsInferVideo
  - name: video_input
    type: nvidia::deepstream::NvDsStaticInput
  - name: video_output
    type: nvidia::deepstream::NvDsStaticOutput
name: object_detector
ui_property:
  position:
    x: 370.43316650390625
    y: 4.251962661743164
  state: 0
---
components:
- name: nv_ds_connection49..9550
  parameters:
    source: streammux/video_output
    target: object_detector/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..2c50
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
  - name: lpr
    parameters:
      config-file-path: '/opt/nvidia/deepstream/deepstream-6.4/deepstream_lpr_app/deepstream-lpr-app/lpr_config_sgie_ch.txt'
      # unique-id: 2
      video-in: video_input
      video-out: video_output
    type: nvidia::deepstream::NvDsInferVideo
  - name: video_input
    type: nvidia::deepstream::NvDsStaticInput
  - name: video_output
    type: nvidia::deepstream::NvDsStaticOutput
name: lpr
ui_property:
  position:
    x: 831.5119018554688
    y: 37.25985336303711
  state: 0
---
components:
  - name: nv_ds_connection17..ac88
    parameters:
      source: object_detector/video_output
      target: lpr/video_input
    type: nvidia::deepstream::NvDsConnection
name: node..aeb8
ui_property: {}
---
components:
- name: onscreen_display
  parameters:
    video-in: video_input
    video-out: video_output
  type: nvidia::deepstream::NvDsOSD
- name: video_input
  type: nvidia::deepstream::NvDsStaticInput
- name: video_output
  type: nvidia::deepstream::NvDsStaticOutput
name: onscreen_display
ui_property:
  position:
    x: 586.0159912109375
    y: 22.818899154663086
  state: 0
---
components:
- name: nv_ds_connection53..8b00
  parameters:
    source: lpr/video_output
    target: onscreen_display/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..8cf8
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: tee_1
  parameters:
    in: input
    out-%u: output
  type: nvidia::deepstream::NvDsTee
- name: input
  type: nvidia::deepstream::NvDsStaticInput
- name: output
  type: nvidia::deepstream::NvDsOnRequestOutput
name: tee_1
ui_property:
  position:
    x: 809.850341796875
    y: 42.417572021484375
  state: 0
---
components:
- name: nv_ds_connection57..47f0
  parameters:
    source: onscreen_display/video_output
    target: tee_1/input
  type: nvidia::deepstream::NvDsConnection
name: node..8cf8
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: video_renderer
  parameters:
    video-in: video_input
    output-file: output.mp4
  type: nvidia::deepstream::NvDsFileOut
- name: video_input
  type: nvidia::deepstream::NvDsStaticInput
name: video_renderer
ui_property:
  position:
    x: 1083.1968994140625
    y: -44.2282600402832
  state: 0
---
components:
- name: nv_ds_connection60..3e48
  parameters:
    source: tee_1/output
    target: video_renderer/video_input
  type: nvidia::deepstream::NvDsConnection
name: node..4550
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: message_converter_broker
  parameters:
    in: input
    debug-payload-dir: /tmp/payload-dir
    config: /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test4/dstest4_msgconv_config.txt
    msg2p-newapi: True
    multiple-payloads: True
    # topic: test-topic
  type: nvidia::deepstream::NvDsMsgConverter
- name: input
  type: nvidia::deepstream::NvDsStaticInput
name: message_converter_broker
ui_property:
  position:
    x: 1085.259765625
    y: 145.56710815429688
  state: 0
---
components:
- name: nv_ds_connection63..0198
  parameters:
    source: tee_1/output
    target: message_converter_broker/input
  type: nvidia::deepstream::NvDsConnection
name: node..4b70
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: scheduler
  type: nvidia::deepstream::NvDsScheduler
name: scheduler
ui_property:
  position:
    x: 178.57481384277344
    y: 316.79534912109375
  state: 0
---
components:
- name: sample_probe_message_meta_creation
  parameters:
    # generate-dummy-data: true
    probe-connector: probe_connector
  type: nvidia::deepstream::NvDsSampleProbeMessageMetaCreation
- name: probe_connector
  type: nvidia::deepstream::NvDsProbeConnector
name: sample_probe_message_meta_creation
ui_property:
  position:
    x: 697.4172973632812
    y: 431.29132080078125
  state: 0
---
components:
- name: nv_ds_probe2..cc88
  parameters:
    io: lpr/video_output
    probe-connector: sample_probe_message_meta_creation/probe_connector
  type: nvidia::deepstream::NvDsProbe
name: node..13c8
ui_property: {}

Output that I am getting:-

{
 "messageid" : "0defcf33-b1c8-4f94-afe6-75fa7228022f",
 "mdsversion" : "1.0",
 "@timestamp" : "2024-04-24T06:20:43.078Z",
 "place" : null,
 "sensor" : null,
 "analyticsModule" : null,
 "object" : {
   "id" : "18446744073709551615",
   "speed" : 0.0,
   "direction" : 0.0,
   "orientation" : 0.0,
   "" : {
     "confidence" : 0.33419784903526306
   },
   "bbox" : {
     "topleftx" : 241,
     "toplefty" : 185,
     "bottomrightx" : 540,
     "bottomrighty" : 228
   },
   "location" : {    },
   "coordinate" : {    }
 },
 "event" : {
   "id" : "8709b57c-4e54-440d-babe-d9342a0ccca4",
   "type" : ""
 },
 "videoPath" : ""
} 

You need to customize your own NVDS_EVENT_MSG_META and add the number plate to the metadata.

The OSD component displays the license plate number. So, I assumed that the metadata of classifier must be attached.

Yes. It has attached to the display metadata. But you want to send that through the msgbroker. So you need to attach that to the NVDS_EVENT_MSG_META type. You can refer to our source code deepstream-test5 to learn how to attach that to the NVDS_EVENT_MSG_META type.

I decided to modify /opt/nvidia/deepstream/sources/libs/nvmsgconv and rebuild it. However I am not able to build it(inside DeepStream-6.4 container). To be sure, I tried to build unmodified source, that is failing as well.
Steps I performed:-

  1. sudo apt-get install libglib2.0-dev libjson-glib-dev uuid-dev libyaml-cpp-dev protobuf-compiler
  2. make

PFA the error I am getting
err.txt (622.1 KB)

Could you use the command below to change the env path and try?

export PATH="/opt/proto/bin:$PATH"

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks