Tracking ID is broken for messages sent to Azure IoT hub using NvDsMsgConvBroker

ylashin · June 12, 2022, 5:45am

Environment

TensorRT Version : 8.2.5-1+cuda11.4
GPU Type : T4 GPU on AWS EC2 g4dn
Nvidia Driver Version : 515.43.04
CUDA Version : 11.6 in host and 11.7 in container
CUDNN Version : 8.4.0
Operating System + Version : Ubuntu 20.04 LTS
Python Version (if applicable) : 2.7.18
TensorFlow Version (if applicable) : N/A
PyTorch Version (if applicable) : N/A
Baremetal or Container (if container which image + tag) : Container image nvcr.io/nvidia/deepstream:6.1-devel

Relevant Files

Graph composer file

application:
  name: iot-hub
---
dependencies:
- extension: NvDsSourceExt
  uuid: a632d022-3425-4848-9074-e6483ef74366
  version: 1.0.0
- extension: NvDsBaseExt
  uuid: 56d7e3ec-62c6-4652-bcc8-4f1c3b00df03
  version: 1.0.0
- extension: NvDsMuxDemuxExt
  uuid: 89b8398c-5820-4051-835c-a91f2d49766b
  version: 1.0.0
- extension: NvDsInferenceExt
  uuid: 0b02963e-c24e-4c13-ace0-c4cdf36c7c71
  version: 1.0.0
- extension: NvDsSampleModelsExt
  uuid: 909dc1ea-c654-44c1-97a3-0b8fec12141a
  version: 1.0.0
- extension: NvDsVisualizationExt
  uuid: 25903cd4-fc5c-4139-987b-47bb27e8b424
  version: 1.0.1
- extension: NvDsCloudMsgExt
  uuid: d16b1366-dba2-47f6-95fc-b09b39f885de
  version: 1.0.0
- extension: NvDsInferenceUtilsExt
  uuid: 27856a43-5ad4-4d8e-be36-0ec2cf9bbb58
  version: 1.0.0
- extension: NvDsSampleExt
  uuid: 44a711e4-8519-4a68-81e8-be7ee4af3ff0
  version: 1.0.0
- extension: NvDsOutputSinkExt
  uuid: 3fc9ad87-03e7-47a8-bbfc-8501c3f7ff2f
  version: 1.0.0
- extension: NvDsTrackerExt
  uuid: f5aaeb53-e9b8-4317-87d1-9549d67c1638
  version: 1.0.0
---
components:
- name: nv_ds_single_src_input39..3c18
  parameters:
    audio-out-%u: nv_ds_dynamic_output40..2d30
    uri: file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
    video-out-%u: nv_ds_dynamic_output41..2f28
  type: nvidia::deepstream::NvDsSingleSrcInput
- name: nv_ds_dynamic_output40..2d30
  type: nvidia::deepstream::NvDsDynamicOutput
- name: nv_ds_dynamic_output41..2f28
  type: nvidia::deepstream::NvDsDynamicOutput
name: NvDsSingleSrcInput..1b00
ui_property:
  position:
    x: -2343.3974609375
    y: 0.0
  state: 0
---
components:
- name: nv_ds_stream_mux42..36d8
  parameters:
    batch-size: 1
    height: 1080
    video-in-%u: nv_ds_on_request_input43..d668
    video-out: nv_ds_static_output44..d710
    width: 1920
  type: nvidia::deepstream::NvDsStreamMux
- name: nv_ds_on_request_input43..d668
  type: nvidia::deepstream::NvDsOnRequestInput
- name: nv_ds_static_output44..d710
  type: nvidia::deepstream::NvDsStaticOutput
name: NvDsStreamMux..16d8
ui_property:
  position:
    x: -1916.2255859375
    y: 0.0
  state: 0
---
components:
- name: nv_ds_connection45..d320
  parameters:
    source: NvDsSingleSrcInput..1b00/nv_ds_dynamic_output41..2f28
    target: NvDsStreamMux..16d8/nv_ds_on_request_input43..d668
  type: nvidia::deepstream::NvDsConnection
name: node..9208
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: nv_ds_infer46..d5c0
  parameters:
    infer-model-config: resnet10_4_class_detector68..06a0
    video-in: nv_ds_static_input47..fb00
    video-out: nv_ds_static_output48..fba8
  type: nvidia::deepstream::NvDsInferVideo
- name: nv_ds_static_input47..fb00
  type: nvidia::deepstream::NvDsStaticInput
- name: nv_ds_static_output48..fba8
  type: nvidia::deepstream::NvDsStaticOutput
- name: resnet10_4_class_detector68..06a0
  type: nvidia::deepstream::NvDsResnet10_4ClassDetectorModel
name: NvDsInferVideo..d780
ui_property:
  position:
    x: -1605.4539794921875
    y: -26.326353073120117
  state: 0
---
components:
- name: nv_ds_connection49..9550
  parameters:
    source: NvDsStreamMux..16d8/nv_ds_static_output44..d710
    target: NvDsInferVideo..d780/nv_ds_static_input47..fb00
  type: nvidia::deepstream::NvDsConnection
name: node..2c50
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: nv_ds_osd50..feb8
  parameters:
    video-in: nv_ds_static_input51..3978
    video-out: nv_ds_static_output52..39b0
  type: nvidia::deepstream::NvDsOSD
- name: nv_ds_static_input51..3978
  type: nvidia::deepstream::NvDsStaticInput
- name: nv_ds_static_output52..39b0
  type: nvidia::deepstream::NvDsStaticOutput
name: NvDsOSD..bd68
ui_property:
  position:
    x: -978.8258666992188
    y: -139.40281677246094
  state: 0
---
components:
- name: nv_ds_tee54..0748
  parameters:
    in: nv_ds_static_input55..ed30
    out-%u: nv_ds_on_request_output56..eda0
  type: nvidia::deepstream::NvDsTee
- name: nv_ds_static_input55..ed30
  type: nvidia::deepstream::NvDsStaticInput
- name: nv_ds_on_request_output56..eda0
  type: nvidia::deepstream::NvDsOnRequestOutput
name: NvDsTee..d358
ui_property:
  position:
    x: -665.5477905273438
    y: -142.5000457763672
  state: 0
---
components:
- name: nv_ds_connection57..47f0
  parameters:
    source: NvDsOSD..bd68/nv_ds_static_output52..39b0
    target: NvDsTee..d358/nv_ds_static_input55..ed30
  type: nvidia::deepstream::NvDsConnection
name: node..8cf8
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: nv_ds_msg_conv_broker61..4748
  parameters:
    in: nv_ds_static_input62..ac88
    msg-broker-conn-str: 127.0.0.1;9092;test-topic
    msg-broker-proto-lib: /opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
    msg-conv-config: /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test4/dstest4_msgconv_config.txt
    msg-conv-payload-type: 0
    topic: test-topic
  type: nvidia::deepstream::NvDsMsgConvBroker
- name: nv_ds_static_input62..ac88
  type: nvidia::deepstream::NvDsStaticInput
name: NvDsMsgConvBroker..bb00
ui_property:
  position:
    x: -182.77389526367188
    y: -216.25006103515625
  state: 0
---
components:
- name: nv_ds_connection63..0198
  parameters:
    source: NvDsTee..d358/nv_ds_on_request_output56..eda0
    target: NvDsMsgConvBroker..bb00/nv_ds_static_input62..ac88
  type: nvidia::deepstream::NvDsConnection
name: node..4b70
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: per_class_object_counting64..4a90
  parameters:
    probe-connector: nv_ds_probe_connector65..ff60
  type: nvidia::deepstream::NvDsPerClassObjectCounting
- name: nv_ds_probe_connector65..ff60
  type: nvidia::deepstream::NvDsProbeConnector
name: NvDsPerClassObjectCounting..9588
ui_property:
  position:
    x: -950.8838500976562
    y: 203.0209197998047
  state: 0
---
components:
- name: nv_ds_probe66..15f8
  parameters:
    io: NvDsInferVideo..d780/nv_ds_static_output48..fba8
    probe-connector: NvDsPerClassObjectCounting..9588/nv_ds_probe_connector65..ff60
  type: nvidia::deepstream::NvDsProbe
name: node..4d68
ui_property:
  position:
    x: 0
    y: 0
  state: 0
---
components:
- name: sample_probe_message_meta_creation0..39e8
  parameters:
    generate-dummy-data: false
    probe-connector: nv_ds_probe_connector1..c940
  type: nvidia::deepstream::NvDsSampleProbeMessageMetaCreation
- name: nv_ds_probe_connector1..c940
  type: nvidia::deepstream::NvDsProbeConnector
name: NvDsSampleProbeMessageMetaCreation..37f0
ui_property:
  position:
    x: -975.6614990234375
    y: 57.77784729003906
  state: 0
---
components:
- name: nv_ds_probe2..cc88
  parameters:
    io: NvDsInferVideo..d780/nv_ds_static_output48..fba8
    probe-connector: NvDsSampleProbeMessageMetaCreation..37f0/nv_ds_probe_connector1..c940
  type: nvidia::deepstream::NvDsProbe
name: node..13c8
ui_property:
  position:
    x: -662.0599365234375
    y: 458.79290771484375
---
components:
- name: nv_ds_rtsp_out0
  parameters:
    audio-in: nv_ds_on_request_input1
    video-in: nv_ds_on_request_input2
  type: nvidia::deepstream::NvDsRtspOut
- name: nv_ds_on_request_input1
  type: nvidia::deepstream::NvDsOnRequestInput
- name: nv_ds_on_request_input2
  type: nvidia::deepstream::NvDsOnRequestInput
name: NvDsRtspOut
ui_property:
  position:
    x: -274.847412109375
    y: 6.645165920257568
---
components:
- name: nv_ds_connection3
  parameters:
    source: NvDsTee..d358/nv_ds_on_request_output56..eda0
    target: NvDsRtspOut/nv_ds_on_request_input2
  type: nvidia::deepstream::NvDsConnection
name: node0
ui_property:
  position:
    x: -662.0599365234375
    y: 991.29296875
---
components:
- name: nv_ds_tracker4
  parameters:
    compute-hw: 1
    enable-batch-process: true
    ll-config-file: /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
    ll-lib-file: /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
    tracker-height: 384
    tracker-width: 640
    video-in: nv_ds_static_input5
    video-out: nv_ds_static_output6
  type: nvidia::deepstream::NvDsTracker
- name: nv_ds_static_input5
  type: nvidia::deepstream::NvDsStaticInput
- name: nv_ds_static_output6
  type: nvidia::deepstream::NvDsStaticOutput
name: NvDsTracker
ui_property:
  position:
    x: -1302.326904296875
    y: -121.1803970336914
  state: 0
---
components:
- name: nv_ds_connection7
  parameters:
    source: NvDsInferVideo..d780/nv_ds_static_output48..fba8
    target: NvDsTracker/nv_ds_static_input5
  type: nvidia::deepstream::NvDsConnection
name: node1
ui_property:
  position:
    x: -662.0599365234375
    y: -276.2500915527344
---
components:
- name: nv_ds_connection8
  parameters:
    source: NvDsTracker/nv_ds_static_output6
    target: NvDsOSD..bd68/nv_ds_static_input51..3978
  type: nvidia::deepstream::NvDsConnection
name: node2
ui_property:
  position:
    x: -662.0599365234375
    y: -611.2501220703125
---
components:
- name: nv_ds_scheduler0
  type: nvidia::deepstream::NvDsScheduler
name: NvDsScheduler
ui_property:
  position:
    x: -1431.155517578125
    y: 367.4088439941406

Parameters file

components:
- name: nv_ds_msg_conv_broker61..4748
  parameters:
    disable-msgconv: false
    msg-broker-conn-str: <<azure iot hub device connection string>>
    msg-broker-proto-lib: /opt/nvidia/deepstream/deepstream/lib/libnvds_azure_proto.so
    msg-conv-config: /workspaces/devenv1/demo3/demo3-msg-config.txt
    msg-conv-payload-type: 1
name: NvDsMsgConvBroker..bb00

Message config file

[sensor0]
enable=1
type=Camera
id=CAMERA_ID
location=45.293701447;-75.8303914499;48.1557479338
description="Entrance of Garage Right Lane"
coordinate=5.2;10.1;11.2

[place0]
enable=0
id=0
type=garage
name=XYZ
location=30.32;-40.55;100.0
coordinate=1.0;2.0;3.0
place-sub-field1=walsh
place-sub-field2=lane1
place-sub-field3=P2

[place1]
enable=0
id=1
type=garage
name=XYZ
location=28.47;47.46;1.53
coordinate=1.0;2.0;3.0
place-sub-field1="C-76-2"
place-sub-field2="LEV/EV/CP/ADA"
place-sub-field3=P2

[analytics0]
enable=1
id=XYZ
description="Vehicle Detection and License Plate Recognition"
source=OpenALR
version=1.0

Steps To Reproduce

EC2 instance with T4 GPU and driver + nvidia docker 2 installed
Using VS Code dev containers to run graph composer applications inside container image nvcr.io/nvidia/deepstream:6.1-devel

Inside the container run:

/opt/nvidia/graph-composer/execute_graph.sh \
   ./demo3/demo3.yaml \
   ./demo3/demo3.parameters.yaml \
   -d /opt/nvidia/graph-composer/config/target_x86_64.yaml

The graph, parameters and message config file are shared with this post.
I get the messages submitted to IoT hub but the tracking Id is fixed to that value of 2^64 - 1

My question is how to fix the tracking such that the messages will have the proper tracking id per each object. The sample deepstream-app (or maybe deepstream-testapp[x]) works fine in this aspect but not the graph composer app in this post.

Here is a sample message from IoT hub.

{
  "version": "4.0",
  "id": "160",
  "@timestamp": "2022-06-12T05:35:43.899Z",
  "sensorId": "CAMERA_ID",
  "objects": [
    "18446744073709551615|86.6055|457.571|196.834|776.01|Person",
    "18446744073709551615|389.809|484.725|436.491|618.371|Person",
    "18446744073709551615|405.912|482.343|448.986|590.122|Person",
    "18446744073709551615|336.106|465.285|404.033|649.971|Person",
    "18446744073709551615|465.391|477.544|520.373|629.898|Person",
    "18446744073709551615|423.005|478.853|463.13|594.552|Person",
    "18446744073709551615|0|678.245|213.187|1013.22|Bicycle",
    "18446744073709551615|506.091|462.338|540.95|483.196|Vehicle|#|||||||0",
    "18446744073709551615|1253.92|480.084|1329.11|519.265|Vehicle|#|||||||0",
    "18446744073709551615|620.744|478.089|665.637|542.661|Vehicle|#|||||||0",
    "18446744073709551615|585.781|476.709|624.796|520.75|Vehicle|#|||||||0",
    "18446744073709551615|641.923|473.229|861.158|645.573|Vehicle|#|||||||0",
    "18446744073709551615|854.097|463.213|1121.58|621.646|Vehicle|#|||||||0",
    "18446744073709551615|1201.25|476.264|1282.18|517.391|Vehicle|#|||||||0",
    "18446744073709551615|1302.05|479.647|1430.51|531.454|Vehicle|#|||||||0"
  ]
}

Fiona.Chen · June 13, 2022, 2:15am

Can you check whether “useUniqueID: 0” is set in your /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml file?

ylashin · June 13, 2022, 3:13am

Hi @Fiona.Chen

The value was 0. As per the docs, switching it to one would help with multi-source pipelines which is not the case here. After changing it to one, that did not make a change to IoT hub messages. They are still fixed to 18446744073709551615.

The on screen display shows bigger values for tracking IDs as follows.

When I reverted back the value to 0, the tracking IDs on display started from the classic small range of 0 onwards.

Hope that helps.

Fiona.Chen · June 13, 2022, 3:19am

The usage of “useUniqueID” is described in Gst-nvtracker — DeepStream 6.1.1 Release documentation

ylashin · June 13, 2022, 3:26am

I guess the tracking functionality itself is working fine and that’s obvious from OSD results.
It’s the broker or message converter or both which seem to have an issue.

Fiona.Chen · June 13, 2022, 3:42am

What is your config of “payload-type” in your nvmsgconv configuration? PAYLOAD_DEEPSTREAM or PAYLOAD_DEEPSTREAM_MINIMAL?

ylashin · June 13, 2022, 3:54am

I’d say my payload type is set to MINIMAL because the numeric value of broker component property is set to 1.

I have the following in graph parameters file:

- name: nv_ds_msg_conv_broker61..4748
  parameters:
    disable-msgconv: false
    msg-broker-conn-str: <<connection string>>
    msg-broker-proto-lib: /opt/nvidia/deepstream/deepstream/lib/libnvds_azure_proto.so
    msg-conv-config: /workspaces/devenv1/demo3/demo3-msg-config.txt
    msg-conv-payload-type: 1

demo3-msg-config.txt file has the following contents:

[sensor0]
enable=1
type=Camera
id=CAMERA_ID
location=45.293701447;-75.8303914499;48.1557479338
description="Entrance of Garage Right Lane"
coordinate=5.2;10.1;11.2

[place0]
enable=0
id=0
type=garage
name=XYZ
location=30.32;-40.55;100.0
coordinate=1.0;2.0;3.0
place-sub-field1=walsh
place-sub-field2=lane1
place-sub-field3=P2

[place1]
enable=0
id=1
type=garage
name=XYZ
location=28.47;47.46;1.53
coordinate=1.0;2.0;3.0
place-sub-field1="C-76-2"
place-sub-field2="LEV/EV/CP/ADA"
place-sub-field3=P2

[analytics0]
enable=1
id=XYZ
description="Vehicle Detection and License Plate Recognition"
source=OpenALR
version=1.0

ylashin · June 14, 2022, 9:18am

Hi @Fiona.Chen,

Any ideas on how to have the right tracking Id into D2C messages?

Thanks,
Yousry

Fiona.Chen · June 14, 2022, 12:31pm

I can not reproduce your error. I can get correct track id in the message.

{
  "version" : "4.0",
  "id" : "1441",
  "@timestamp" : "2022-06-14T12:30:13.549Z",
  "sensorId" : "HWY_20_AND_LOCUST__EBA__4_11_2018_4_59_59_508_AM_UTC-07_00",
  "objects" : [
    "1356|838.291|468.542|1032.82|592.623|Car",
    "1380|1585.16|485.414|1756.73|558.632|Car",
    "1376|1412.1|490.452|1547.01|537.59|Car",
    "1280|636.042|475.907|778.663|591.489|Car",
    "1372|790.198|477.91|842.848|530.521|Car",
    "1324|590.155|484.045|631.438|521.023|Car",
    "976|441.91|479.327|470.075|546.164|Person",
    "868|419.352|482.142|442.73|547.516|Person",
    "1188|456.926|473.493|483.792|545.305|Person"
  ]
}

ylashin · June 14, 2022, 11:49pm

Thanks @Fiona.Chen

I wonder if you used a graph composer application or a C++ one.

If you have used graph composer, could you please share the files you used to test in case you did not use my own files.

Thanks,
Yousry

Topic		Replies	Views
Run Deepstream-test5 with AWS IoT DeepStream SDK rtsp , aws , deepstream , configurations , jetson-nano	9	762	September 5, 2023
Trouble running own model with Deepstream Graph Composer DeepStream SDK deepstream , graph-composer	11	60	January 21, 2025
How to set custom tracker in deepstream with detection interval > 0 DeepStream SDK	7	480	September 14, 2023
Trying to integrate Kafka with Deepstream for which is getting stuck at 4th Frame - Error DeepStream SDK	2	1082	October 12, 2021
How verify if the message by the Deepstream application is sent correctly to Azure Iot Hub DeepStream SDK	14	585	April 17, 2023
PERF {stream:0.0} Error in DeepStream Application with Azure IoT Hub Integration DeepStream SDK deepstream	5	29	December 30, 2024
Windows through SSH Connect to ubuntu DeepStream SDK	20	607	March 22, 2024
Libnvds_azure_edge_proto.so VS libnvds_azure_proto.so DeepStream SDK inception	10	740	April 11, 2023
New NvStreammux shows 「[ERROR push 317] push failed [-5]」 DeepStream SDK	9	596	June 4, 2024
Graph Composer[Errno 13] Permission denied: '/tmp/nvgraph_registry.log' DeepStream SDK graph-composer	2	666	November 17, 2022

Tracking ID is broken for messages sent to Azure IoT hub using NvDsMsgConvBroker

Environment

Relevant Files

Steps To Reproduce

Related topics