Deepstream 7.1 will not publish messages/detections using Kafka

[marcelof00]

My system:

Jetson Orin Nano (6-core Arm® Cortex®-A78AE v8.2 64-bit CPU 1.5MB L2 + 4MB L3) (8Gb RAM)
DeepStream Version: 7.1
JetPack: 6.2
TensorRT Version: 10.3.0.30
CUDA: 12.6.68

Background information:
I have developed an application that leverages the ultralytics yolov8n model. The app is working properly and visualizing the results on the screen.

Issue:
Deepstream is not publishing messages to Kafka.

What I have done:

1. I included [sink=1] type=6 using exactly the same configuration used in deepstream-test5, which I understand supports sending messages to Kafka. Below is the configuration of the file deepstream_app_config.txt

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=3
uri=file://video.mp4
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=1
type=6
msg-conv-config=/home/marcelo/DeepStream-Yolo/dstest5_msgconv_sample_config.txt
msg-conv-payload-type=0
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream-7.1/lib/libnvds_kafka_proto.so
msg-broker-conn-str=localhost;9092;deepstream-output
topic=deepstream-output
#msg-broker-config=cfg_kafka.txt
#Optional:
#msg-broker-config: …/…/deepstream-test4/cfg_kafka.txt

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV8.txt

[tests]
file-loop=0

Below is the content of the file called dstest5_msgconv_sample_config.txt

################################################################################

SPDX-FileCopyrightText: Copyright (c) 2018-2020 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

SPDX-License-Identifier: LicenseRef-NvidiaProprietary

NVIDIA CORPORATION, its affiliates and licensors retain all intellectual

property and proprietary rights in and to this material, related

documentation and any modifications thereto. Any use, reproduction,

disclosure or distribution of this material and related documentation

without an express license agreement from NVIDIA CORPORATION or

its affiliates is strictly prohibited.

################################################################################

[sensor0]
enable=1
type=Camera
id=HWY_20_AND_LOCUST__EBA__4_11_2018_4_59_59_508_AM_UTC-07_00
location=45.293701447;-75.8303914499;48.1557479338
description=Aisle Camera
coordinate=5.2;10.1;11.2

[sensor1]
enable=1
type=Camera
id=HWY_20_AND_LOCUST__WBA__4_11_2018_4_59_59_379_AM_UTC-07_00
location=45.293701447;-75.8303914499;48.1557479338
description=Aisle Camera
coordinate=5.2;10.1;11.2

[sensor2]
enable=1
type=Camera
id=HWY_20_AND_DEVON__WBA__4_11_2018_4_59_59_134_AM_UTC-07_00
location=45.293701447;-75.8303914499;48.1557479338
description=Aisle Camera
coordinate=5.2;10.1;11.2

[sensor3]
enable=1
type=Camera
id=HWY_20_AND_LOCUST__4_11_2018_4_59_59_320_AM_UTC-07_00
location=45.293701447;-75.8303914499;48.1557479338
description=Aisle Camera
coordinate=5.2;10.1;11.2

[place0]
enable=1
id=0
type=intersection/road
name=HWY_20_AND_LOCUST__EBA
location=30.32;-40.55;100.0
coordinate=1.0;2.0;3.0
place-sub-field1=C_127_158
place-sub-field2=Lane 1
place-sub-field3=P1

[place1]
enable=1
id=1
type=intersection/road
name=HWY_20_AND_LOCUST__WBA
location=30.32;-40.55;100.0
coordinate=1.0;2.0;3.0
place-sub-field1=C_127_158
place-sub-field2=Lane 1
place-sub-field3=P1

[place2]
enable=1
id=2
type=intersection/road
name=HWY_20_AND_DEVON__WBA
location=30.32;-40.55;100.0
coordinate=1.0;2.0;3.0
place-sub-field1=C_127_158
place-sub-field2=Lane 1
place-sub-field3=P1

[place3]
enable=1
id=3
type=intersection/road
name=HWY_20_AND_LOCUST
location=30.32;-40.55;100.0
coordinate=1.0;2.0;3.0
place-sub-field1=C_127_158
place-sub-field2=Lane 1
place-sub-field3=P1

[analytics0]
enable=1
id=XYZ_1
description=Vehicle Detection and License Plate Recognition
source=OpenALR
version=1.0

[analytics1]
enable=1
id=XYZ_2
description=Vehicle Detection and License Plate Recognition 1
source=OpenALR
version=1.0

[analytics2]
enable=1
id=XYZ_3
description=Vehicle Detection and License Plate Recognition 2
source=OpenALR
version=1.0

[analytics3]
enable=1
id=XYZ_4
description=Vehicle Detection and License Plate Recognition 4
source=OpenALR
version=1.0

2. I have sent a “test message” to Kafka to make sure it is working as expected:
   echo “test message” | bin/kafka-console-producer.sh --broker-list localhost:9092 --topic deepstream-output

and I was able to receive it (i had this running in another terminal):
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic deepstream-output --from-beginning

3. I checked just in case and the file libnvds_kafka_proto.so exists.

4. I tried to use the specify the file cfg_kafka.txt in the [sink1] section, with no luck, hence it shows documented now.

5. I executed GST_DEBUG=4 deepstream-app -c deepstream_app_config.txt I did not find any error related to Kafka

6. Just in case, I checked Kafka log, but did not see anything, which confirms that the issues seems to be that Deepstream is not sending any message to Kafka:
   cat /opt/kafka/logs/server.log

I would appreciate your help!

[fanzh]

which sample are you using? deepstraem-app or deepstream-test5?

[marcelof00]

I am using deepsteam-app based on the configuration of test5. As you can see in the configuration I included in the ticket, it matches with the configuration used by test5. Please, advise.

[marcelof00]

Ok, I fixed it.