How do I tell Kafka about SGIE results in deepstream-test5?

[bluerayi]

I am trying to send a message to Kafka using deepstream-test5.

In the configuration file that comes with deepstream-test5, I can set [sink1] to enable and edit the server information to see the messages immediately.

BTW, the screen output that starts in this state and is connected to xhost shows information about the type and make of the car. I am aware that these information are inferred by Secondary GIE.
On the other hand, the message sent to Kafka only shows the class name of the object detection, which seems to be inferred by Primary GIE, and I cannot confirm detailed information about these cars.

I recognize that I can customize the messages by compiling libnvds_msgconv.so by myself. In my environment, customization was possible by editing eventmsg_payload.cpp.
However, I checked some values of the NvDsEventMsgMeta structure handled by eventmsg_payload.cpp, but I can’t find the Secondary GIE inference result in any way.
Is there any way to retrieve this?

Or is it possible with the code in dsmeta_payload.cpp? However, I tried setting msg-conv-msg2p-newapi in [sink1] to 1, but there is no change in the message.

Any help would be appreciated.
Thanks.

• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 7.0
• TensorRT Version: 8.6.1.6-1+cuda12.0
• NVIDIA GPU Driver Version (valid for GPU only): NVIDIA UNIX x86_64 Kernel Module 550.90.07 Fri May 31 09:35:42 UTC 2024
• Issue Type( questions, new requirements, bugs): questions

[fanzh]

if msg-conv-msg2p-newapi is 1, you need to modify dsmeta_payload.cpp. please refer to the doc for the difference. you can add log to check.

[bluerayi]

Thanks for the reply.

Yes, I too thought that changing msg-conv-msg2p-newapi should change the referenced source code. However, as I mentioned in the text, changing this setting does not work as I expected.

Specifically, I did the following as a simple test. Note that the container used is nvcr.io/nvidia/deepstream:7.0-gc-triton-devel.

1. in /opt/nvidia/deepstream/deepstream-7.0/sources/libs/nvmsgconv/deepstream_schema/dsmeta_payload.cpp, insert the following code at line 282:

json_object_set_string_member(objectObj, "source", "dsmeta");

2. in /opt/nvidia/deepstream/deepstream-7.0/sources/libs/nvmsgconv/deepstream_schema/eventmsg_payload.cpp, insert the following code at line 727:

json_object_set_string_member(objectObj, "source", "eventmsg");

3. run make in /opt/nvidia/deepstream/deepstream-7.0/sources/libs/nvmsgconv/ to compile libnvds_msgconv.so
4. edit /opt/nvidia/deepstream/deepstream-7.0/sources/apps/sample_apps/deepstream-test5/config/test5_config_file_nvmultiurisrcbin_src_list_attr_all.txt and replace lines 99 and after with the following:

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=UDPSink 5=nvdrmvideosink 6=MsgConvBroker
type=6
msg-conv-config=dstest5_msgconv_sample_config.txt
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM   - Custom schema payload
msg-conv-payload-type=0
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
#Provide your msg-broker-conn-str here
msg-broker-conn-str=localhost;9092;ds
topic=ds
#Optional:
#msg-broker-config=../../deepstream-test4/cfg_kafka.txt
#new-api=0
#(0) Use message adapter library api's
#(1) Use new msgbroker library api's
msg-conv-msg2p-lib=/opt/nvidia/deepstream/deepstream-7.0/sources/libs/nvmsgconv/libnvds_msgconv.so
msg-conv-msg2p-newapi=1 #!!

5. Start Kafka on localhost:9092 and subscribe to the ds topic.
6. run . /deepstream-test5 -c configs/test5_config_file_nvmultiurisrcbin_src_list_attr_all.txt.

As a result, the object.source of the received message will be "eventmsg" regardless of whether msg-conv-msg2p-newapi is set to 0 or 1.

Anyway, the main question is: Is there any way to put the SGIE classification results in the deepstream-test5 application, i.e. car type, car make, etc. in the Kafka message?

Thank you in advance.

[fanzh]

@bluerayi

msg-conv-msg2p-newapi=1 #!!

please use msg-conv-msg2p-new-api=1. please refer to opt\nvidia\deepstream\deepstream-7.0\sources\apps\sample_apps\deepstream-test5\configs\test5_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

[bluerayi]

Thank you for your answer.

I was able to switch the metadata used for message construction from event messages to frames and objects by using msg-conv-msg2p-new-api instead of msg-conv-msg2p-newapi. I had mistakenly set it up because the documentation for the configuration item linked from the deepstream-test5 documentation said msg-conv-msg2p-newapi.

And now that we have switched the metadata, the SGIE results are sent as messages without having to change the nvmsgconv code. Therefore, this question has been resolved.

Thank you all for your help.