Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) : N/A • DeepStream Version : 7.0 • JetPack Version (valid for Jetson only) N/A • TensorRT Version : N/A • NVIDIA GPU Driver Version (valid for GPU only) N/A • Issue Type( questions, new requirements, bugs) Feature / Question
Recently, I am experimenting the DeepStream Message Broker with Azure IoT Hub. All the work is working inside DeepStream 7.0 container. There several problems I have checking and see if you have the answers alreadu.
In the message broker, the contentType and contentEncoding are not set. As a result, if you connect the cosmos database to the IoT Hub, you would have base64 encoded the message received. (As seen in here Using the new IoT Hub CosmosDB routing endpoint – Sander van de Velde).
I have tried to dive into both Azure IoT C SDK and DeepStream SDK. From Azure IoT C SDK, I could find an example of how to set these two properties (azure-iot-sdk-c/iothub_client/src/iothub_message.c at main · Azure/azure-iot-sdk-c · GitHub). However, I could not find a place, where I could make this happens in DeepStream SDK. Could you suggest? I have tried to add via custom_msg_properties in message-broker config, but it does not work.
I have tried to set the msg-conv-frame-interval and sleep-time in MsgConvBroker Sink, but these setting are not affecting the message sending at all. My IoT Hub received telemetry message every milli-second.
Thanks!
how did you know it does not work? please refer to opt\nvidia\deepstream\deepstream-7.0\sources\libs\azure_protocol_adaptor\device_client\README for Azure settings.
@fanzh The custom_msg_properties will be part of message send alone with the detection message? Will this be part of systemProperties or added into the body message or added to properties? In the example, it does not mention too many details. When I passed contentType=applicaiton/json and contentEncoding=utf-8, my cosmos db have nothing because the messages were not sent out successfully.
The message broker fails to send the mqtt signal with the following error message if add the extra key-value pair above.
Error: Time:Wed Aug 14 02:08:26 2024 File:/opt/nvidia/deepstream/deepstream-6.4/sources/azure-iot-sdk-c/umqtt/src/mqtt_client.c Func:sendPacketItem Line:387 Failure sending control packet data
Error: Time:Wed Aug 14 02:08:26 2024 File:/opt/nvidia/deepstream/deepstream-6.4/sources/azure-iot-sdk-c/umqtt/src/mqtt_client.c Func:mqtt_client_disconnect Line:1343 Error: mqtt_client_disconnect send failed
Active sources : 1
Wed Aug 14 02:08:28 2024
**PERF: 30.00 (31.10)
Active sources : 1
Wed Aug 14 02:08:33 2024
**PERF: 30.00 (30.91)
Error: Time:Wed Aug 14 02:08:34 2024 File:/opt/nvidia/deepstream/deepstream-6.4/sources/azure-iot-sdk-c/iothub_client/src/iothubtransport_mqtt_common.c Func:mqttOperationCompleteCallback Line:2129 Connection Not Accepted: 0x5: Not Authorized
Error: Time:Wed Aug 14 02:08:34 2024 File:/opt/nvidia/deepstream/deepstream-6.4/sources/azure-iot-sdk-c/umqtt/src/mqtt_client.c Func:sendPacketItem Line:387 Failure sending control packet data
Error: Time:Wed Aug 14 02:08:34 2024 File:/opt/nvidia/deepstream/deepstream-6.4/sources/azure-iot-sdk-c/umqtt/src/mqtt_client.c Func:mqtt_client_disconnect Line:1343 Error: mqtt_client_disconnect send failed
Active sources : 1
Also, refer to my question 2, any idea? I believe that we may get some idea if we could access the libnvds_azure_proto.so. However, I think that this source is not provided? With this, we may know how to add extra message header to turn base64 message into proper json as question 1.
@fanzh Referring to the message sending interval problem:
After further digging into source code, I found that enabling the msg-conv-frame-interval, we should have msg-conv-msg2p-new-api=1. This should be better documented down.
if (self->msg2pNewApi) {
//generate payload at frameInterval(ex: every 30th frame)
if (self->frameInterval
&& (((NvDsFrameMeta *) frame_meta)->frame_num %
self->frameInterval))
continue;
Yes, I want to have the more information about how to pass message header of contentType and contentEncoding, I believe that it is hidden inside libnvds_azure_proto.so. Could you illustrate how Nvidia package the Azure IoT C SDK ?
from the log, the fatal error is “Not Authorized”. To simplify, you can use the tool in \opt\nvidia\deepstream\deepstream\sources\libs\azure_protocol_adaptor\device_client to test sending broker. you can add the cfg into \opt\nvidia\deepstream\deepstream\sources\libs\azure_protocol_adaptor\device_client\cfg_azure.txt.
@fanzh Could you please explain more what custom_msg_properties are pointing to? There is no documentation about this. What is this property injected to…I am doubtful about what is this custom-message-properties are written to.
In sample prog: connect failed
In sample prog: connect success
In sample prog: connect failed
In sample prog: connect success
I need to add extra system properties, not body message, not properties only. Could you please read the web page I have share to understand the problem?
How could we test with extra layer of binary .so? It is about adding “header” to the message for Azure API, and it is encapsulated by Nvidia. Is it really hard to give extra info about what is the custom-message-properties? And explain further on how to add header to the API call ?
Noticing set_content_encoding is called by IoTHubMessage_SetContentEncodingSystemProperty in one link of the issue description. Currently the function which set custom_msg_properties can’t set set_content_encoding. Since azure_protocol_adaptor is not opensource, you can implement custom broker according to this doc.
Thanks, @fanzh. Great to know that Azure Protocol Adaptor does not support this. Will check on the custom broker. In the future update, probably adding a way to add the header would be great too!
