Protobuf support in Deepstream

[sathish.iitd]

nvmsgbroker, nvmsgconv are part of my object detection pipeline. nvmsgconv converts metadata to protobuf message and nvmsgbroker sends to RabbitMQ. I am observing an issue in decoding protobuf message at receiver end. It works fine If I sent a simple protobuf message(AiServiceMsg without Bbox). It will not work if protobuf message contains another protobuf message as below.

Nvmsgbroker is using AMQP protocol(libnvds_amqp_proto.so) to send protobuf message.

syntax = “proto3”;
message Bbox{
float x1=1;
float y1=2;
float x2=3;
float y2=4;
}

message AiServiceMsg{
string cameraId=1;
string msgType=2;
Bbox box=3;

}
C++ code to create a protobuf message.
AiServiceMsg ai_service_msg;

ai_service_msg.set_cameraid(camera_id);
ai_service_msg.set_msgtype("ObjectDetection"); 
Bbox* box = ai_service_msg.mutable_box();

box->set_x1(11);
box->set_y1(11);
box->set_x2(22);
box->set_y2(22);

#if 1
int size = ai_service_msg.ByteSize();

std::string str;

if (!ai_service_msg.SerializeToString(&str)) {
    cout << "generate_event_message_protobuf : Failed to serialize protobuf message to string.\n";
   
    return ;
}

gchar* data = (gchar*)g_memdup(str.c_str(), str.size());

// data sent to RabbitMQ

Let me know what could be the issue.

[fanzh]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

[sathish.iitd]

@fanzh

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): GPU

• DeepStream Version: 6.2

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs) questions

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details(

Send a protobuf message to RabbitMQ using msgbroker and receive the message at RabbitMQ client and parse it.

[fanzh]

please rebuild libnvds_msgconv.so after modifying the proto file in /opt/nvidia/deepstream/deepstream/sources/libs/nvmsgconv, then replace the old /opt/nvidia/deepstream/deepstream/lib/libnvds_msgconv.so with the new so.

[sathish.iitd]

What are the changes in new nvmsgconv. I don’t think the issue is in nvmsgconv or nvmsgbroker. The issue may be in libnvds_amqp_proto.so.
Is AMQP implementation support in sending binary format?

[sathish.iitd]

What is encoding format set in libnvds_amqp_proto.so while publishing the message.

[sathish.iitd]

nvmsgbroker.zip (20.5 KB)

Test code , proto file is attached. Ensure that protobuf is installed in your PC.
When you run we can observe a difference in message sent and message received.

The decoded size will match if I comment below code

[sathish.iitd]

I tested sending protobuf message using only with AMQP adaptor. No other plugins are used. I can still see the issue , the size received at client end is different from sender.
It concludes the issue is with AMQP protocol implementation. Code is attached.

amqp_protocol_adaptor.zip (25.8 KB)

The protocol message is in binary format, It may contain 0x0 in between.
I think AMQP protocol implementation not considering this, they may be using strlen to calculate the size of the message.

[fanzh]

could you print the value of “str.size()”? seems str.size() is the actual sending length. please refer to generate_event_message_protobuf in nvmsgconv.cpp.

[sathish.iitd]

I don’t think any issue with nvmsgconv.cpp. I tested with AMQP adaptor without nvmsgconv and nvmsgbroker. I shared AMQP adaptor code to reproduce this issue.
Please use that to reproduce it. In AMQP adaptor I used str.size() to calculate message size.

[fanzh]

@sathish.iitd

I think AMQP protocol implementation not considering this, they may be using strlen to calculate the size of the message.

thanks for the sharing! nvmsgbroker low-level lib will call thirdpart interface to send protobuf messge without any modification. the return value 34 is also got from thirdpart interface. please refer to low-level log by this method. here is my test log log0401.txt (1.3 KB). the sending data is the same with the receiving data in the message.
did you check from the server side? for example, please check if the receiving data is the same with sending data.

[sathish.iitd]

Thanks for sharing. the protobuf message after serialising to string is as below. The string contains 0 in between. The message size is 53. You may be printing the message by converting to char pointer. We can’t get exact message if we print because the string contains “\r” and 0 in between. Please check message size at sender and receiver end.

[fanzh]

we are investigating, will get back to you if there are any updates. May I know your company name? and does this issue block you? could you share your use scenario? Thanks!

[sathish.iitd]

Hi, I work for Johnson Controls International PLC. Currently, It is not blocking my work.
I need the the fix asap.

[sathish.iitd]

Any update?

[fanzh]

sorry for the late reply! I sent you a forum email. please check.

[fanzh]

the code is opensource in DeepStream 7.1. the path is \opt\nvidia\deepstream\deepstream-7.1\sources\libs\amqp_protocol_adaptor\amqp_client.cpp.