What advantages I do get using msgconv or msgcbroker over KafkaProducer (pip package kafka-python)?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi I am trying to run deepstream test 4 python sample app and was able to run it successfully but I want to send a custom payload instead of the default one and I am confused on how to send those.

To tackle the issue temporary, I came across this pip package kafka-python where we can send data to kafka by using KafkaProducer method like this

producer = KafkaProducer(
bootstrap_servers='<IP>:9092',
value_serializer=lambda x: dumps(x).encode('utf-8')
)
producer.send('test', value=test_dict)

So what benefits do msgbroker offers over Kafka pip package? And what is the right approach? If right approach is to use msgbroker then how to send a custom payload (I tried creating a custom payload but failed with “terminate called after throwing an instance of ‘std::bad_function_call’” )?

PS I ran KafkaProducer with 10 simultaneous streams and it doesn’t affect the FPS.

• Hardware Platform (Jetson / GPU) T4
• DeepStream Version 5.01
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

3

Hey, we will check it internally and update ASAP.

4

@bcao any Update?

5

+1 @bcao

6

Sorry for the late, it should be fine to use the KafkaProducer Python package . Using the msgconv+msgbroker plugins allows tee/queue elements in the pipeline to parallelize the Kafka message sending with visual output. When sending message directly from a probe function, the message sending blocks the downstream pipeline until it’s done.

1 Like
7

Thanks for the reply. I m trying to send a custom JSON object from deepstream using Kafka but there is no information/documentation on NvDsPayloadObject.
Can you please guide me through it?
REFER: How to build a custom object to use on payloads for message broker with Python Bindings - #3 by bcao

8

Sorry for the delay, yeah, we will check and update it in that topic.