Deepstream stuck with multi-msgbroker

[yamiefun]

Hi, we’re using DS 5.1 python with multi-rtsp sources on a dGPU server.
The images below briefly show our pipeline, pins on the element means python probe function.

We use multi-rtsp as sources and do some experiment.

1. This pipeline can run properly with single rtsp source.
2. When using more than 2 sources, this pipeline will stuck for a long time and print the error message below.(Sometimes these error message will be printed after we shutdown DS.)

   %3|1634285296.652|FAIL|rdkafka#producer-4| [thrd:192.168.1.23:9092/bootstrap]: 192.168.1.23:9092/1001: 
   Receive failed: Disconnected
   %3|1634285296.652|ERROR|rdkafka#producer-4| [thrd:192.168.1.23:9092/bootstrap]: 192.168.1.23:9092/1001: 
   Receive failed: Disconnected
   %3|1634285296.652|ERROR|rdkafka#producer-4| [thrd:192.168.1.23:9092/bootstrap]: 1/1 brokers are down
   

   We tried to access kafka server through command line when DS stucked, and it seemed that kafka was working properly.

We tried to decompose the pipeline to find the problem, and finally found that this pipeline can run properly with multi-sources without msgbroker element.

Is there any rule or config need to be set if a pipeline contains multiple msgbroker elements? Or our pipeline is not reasonable?

[Amycao]

We support sending message to multi brokers.

Check this:

Multiple broker sinks might be required for sending a
message to multipe backends simultaneously or to send
specific message to particular backend.

By default sink of type = 6 adds message converter and message broker components in the pipeline. In case of multiple brokers [message-converter] group can be used to add single message converter in the pipeline with multiple sinks of type = 6 having disable-msgconv set to 1.

If multiple message converters are also required along with multiple brokers
then “msg-conv-comp-id” and “msg-broker-comp-id” should be set properly to avoid duplicate messages. These fields force converter / broker components to process only those messages having same value for componentId field and ignore other messages. User should modify the application to fill componentId field of NvDsEventMsgMeta structure.