Custom Payload for Kafka Integration in DeepStream App on Kubernetes

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

• Hardware Platform:
GPU NVIDIA L40S
• DeepStream Version:
7.1
• TensorRT Version:
Aligned with DeepStream 7.1 recommendations
• Issue Type:
Questions
• Requirement Details:

Hi everyone,

I have deployed a DeepStream application using Kubernetes and integrated the Kafka message broker plugin (nvmsgconv and nvmsgbroker). Now, I would like to send a custom payload that includes information such as tracking ID, bounding box (bbox), region of interest (ROI), and detected object details.

How can I achieve this?

Here is my current configuration file:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=60

[source0]
enable=1
type=4
uri=<my_rtsp>
num-sources=1
gpu-id=0
latency=200
drop-frame-interval=2
rtsp-reconnect-interval-sec=30
select-rtp-protocol=4
camera-fps-n=12
camera-fps-d=1

[primary-gie]
enable=1
batch-size=1
gie-unique-id=1
labelfile-path=labels.txt
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[sink0]
enable=1
gpu-id=0
type=4
sync=0
codec=1
nvbuf-memory-type=0
bitrate=4000000
iframeinterval=10
rtsp-port=8554
udp-port=5555
profile=0
udp-buffer-size=1000000
qos=0

[sink1]
type=6
enable=1
msg-conv-payload-type=0
msg-conv-msg2p-new-api=1
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream/lib/libnvds_kafka_proto.so
msg-broker-conn-str=<my_IP>;9092;ds-avio-test
topic=ds-avio-test
nvbuf-memory-type=0
msg-conv-msg2p-lib=/opt/nvidia/deepstream/deepstream-7.1/lib/libnvds_msgconv.so
msg-conv-comp-id=1
msg-broker-comp-id=1
msg-conv-config=msgconv_config.txt
sync=0

[streammux]
gpu-id=0
live-source=1
batch-size=1
batched-push-timeout=4000
width=1920
height=1080
enable-padding=1
nvbuf-memory-type=0
attach-sys-ts-as-ntp=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=10
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[tiled-display]
enable=1
rows=1
columns=1
width=1920
height=1080
gpu-id=0

[tracker]
enable=1
tracker-width=320
tracker-height=320
ll-lib-file=../libs/libnvds_nvmultiobjecttracker.so
ll-config-file=config_tracker_NvDCF_accuracy.yml
gpu-id=0
display-tracking-id=1

Any guidance on how to modify msgconv_config.txt or other settings to include custom metadata would be greatly appreciated.

Thanks in advance!

3

Noticing msg-conv-msg2p-new-api=1, Deepstream will use NvDsFrameMeta and NvDsObjectMeta to generate payload. please refer to the doc. If you want to add some custom payload, please rerfer to this doc. pease refer to sample code osd_sink_pad_buffer_image_probe of test4.

4

Thank you!

Also, is it possible to analyze both objects inside and outside the ROI separately within a single inference and stream?
The inverse-roi parameter in the Gst-nvdsanalytics plugin only allows analyzing either one or the other, but not both simultaneously.

5
  1. what nvanalytics features did you add in cfg? could you share the cfg of nvanalytics?
  2. If inverse-roi is 0, to get the number outside the ROI, you may use the objects number in the full frame to subtract the number in the ROI. and vice versa.
6

Here my cfg:

[property]
enable=1
osd-mode=2
display-font-size=12
config-width=1920
config-height=1080

[line-crossing-stream-0]
enable=0
mode=loose
extended=0
class-id=-1
line-crossing-line_1=353;185;488;221;498;69;238;534;

[roi-filtering-stream-0]
enable=1
roi-RFframe=0;0;1920;0;1920;1080;0;1080;
inverse-roi=0
class-id=-1
roi-zone_0=85;204;1736;204;1736;901;85;901

In my application I noticed that when a ROI is defined, Kafka messages are only sent for objects inside the ROI. The inverse-roi parameter allows switching between analyzing objects inside or outside the ROI, but not both simultaneously.

Is there a way to configure nvdsanalytics (or another approach) to send Kafka messages for both objects inside and outside the ROI within the same inference and stream?

Thanks

7

Here is a solution.

  1. add two ROI rect with the same coordinates, for example:
roi-RFframe_in=0;0;1920;0;1920;1080;0;1080;
roi-RFframe_out=0;0;1920;0;1920;1080;0;1080;
  1. modify nvdsanalytics_parse_roi_filtering_group by referring to this topic to let the different ROIs use the different inverse_roi. please refer to the following key code.
   if(i == 0) {
     roi.inverse_roi= 0;
   } else  if(i == 1){
    roi.inverse_roi= 1;
   }
  1. build libnvdsgst_dsanalytics.so according to /opt/nvidia/deepstream/deepstream-7.1/sources/gst-plugins/gst-nvdsanalytics/readme. then replace /opt/nvidia/deepstream/deepstream/lib/gst-plugins/libnvdsgst_dsanalytics.so.