Deepstream - Python - Image and Msgbroker

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) : Jetson Orin Nano
• DeepStream Version : 7.1
• JetPack Version (valid for Jetson only) : 6.2
• TensorRT Version : 10.3.0.30

I am using deepstream docker image - nvcr.io/nvidia/deepstream:7.1-triton-multiarch, in that, i used the deepstream-imagedata-multistream app for testing,

this app has this flow -

streammux -> pgie -> nvvidconv1[for taking input frame] -> filter1[converting the input buffer to rgb format] -> tiler[for displaying multiple streams] -> nvvidconv[for osd] -> nvosd[plotting prediction on the input frame] -> sink

prob is added to the tiler element for getting input frame

and there is another app - 'deepstream-test4` for sending the objects info to cloud, in my experiment, I am using redis

filesrc -> h264parser -> nvv4l2decoder -> nvstreammux -> nvinfer -> nvvideoconvert -> nvdsosd -> tee

tee -> queue1 -> nvmsgconv -> nvmsgbroker

tee -> queue2 -> sink

My plan is to get the image and then send to redis, I modified the pipeline like this -

streammux -> pgie -> nvvidconv1[for taking input frame] -> filter1[converting the input buffer to rgb format] -> tiler[for displaying multiple streams] -> nvvidconv[for osd] -> nvosd[plotting prediction on the input frame] -> tee

tee -> queue1 -> nvmsgconv -> nvmsgbroker

tee -> queue2 -> sink

Added prob tiler_sink_pad_buffer_probe to tiler, and then in the same ,

Then for each object,

            user_event_meta = pyds.nvds_acquire_user_meta_from_pool(batch_meta)
            if user_event_meta:
                # Allocating an NvDsEventMsgMeta instance and getting
                # reference to it. The underlying memory is not manged by
                # Python so that downstream plugins can access it. Otherwise
                # the garbage collector will free it when this probe exits.
                msg_meta = pyds.alloc_nvds_event_msg_meta(user_event_meta)
                msg_meta.bbox.top = obj_meta.rect_params.top
                msg_meta.bbox.left = obj_meta.rect_params.left
                msg_meta.bbox.width = obj_meta.rect_params.width
                msg_meta.bbox.height = obj_meta.rect_params.height
                msg_meta.frameId = frame_number
                msg_meta.trackingId = long_to_uint64(obj_meta.object_id)
                msg_meta.confidence = obj_meta.confidence
                msg_meta = generate_event_msg_meta(msg_meta, obj_meta.class_id)

                user_event_meta.user_meta_data = msg_meta
                user_event_meta.base_meta.meta_type = pyds.NvDsMetaType.NVDS_EVENT_MSG_META
                pyds.nvds_add_user_meta_to_frame(frame_meta,
                                             user_event_meta)
            else:
                print("Error in attaching event meta to buffer\n")

def generate_event_msg_meta(data, class_id):
    meta = pyds.NvDsEventMsgMeta.cast(data)
    meta.sensorId = 0
    meta.placeId = 0
    meta.moduleId = 0
    meta.sensorStr = "sensor-0"
    meta.ts = pyds.alloc_buffer(MAX_TIME_STAMP_LEN + 1)
    pyds.generate_ts_rfc3339(meta.ts, MAX_TIME_STAMP_LEN)

    return meta

using this I got the object bbox in redis with msgconv payload-type as 1.

When I added the image like this -

                msg_meta.extMsg = frame_bytes
               msg_meta.extMsgSize = len(frame_bytes)
                msg_meta = generate_event_msg_meta(msg_meta, obj_meta.class_id)

                user_event_meta.user_meta_data = msg_meta
                user_event_meta.base_meta.meta_type = pyds.NvDsMetaType.NVDS_EVENT_MSG_META
                pyds.nvds_add_user_meta_to_frame(frame_meta,
                                             user_event_meta)

It is not sending any data, not even the bbox also

what is the not correct, and
how to add and send the images to redis through msgconv/msgbroker

thanks

3

please refer to ready-made sample deepstream-test4 for sending images to broker. please find ‘3.Send the image by the broker based on Kafka’ in readme of test4.

4

@fanzh , Thanks for writing back

Based on the test4 and deepstream_imagedata-multistream apps, i have combained, then only i had this issue

could you explain what to find

5

In “4. Usage:” section of readme in deepstream-test4, you can find “3.Send the image by the broker based on Kafka” text, which explains how to send images to broker.

6

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.