Deepstream Additional Parameters

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, , ,
1

Hi,

I am using the deepstream for yoloV8 model with redis cache to get the predicted output for each frame, Using Deepstream docker image in that I am using deepstream_test5 app

Device info -

**• Hardware Platform (Jetson / GPU)** - Jetson Nano Dev Kit 4GB
**• DeepStream Version** - 6.0.1
**• JetPack Version (valid for Jetson only)** - 4.6.1
**• TensorRT Version** - 8.2.1.8
**• CUDA Version** - 10.2.300
**• cuDNN Version** - 8.2.1.32

configs -

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=UDPSink 5=nvoverlaysink 6=MsgConvBroker
type=6
msg-conv-config=dstest5_msgconv_sample_config.txt
#(0): PAYLOAD_DEEPSTREAM - Deepstream schema payload
#(1): PAYLOAD_DEEPSTREAM_MINIMAL - Deepstream schema payload minimal
#(256): PAYLOAD_RESERVED - Reserved type
#(257): PAYLOAD_CUSTOM   - Custom schema payload
msg-conv-payload-type=1
msg-broker-proto-lib=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_redis_proto.so
#Provide your msg-broker-conn-str here
msg-broker-conn-str=0.0.0.0;6379;deepstream_topic
topic=deepstream_topic
#Optional:
msg-broker-config=./configs/cfg_redis.txt
#msg-broker-config=../../deepstream-test4/cfg_kafka.txt
msg-conv-frame-interval=1

message-converter and message-consumer0 is not enabled, as the sink_type=6 includes the message conv. in default.

After few output data modification from redis cache, I got the output like this

42 Camera1 2025-04-11T11:07:31.523Z ['1|772.632|246.927|1061.42|1031.4|person']

Now, I need two extra values to send in the redis cache -
1. the confidence score for each object
2. the input frame for each frame_id

So, I gothorugh some documentation and in this directory /opt/nvidia/deepstream/deepstream-6.0/sources/libs/nvmsgconv/deepstream_schema/, the file name eventmsg_payload.cpp is handling the output parameters for each predicted objects

In line no, 671, a function is defined gchar* generate_event_message_minimal (void *privData, NvDsEvent *events, guint size)

I modified the function to get the confidence score for each object, for that I modified this line

ss << meta->trackingId << "|" << meta->bbox.left << "|" << meta->bbox.top << "|" << meta->bbox.left + meta->bbox.width << "|" << meta->bbox.top + meta->bbox.height << "|" << object_enum_to_str (meta->objType, meta->objectId);

to

ss << meta->trackingId << "|" << meta->bbox.left << "|" << meta->bbox.top << "|" << meta->bbox.left + meta->bbox.width << "|" << meta->bbox.top + meta->bbox.height << "|" << object_enum_to_str (meta->objType, meta->objectId) << "||" << meta->confidence;

then from redis cache, I got the confidence score for each object, but with value as ‘0’ for all detected object

I tried few modification, in other files also without knowing the C++, and none of the modification is succeeded to get the proper value of confidence.

I have checked the type of data the confidence is holdingtypeid(meta->confidence).name() and it returns d which is the double

image
image1310×618 83.4 KB

I checked this value meta->objType by adding this parameter to send in redis cache, and i got the ‘258’ value for this parameter, what this means

and also for meta->extMsg, meta->extMsgSize both are having the value 0

How to get the confidence value and the input image from deepstream , could you please help me on this, thanks

3

Refer to typedef enum NvDsObjectType in nvdsmeta_schema.h

  /** "object" key will be missing in the schema */
  NVDS_OBJECT_TYPE_UNKNOWN = 0x102,

I think this problem is caused by not modifying deepstream-test5 correctly. Please refer to the following code and comments.

/*
   * This demonstrates how to attach custom objects.
   * Any custom object as per requirement can be generated and attached
   * like NvDsVehicleObject / NvDsPersonObject. Then that object should
   * be handled in gst-nvmsgconv component accordingly.
   */
  if (model_used == APP_CONFIG_ANALYTICS_RESNET_PGIE_3SGIE_TYPE_COLOR_MAKE) {

You can set msg-conv-msg2p-new-api to 1, then you can directly access frame_metadata/object_metadata in dsmeta_payload.cpp.

#(0): Create payload using NvdsEventMsgMeta
#(1): New Api to create payload using NvDsFrameMeta
msg-conv-msg2p-new-api=0
4

@junshengy , thanks for writing back
I have modified this file dsmeta_payload.cpp and got the confidence score for every objects.

From reading this file, i get to know we can get the width, height, id, number of frame, etc from the frame_metadata

Could you explain about the structure of frame_metadata, to understand how the input frames or the cropped objects images from the input frames which are saving for the prediction.

your help is very much appreciated to understand the flows, thanks

5

Update:, I go through the documents, from this link MetaData in the DeepStream SDK — DeepStream documentation

I got to understand the structure

image
image699×498 87.8 KB

and also, I get to know that the input images are save in the buffer, how to access that from the frame_meta

if you know please share, thanks

6

Do you want to save images or objects and send them to redis?

Please refer to /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test4/deepstream_test4_app.c

Implemented in the pgie_src_pad_buffer_probe function using nvds_obj_enc_process and nvds_obj_enc_finish. Then access the user meta which type is NVDS_CROP_IMAGE_META.

Since you are using Jetson nano，please port this code from DS-7.1 to your version.

7

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.