Need to save full frame from deepstream when no object is detected

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) - GPU
• DeepStream Version - 6.1.0
• Issue Type( questions, new requirements, bugs) - new requirements
• Requirement details
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I want to be able to encode full frames when there are no objects detected in a frame as well.

NvDsObjEncUsrArgs frameData {};
/* Preset */
frameData.isFrame = 1;
/* To be set by user */
frameData.saveImg = false;
frameData.attachUsrMeta = false;
/* Set if Image scaling Required */
frameData.scaleImg = (scaleImage == "true") ;
frameData.scaledWidth = fullFrameWidth_;
frameData.scaledHeight = fullFrameHeight_;
/* Quality */
frameData.quality = 80;

NvDsObjectMeta dummyObjMeta;
dummyObjMeta.rect_params.width = width;
dummyObjMeta.rect_params.height = height;
dummyObjMeta.rect_params.top = 0;
dummyObjMeta.rect_params.left = 0;

/* Main Function Call */
nvds_obj_enc_process (encCTX_, &frameData, surface, &dummyObjMeta, frameMeta);

NvDsFrameMeta* frame_meta  = nullptr;

for (auto listFrame = batchMeta->frame_meta_list; listFrame != nullptr; listFrame = listFrame->next)
{
    frame_meta  = reinterpret_cast<NvDsFrameMeta*>(listFrame->data);

    for (auto listObj = frame_meta ->obj_meta_list; listObj != nullptr; listObj = listObj->next)
    {

        auto objMeta = reinterpret_cast<NvDsObjectMeta*>(listObj->data);
        NvDsUserMetaList* usrMetaList = objMeta->obj_user_meta_list;
        for (; usrMetaList != nullptr; usrMetaList = usrMetaList->next)
        {
            auto usrMetaData = reinterpret_cast<NvDsUserMeta*>(usrMetaList->data);
            if (usrMetaData->base_meta.meta_type == NVDS_CROP_IMAGE_META)
            {
                // encode the jpeg binary buffer to base64
                auto* encJpegImage = reinterpret_cast<NvDsObjEncOutParams*>(usrMetaList->data);
                std::string msg = gst_create_full_frame_message(
                sink,
                encJpegImage,
                time);
                GST_DEBUG("Sending message: %s", msg.c_str());
                gst_send_message(sink, msg);
                break;
            }
        }
    }
}

But in my case the usrMetaList is always null.
Please advise.

3

You can refer to our demo code:opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-image-meta-test\deepstream_image_meta_test.c. As long as your frameData is available, it can be saved.

static GstPadProbeReturn
pgie_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer ctx)
{
...
      nvds_obj_enc_process (ctx, &frameData, ip_surf, NULL, frame_meta);
...

}
4

I did, that is how I am saving the image locally. But I need to encode it to base64 as well. So that I can use it as encJpegImage = (NvDsObjEncOutParams*) (data);

Anyway to do something like this ?

5

We’ll release this feature in our next version. About the issue of usrMetaList is null, did you set the frameData.attachUsrMeta = TRUE when you use the nvds_obj_enc_process?

6

I did, even attached a dummyObject with rect params set to the dimensions of the frame. Still user meta list is always null.

7

But from your source code, the parameter is false:frameData.attachUsrMeta = false;. If you want to get the usermetadata, you need to set the parameters to true.

8

Yes sorry. That was me trying to try different things when I run out of ideas. It is set to true I just ran it and same result. User-meta-list is null.

9

It’s werid that if you set the attachUsrMeta = true, you can definitely obtain user metadata. Could you use our demo code to reproduce your problem? opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-image-meta-test\deepstream_image_meta_test.c

10

Yes that works. That is where I got the code from. But the example stops after saving image locally after call to nvds_obj_enc_process (ctx, &objData, ip_surf, obj_meta, frame_meta);

Basically, I want to get the image and base64 encode it through accessing user-meta-list with:

image = reinterpret_cast<NvDsObjEncOutParams*>(usrMetaData->user_meta_data);

But as I mentioned before NvDsUserMetaList* usrMetaList = objMeta->obj_user_meta_list; is always null.

11

OK. If the demo works properly, there may be some issues with your code flow. You need to check the flow yourself, especially the location of the probe function. You can see from our demo, one is added to the pgie plugin, the other is added to the osd plugin.

12

I feel like we are talking in circles. I can save the image locally using nvds_obj_enc_process () function same as the example code. Which is able to save the image locally. However, when I want to encode the image that is when the metalist is null.

13

What I mean is that you need to ensure the timing of these two operations. You can see from our demo code deepstream_image_meta_test.c. We set the frameData.attachUsrMeta=true and use the nvds_obj_enc_process in the pgie_src_pad_buffer_probe API. But we get the usrMetaList in the osd_sink_pad_buffer_probe API.

In the next version, we have already implemented your similar requirement in this way, receiving image data and then base64 encryption， send the data from kafka. You can wait for our next version.

1 Like
14

Yes I followed the same way. But for me the frame_user_meta_list is always null. I feel like the sequence of events is incorrect, however, I have a work around working for me now. I will wait for the next version. Please let us know in this post once the next version is kicked up.

Thank you.

15

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

OK. You can also pay attention to our official web: deepstream-getting-started.

16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.