How can I obtain raw image datas in deepstream-app sample?

[Leozih]

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)**Jetson
• DeepStream Version6.1.1
**• Issue Type( questions, new requirements, bugs)**questions

Hi,guys！I am trying to get the angle of each detected object.To realize this ,I write the code below in the

gie_processing_done_buf_prob (GstPad * pad, GstPadProbeInfo * info,
gpointer u_data)

this function belongs to deepstream_app.c,the codes is :

  GstBuffer *buf = (GstBuffer *) info->data;
  NvDsInstanceBin *bin = (NvDsInstanceBin *) u_data;
  guint index = bin->index;
  AppCtx *appCtx = bin->appCtx;
  NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta(buf);
  for (NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
      NvDsFrameMeta *frame_meta = l_frame->data;
      for (NvDsMetaList * l_obj = frame_meta->obj_meta_list; l_obj != NULL;
        l_obj = l_obj->next) {
        NvDsObjectMeta *obj = (NvDsObjectMeta *) l_obj->data;
        cv::Rect bbox(obj->rect_params.left, obj->rect_params.top,
                      obj->rect_params.width, obj->rect_params.height);
        GstMapInfo map;
        gst_buffer_map(buf, &map, GST_MAP_READ);
        cv::Mat frame(cv::Size(frame_meta->source_frame_width, frame_meta->source_frame_height),
                      CV_8UC3, (uchar*)map.data, cv::Mat::AUTO_STEP);
        gst_buffer_unmap(buf, &map);
        cv::Mat object_img = frame(bbox);
        float angle;
        Ellipse_feature_extraction2(object_img,angle);

As you can see there are some codes involving opencv like cv::Rect which shouldn’t appear in C file.I will handle that later.
The logic,as you can see, is very simple.I want to cope with every object to get an angle using my costum function Ellipse_feature_extraction2 in every frame.
To realize that ,I need the frame by frame raw image and do ROI,so the codes:

        GstMapInfo map;
        gst_buffer_map(buf, &map, GST_MAP_READ);
        cv::Mat frame(cv::Size(frame_meta->source_frame_width, frame_meta->source_frame_height),
                      CV_8UC3, (uchar*)map.data, cv::Mat::AUTO_STEP);
        gst_buffer_unmap(buf, &map);

The codes above are the steps to get the raw image and do ROI,but it seems that map.data are not the raw image datas I want.
So my question is, to realize the opinions I mention,what changes the codes should make.

[yuweiw]

Please refer to our source code convert_batch_and_push_to_process_thread in the sources\gst-plugins\gst-dsexample\gstdsexample_optimized.cpp .
The inter_buf in that function is similar to your buf.

[Leozih]

I have roughly see the codes,it involve the dsexample plugin,if I don’t want to add the element in my pipeline,are there others way to get the raw images frames?

[yuweiw]

I’m not asking you to add the dsexample plugin, just refer to the source code it uses to process the buffers. We have many similar examples for this, like 142683 etc…

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks