How to save frame as jpg and send the filename to kafka in deepstream 5.0 deepstream_test5

I am getting below error!!

make
cc -c -o deepstream_test5_app_main.o -DPLATFORM_TEGRA -I…/…/apps-common/includes -I /usr/local/cuda-10.2/include -I…/…/…/includes -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=5 pkg-config --cflags gstreamer-1.0 gstreamer-video-1.0 x11 opencv4 deepstream_test5_app_main.c
In file included from /usr/include/opencv4/opencv2/imgproc.hpp:46:0,
from /usr/include/opencv4/opencv2/imgproc/imgproc.hpp:48,
from deepstream_app.h:49,
from deepstream_test5_app_main.c:32:
/usr/include/opencv4/opencv2/core.hpp:49:4: error: #error core.hpp header must be compiled as C++
# error core.hpp header must be compiled as C++
^~~~~
In file included from /usr/include/opencv4/opencv2/core.hpp:52:0,
from /usr/include/opencv4/opencv2/imgproc.hpp:46,
from /usr/include/opencv4/opencv2/imgproc/imgproc.hpp:48,
from deepstream_app.h:49,
from deepstream_test5_app_main.c:32:
/usr/include/opencv4/opencv2/core/cvdef.h:690:4: error: #error “OpenCV 4.x+ requires enabled C++11 support”
# error “OpenCV 4.x+ requires enabled C++11 support”
^~~~~
/usr/include/opencv4/opencv2/core/cvdef.h:695:10: fatal error: array: No such file or directory
#include
^~~~~~~
compilation terminated.
Makefile:58: recipe for target ‘deepstream_test5_app_main.o’ failed
make: *** [deepstream_test5_app_main.o] Error 1

Hi,
You would need to refer to the Makefile and try to fix the compiling error.

An easier way is to enable dsexample in config file. You have mentioned you hit segment fault. Does it happen in running default sample? We have verified the default sample and this should not happen. You would need to check this first. To run the default dsexample successfully.

No this doesnt happen when running default app and even if dsexample is enabled

It happens only when i try to save images cv::imwrite

In gstdsexample.cpp , saving images is disabled, i.e. DSEXAMPLE_DEBUG is not defined.
Saving images works well with opencv-test-app but not with deepstream-test5

////#ifdef DSEXAMPLE_DEBUG
        /* Use openCV to remove padding and convert RGBA to BGR. Can be skipped if
        * algorithm can handle padded RGBA data. */
#if (CV_MAJOR_VERSION >= 4)
        cv::cvtColor (in_mat, *dsexample->cvmat, cv::COLOR_RGBA2BGR);
#else
        cv::cvtColor (in_mat, *dsexample->cvmat, CV_RGBA2BGR);
#endif
        /* used to dump the converted mat to files for debug */
        static guint cnt = 0;
        cv::imwrite("out_" + std::to_string (cnt) + ".jpeg", *dsexample->cvmat);
        cnt++;
////#endif

Hi,
We tried the following code and it works well. Pleas give it a try.

  static gint dump = 0;
  if (dump < 150) {
      char filename[64];
      snprintf(filename, 64, "/tmp/image%03d.jpg", dump);
      cv::imwrite(filename, *dsexample->cvmat);
      dump++;
  }

Hi,
Please also try deepstream-app. If you still hit segment fault in running deepstrem-app, please consider to do clean re-flash through SDKManager. We have verified the working well. A bit strange that the same patch does not work on your system. May help to do clean re-flash.

I am able to save the images, now i would like to send filename to kafka

how do i attach filename to metadata from gstdsexample.cpp?

I referred deepstream_user_metadata_app.c

Not able to retrieve the data from deepstream_test5_app_main.c

Hi,
You would need to customize the metadata, please refer to

In gstdsexample.cpp, i have added below code

static gpointer meta_copy_func (gpointer data, gpointer user_data)
{
  NvDsUserMeta *user_meta = (NvDsUserMeta *) data;
  NvDsEventMsgMeta *srcMeta = (NvDsEventMsgMeta *) user_meta->user_meta_data;
  NvDsEventMsgMeta *dstMeta = (NvDsEventMsgMeta *) g_malloc (sizeof (NvDsEventMsgMeta));

  //dstMeta = g_memdup (srcMeta, sizeof (NvDsEventMsgMeta));

  if (srcMeta->frame_image){
    dstMeta->frame_image = g_strdup (srcMeta->frame_image);  }

  return dstMeta;
}


static void meta_free_func (gpointer data, gpointer user_data)
{
  NvDsUserMeta *user_meta = (NvDsUserMeta *) data;
  NvDsEventMsgMeta *srcMeta = (NvDsEventMsgMeta *) user_meta->user_meta_data;
  user_meta->user_meta_data = NULL;

  if (srcMeta->frame_image) {
    g_free (srcMeta->frame_image);
  }

  delete srcMeta;
}



static GstFlowReturn
gst_dsexample_transform_ip (GstBaseTransform * btrans, GstBuffer * inbuf)
{
  GstDsExample *dsexample = GST_DSEXAMPLE (btrans);
  GstMapInfo in_map_info;
  GstFlowReturn flow_ret = GST_FLOW_ERROR;
  gdouble scale_ratio = 1.0;
  // DsExampleOutput *output;

  NvBufSurface *surface = NULL;
  NvDsBatchMeta *batch_meta = NULL;
  NvDsFrameMeta *frame_meta = NULL;
  NvDsMetaList * l_frame = NULL;
  guint i = 0;

  NvDsUserMeta *user_meta = NULL;   
  NvDsMetaType user_meta_type = NVDS_USER_FRAME_META_EXAMPLE;  

  dsexample->frame_num++;
  CHECK_CUDA_STATUS (cudaSetDevice (dsexample->gpu_id),
      "Unable to set cuda device");

  memset (&in_map_info, 0, sizeof (in_map_info));
  if (!gst_buffer_map (inbuf, &in_map_info, GST_MAP_READ)) {
    g_print ("Error: Failed to map gst buffer\n");
    goto error;
  }

  surface = (NvBufSurface *) in_map_info.data;
  GST_DEBUG_OBJECT (dsexample,
      "Processing Frame %" G_GUINT64_FORMAT " Surface %p\n",
      dsexample->frame_num, surface);

  if (CHECK_NVDS_MEMORY_AND_GPUID (dsexample, surface))
    goto error;

  batch_meta = gst_buffer_get_nvds_batch_meta (inbuf);
  if (batch_meta == nullptr) {
    GST_ELEMENT_ERROR (dsexample, STREAM, FAILED,
        ("NvDsBatchMeta not found for input buffer."), (NULL));
    return GST_FLOW_ERROR;
  }

  if (dsexample->process_full_frame) {
        NvDsMetaList * l_obj = NULL;
    NvDsObjectMeta *obj_meta = NULL;
    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next)
    {
      frame_meta = (NvDsFrameMeta *) (l_frame->data);
      NvOSD_RectParams rect_params;

      /* Scale the entire frame to processing resolution */
      rect_params.left = 0;
      rect_params.top = 0;
      rect_params.width = dsexample->video_info.width;
      rect_params.height = dsexample->video_info.height;

      /* Scale and convert the frame */
      if (get_converted_mat (dsexample, surface, i, &rect_params,
            scale_ratio, dsexample->video_info.width,
            dsexample->video_info.height, frame_meta, batch_meta, user_meta, user_meta_type) != GST_FLOW_OK) {
        goto error;
      }
       for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
          l_obj = l_obj->next)
      {
        
        obj_meta = (NvDsObjectMeta *) (l_obj->data);
        /** Generate NvDsEventMsgMeta for every object */

        NvDsEventMsgMeta *msg_meta =
            (NvDsEventMsgMeta *) g_malloc0 (sizeof (NvDsEventMsgMeta));
        
        char *fileNameString = new char [128];
        
        snprintf (fileNameString, 128, "d.jpg");
        msg_meta->frame_image = (gchar *) g_malloc0 (128);
        strncpy (msg_meta->frame_image, fileNameString, 128);
      
        NvDsUserMeta *user_event_meta =
            nvds_acquire_user_meta_from_pool (batch_meta);
        if (user_event_meta) {
          /*
           * Since generated event metadata has custom objects for
           * Vehicle / Person which are allocated dynamically, we are
           * setting copy and free function to handle those fields when
           * metadata copy happens between two components.
           */
          user_event_meta->user_meta_data = (void *) msg_meta;
          user_event_meta->base_meta.batch_meta = batch_meta;
          user_event_meta->base_meta.meta_type = NVDS_EVENT_MSG_META;
          user_event_meta->base_meta.copy_func = (NvDsMetaCopyFunc) meta_copy_func;
          user_event_meta->base_meta.release_func = (NvDsMetaReleaseFunc) meta_free_func;
          nvds_add_user_meta_to_frame (frame_meta, user_event_meta);
        } else {
          g_print ("Error in attaching event meta to buffer\n");
        }
      }
    
    }

  }
flow_ret = GST_FLOW_OK;

error:
  gst_buffer_unmap (inbuf, &in_map_info);
  return flow_ret;
}

In nvdsmeta_schema.h,

typedef struct NvDsEventMsgMeta {

  gchar *frame_image;
.......
.......
.......
}

I am not able to fetch frame_image from nvmsgconv.cpp

 if (events[0].metadata->frame_image){
    json_object_set_string_member (rootObj, "frame_image", events[0].metadata->frame_image);
    cout << "\n frame_image --> "<< events[0].metadata->frame_image ;
    
  }
  else{
    cout << "\n#frame_image \n " ;
    json_object_set_string_member (rootObj, "frame_image", "");}

Hi,
You would need to add custom metadata. Please refer to the sample:


@prerana5626 As the pipeline image shows, dsexample is not before the msgconv by default, so you can’t receive the frame_image. You can move dsexample bin to the right position.

1 Like

I have the same problem. imwrite cause crash…

The same code i use deepstream-app works well, but use deepstream-test5 cause crash. when save the image.

Please help to open a new topic for your issue. Thanks

Hello, I tried to add a new callback function, but I can’t distinguish the stream frame. What should I do

This is where I add the callback function, because I don’t know exactly where the callback function should be added

I am using test5, and I want to save frame img, which place should I put these code ?

Hi DaneLLL,
I did’t find the function tracking_done_buf_prob() you mentioned in Saving frame with detected object [Jetson Nano, DS4.0.2] - #3 by DaneLLL

As I am using deepstream-test5 (deepstream 5.1) , I want to save frame image. I can get the batchmeta and framemata from “bbox_generated_probe_after_analytics ()” . can you tell me how to use this code? how can I get “*dsexample” ?

static gint dump = 0;
if (dump < 150) {
char filename[64];
snprintf(filename, 64, “/tmp/image%03d.jpg”, dump);
cv::imwrite(filename, *dsexample->cvmat);
dump++;
}

Please create a new topic for clearness. Thanks.

hi DaneLLL, thanks for your quick response, a new topic is created here: