How to save frame as jpg and send the filename to kafka in deepstream 5.0 deepstream_test5

Yes i am doing that,
cv::cvtColor (in_mat, bgr_frame, cv::COLOR_RGBA2BGR);

Could you tell me on how to convert frame to RGBA in deepstream_app.c for multiple streams??

The sample code is get_converted_mat() in


It convert NV12(ip_surf) to RGBA(dsexample->inter_buf) in

err = NvBufSurfTransform (&ip_surf, dsexample->inter_buf, &transform_params);

NvBufSurfTransform is not converting frame to RGBA

/* Transformation scaling+format conversion if any. */
  err = NvBufSurfTransform (&ip_surf, dsexample->inter_buf, &transform_params);
  if (err != NvBufSurfTransformError_Success) {
        ("NvBufSurfTransform failed with error %d while converting buffer", err),
    goto error;
  else{ g_print("\nxxxxxxx %d %d\n",dsexample->video_info.finfo->format,GST_VIDEO_FORMAT_RGBA);

xxxxxxx 23 11

I am still getting segmentation fault error on cv::imwrite

Could you please share any sample code which can save image frames for multiple sources in deepstream_app.c ??

You probably don’t create dsexample->inter_buf in NVBUF_COLOR_FORMAT_RGBA. Please refer to the code of creating the buffer:

  /* An intermediate buffer for NV12/RGBA to BGR conversion  will be
   * required. Can be skipped if custom algorithm can work directly on NV12/RGBA. */
  create_params.gpuId  = dsexample->gpu_id;
  create_params.width  = dsexample->processing_width;
  create_params.height = dsexample->processing_height;
  create_params.size = 0;
  create_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
  create_params.layout = NVBUF_LAYOUT_PITCH;
#ifdef __aarch64__
  create_params.memType = NVBUF_MEM_DEFAULT;
  create_params.memType = NVBUF_MEM_CUDA_UNIFIED;

  if (NvBufSurfaceCreate (&dsexample->inter_buf, 1,
          &create_params) != 0) {
    GST_ERROR ("Error: Could not allocate internal buffer for dsexample");
    goto error;

I am doint that also , inside gstbasetransform_class->start = GST_DEBUG_FUNCPTR (gst_dsexample_start);

Below code inside deepstream_app.c is converting to rgba

 /* Set properties of the caps_filter element */
            caps_filter = gst_element_factory_make ("capsfilter", NULL);
        GstCaps *caps = gst_caps_new_simple ("video/x-raw", "format", G_TYPE_STRING, "RGBA",NULL);
        GstCapsFeatures *feature = gst_caps_features_new (MEMORY_FEATURES, NULL);
        gst_caps_set_features (caps, 0, feature);

        g_object_set (G_OBJECT (caps_filter), "caps", caps, NULL);
        gst_bin_add (GST_BIN (pipeline->pipeline), caps_filter);
          NVGSTDS_LINK_ELEMENT (caps_filter, last_elem);
          last_elem = caps_filter;
  gst_caps_unref (caps);

But getting segmentation fault while saving image

If i comment out the cv::imwrite() then it blur blurs the objects but tiler not working as expected, if i left click on display to open particular camera, it shows blank

You may check if you can apply

to gie_processing_done_buf_prob() in deepstream_app.c. We have verified it and you should have it work first.

1 Like

how do i use opencv library in deepstream_app.c?

You need to include necessary header files and modify Makefile

Header files:

/* Open CV headers */
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"

#include "nvbufsurface.h"
#include "nvbufsurftransform.h"


I am getting below error!!

cc -c -o deepstream_test5_app_main.o -DPLATFORM_TEGRA -I…/…/apps-common/includes -I /usr/local/cuda-10.2/include -I…/…/…/includes -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=5 pkg-config --cflags gstreamer-1.0 gstreamer-video-1.0 x11 opencv4 deepstream_test5_app_main.c
In file included from /usr/include/opencv4/opencv2/imgproc.hpp:46:0,
from /usr/include/opencv4/opencv2/imgproc/imgproc.hpp:48,
from deepstream_app.h:49,
from deepstream_test5_app_main.c:32:
/usr/include/opencv4/opencv2/core.hpp:49:4: error: #error core.hpp header must be compiled as C++
# error core.hpp header must be compiled as C++
In file included from /usr/include/opencv4/opencv2/core.hpp:52:0,
from /usr/include/opencv4/opencv2/imgproc.hpp:46,
from /usr/include/opencv4/opencv2/imgproc/imgproc.hpp:48,
from deepstream_app.h:49,
from deepstream_test5_app_main.c:32:
/usr/include/opencv4/opencv2/core/cvdef.h:690:4: error: #error “OpenCV 4.x+ requires enabled C++11 support”
# error “OpenCV 4.x+ requires enabled C++11 support”
/usr/include/opencv4/opencv2/core/cvdef.h:695:10: fatal error: array: No such file or directory
compilation terminated.
Makefile:58: recipe for target ‘deepstream_test5_app_main.o’ failed
make: *** [deepstream_test5_app_main.o] Error 1

You would need to refer to the Makefile and try to fix the compiling error.

An easier way is to enable dsexample in config file. You have mentioned you hit segment fault. Does it happen in running default sample? We have verified the default sample and this should not happen. You would need to check this first. To run the default dsexample successfully.

No this doesnt happen when running default app and even if dsexample is enabled

It happens only when i try to save images cv::imwrite

In gstdsexample.cpp , saving images is disabled, i.e. DSEXAMPLE_DEBUG is not defined.
Saving images works well with opencv-test-app but not with deepstream-test5

        /* Use openCV to remove padding and convert RGBA to BGR. Can be skipped if
        * algorithm can handle padded RGBA data. */
        cv::cvtColor (in_mat, *dsexample->cvmat, cv::COLOR_RGBA2BGR);
        cv::cvtColor (in_mat, *dsexample->cvmat, CV_RGBA2BGR);
        /* used to dump the converted mat to files for debug */
        static guint cnt = 0;
        cv::imwrite("out_" + std::to_string (cnt) + ".jpeg", *dsexample->cvmat);

We tried the following code and it works well. Pleas give it a try.

  static gint dump = 0;
  if (dump < 150) {
      char filename[64];
      snprintf(filename, 64, "/tmp/image%03d.jpg", dump);
      cv::imwrite(filename, *dsexample->cvmat);

Please also try deepstream-app. If you still hit segment fault in running deepstrem-app, please consider to do clean re-flash through SDKManager. We have verified the working well. A bit strange that the same patch does not work on your system. May help to do clean re-flash.

I am able to save the images, now i would like to send filename to kafka

how do i attach filename to metadata from gstdsexample.cpp?

I referred deepstream_user_metadata_app.c

Not able to retrieve the data from deepstream_test5_app_main.c

You would need to customize the metadata, please refer to

In gstdsexample.cpp, i have added below code

static gpointer meta_copy_func (gpointer data, gpointer user_data)
  NvDsUserMeta *user_meta = (NvDsUserMeta *) data;
  NvDsEventMsgMeta *srcMeta = (NvDsEventMsgMeta *) user_meta->user_meta_data;
  NvDsEventMsgMeta *dstMeta = (NvDsEventMsgMeta *) g_malloc (sizeof (NvDsEventMsgMeta));

  //dstMeta = g_memdup (srcMeta, sizeof (NvDsEventMsgMeta));

  if (srcMeta->frame_image){
    dstMeta->frame_image = g_strdup (srcMeta->frame_image);  }

  return dstMeta;

static void meta_free_func (gpointer data, gpointer user_data)
  NvDsUserMeta *user_meta = (NvDsUserMeta *) data;
  NvDsEventMsgMeta *srcMeta = (NvDsEventMsgMeta *) user_meta->user_meta_data;
  user_meta->user_meta_data = NULL;

  if (srcMeta->frame_image) {
    g_free (srcMeta->frame_image);

  delete srcMeta;

static GstFlowReturn
gst_dsexample_transform_ip (GstBaseTransform * btrans, GstBuffer * inbuf)
  GstDsExample *dsexample = GST_DSEXAMPLE (btrans);
  GstMapInfo in_map_info;
  GstFlowReturn flow_ret = GST_FLOW_ERROR;
  gdouble scale_ratio = 1.0;
  // DsExampleOutput *output;

  NvBufSurface *surface = NULL;
  NvDsBatchMeta *batch_meta = NULL;
  NvDsFrameMeta *frame_meta = NULL;
  NvDsMetaList * l_frame = NULL;
  guint i = 0;

  NvDsUserMeta *user_meta = NULL;   
  NvDsMetaType user_meta_type = NVDS_USER_FRAME_META_EXAMPLE;  

  CHECK_CUDA_STATUS (cudaSetDevice (dsexample->gpu_id),
      "Unable to set cuda device");

  memset (&in_map_info, 0, sizeof (in_map_info));
  if (!gst_buffer_map (inbuf, &in_map_info, GST_MAP_READ)) {
    g_print ("Error: Failed to map gst buffer\n");
    goto error;

  surface = (NvBufSurface *);
  GST_DEBUG_OBJECT (dsexample,
      "Processing Frame %" G_GUINT64_FORMAT " Surface %p\n",
      dsexample->frame_num, surface);

  if (CHECK_NVDS_MEMORY_AND_GPUID (dsexample, surface))
    goto error;

  batch_meta = gst_buffer_get_nvds_batch_meta (inbuf);
  if (batch_meta == nullptr) {
        ("NvDsBatchMeta not found for input buffer."), (NULL));
    return GST_FLOW_ERROR;

  if (dsexample->process_full_frame) {
        NvDsMetaList * l_obj = NULL;
    NvDsObjectMeta *obj_meta = NULL;
    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next)
      frame_meta = (NvDsFrameMeta *) (l_frame->data);
      NvOSD_RectParams rect_params;

      /* Scale the entire frame to processing resolution */
      rect_params.left = 0; = 0;
      rect_params.width = dsexample->video_info.width;
      rect_params.height = dsexample->video_info.height;

      /* Scale and convert the frame */
      if (get_converted_mat (dsexample, surface, i, &rect_params,
            scale_ratio, dsexample->video_info.width,
            dsexample->video_info.height, frame_meta, batch_meta, user_meta, user_meta_type) != GST_FLOW_OK) {
        goto error;
       for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
          l_obj = l_obj->next)
        obj_meta = (NvDsObjectMeta *) (l_obj->data);
        /** Generate NvDsEventMsgMeta for every object */

        NvDsEventMsgMeta *msg_meta =
            (NvDsEventMsgMeta *) g_malloc0 (sizeof (NvDsEventMsgMeta));
        char *fileNameString = new char [128];
        snprintf (fileNameString, 128, "d.jpg");
        msg_meta->frame_image = (gchar *) g_malloc0 (128);
        strncpy (msg_meta->frame_image, fileNameString, 128);
        NvDsUserMeta *user_event_meta =
            nvds_acquire_user_meta_from_pool (batch_meta);
        if (user_event_meta) {
           * Since generated event metadata has custom objects for
           * Vehicle / Person which are allocated dynamically, we are
           * setting copy and free function to handle those fields when
           * metadata copy happens between two components.
          user_event_meta->user_meta_data = (void *) msg_meta;
          user_event_meta->base_meta.batch_meta = batch_meta;
          user_event_meta->base_meta.meta_type = NVDS_EVENT_MSG_META;
          user_event_meta->base_meta.copy_func = (NvDsMetaCopyFunc) meta_copy_func;
          user_event_meta->base_meta.release_func = (NvDsMetaReleaseFunc) meta_free_func;
          nvds_add_user_meta_to_frame (frame_meta, user_event_meta);
        } else {
          g_print ("Error in attaching event meta to buffer\n");

flow_ret = GST_FLOW_OK;

  gst_buffer_unmap (inbuf, &in_map_info);
  return flow_ret;

In nvdsmeta_schema.h,

typedef struct NvDsEventMsgMeta {

  gchar *frame_image;

I am not able to fetch frame_image from nvmsgconv.cpp

 if (events[0].metadata->frame_image){
    json_object_set_string_member (rootObj, "frame_image", events[0].metadata->frame_image);
    cout << "\n frame_image --> "<< events[0].metadata->frame_image ;
    cout << "\n#frame_image \n " ;
    json_object_set_string_member (rootObj, "frame_image", "");}

You would need to add custom metadata. Please refer to the sample:

@prerana5626 As the pipeline image shows, dsexample is not before the msgconv by default, so you can’t receive the frame_image. You can move dsexample bin to the right position.

1 Like

I have the same problem. imwrite cause crash…

The same code i use deepstream-app works well, but use deepstream-test5 cause crash. when save the image.