Crop frame and save it in a path

emadomari40 · September 20, 2023, 11:34am

Please provide complete information as applicable to your setup.

Hardware Platform: Jetson
DeepStream Version: 6.0
JetPack Version 4.6.1

I want to get access to the frames in deepstream-app to crop ROI images and save the images to a specific path.

yuweiw · September 21, 2023, 3:24am

About how to save crop image, you can refer to the

static GstPadProbeReturn
osd_sink_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)

in opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-image-meta-test\deepstream_image_meta_test.c.

Below is where to add this part of the code in the deepstream-app:

static GstPadProbeReturn
gie_processing_done_buf_prob (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)

emadomari40 · September 21, 2023, 4:38am

I got this error:

deepstream_app.c:598:16: error: redefinition of ‘buf’
GstBuffer *buf = (GstBuffer *) info->data;
^~~
deepstream_app.c:592:14: note: previous definition of ‘buf’ was here
GstBuffer *buf = (GstBuffer *) info->data;
^~~
deepstream_app.c:608:3: warning: implicit declaration of function ‘cudaGetDevice’ [-Wimplicit-function-declaration]
cudaGetDevice(&current_device);
^~~~~~~~~~~~~
deepstream_app.c:609:25: error: storage size of ‘prop’ isn’t known
struct cudaDeviceProp prop;
^~~~
deepstream_app.c:610:3: warning: implicit declaration of function ‘cudaGetDeviceProperties’ [-Wimplicit-function-declaration]
cudaGetDeviceProperties(&prop, current_device);
^~~~~~~~~~~~~~~~~~~~~~~
deepstream_app.c:618:33: error: ‘PGIE_CLASS_ID_VEHICLE’ undeclared (first use in this function)
if (obj_meta->class_id == PGIE_CLASS_ID_VEHICLE) {
^~~~~~~~~~~~~~~~~~~~~
deepstream_app.c:618:33: note: each undeclared identifier is reported only once for each function it appears in
deepstream_app.c:622:33: error: ‘PGIE_CLASS_ID_PERSON’ undeclared (first use in this function); did you mean ‘PGIE_CLASS_ID_VEHICLE’?
if (obj_meta->class_id == PGIE_CLASS_ID_PERSON) {
^~~~~~~~~~~~~~~~~~~~
PGIE_CLASS_ID_VEHICLE
deepstream_app.c:631:27: error: ‘FILE_NAME_SIZE’ undeclared (first use in this function); did you mean ‘FILENAME_MAX’?
char fileNameString[FILE_NAME_SIZE];
^~~~~~~~~~~~~~
FILENAME_MAX
deepstream_app.c:641:23: error: ‘frame_number’ undeclared (first use in this function); did you mean ‘frame_meta’?
osd_string, frame_number, frame_meta->source_id, num_rects,
^~~~~~~~~~~~
frame_meta
deepstream_app.c:652:13: error: unknown type name ‘NvDsObjEncOutParams’; did you mean ‘NvDsObjectMeta’?
NvDsObjEncOutParams *enc_jpeg_image =
^~~~~~~~~~~~~~~~~~~
NvDsObjectMeta
deepstream_app.c:653:18: error: ‘NvDsObjEncOutParams’ undeclared (first use in this function); did you mean ‘NvDsObjectMeta’?
(NvDsObjEncOutParams *) usrMetaData->user_meta_data;
^~~~~~~~~~~~~~~~~~~
NvDsObjectMeta
deepstream_app.c:653:39: error: expected expression before ‘)’ token
(NvDsObjEncOutParams *) usrMetaData->user_meta_data;
^
deepstream_app.c:656:35: error: request for member ‘outBuffer’ in something not a structure or union
fwrite (enc_jpeg_image->outBuffer, sizeof (uint8_t),
^~
deepstream_app.c:657:31: error: request for member ‘outLen’ in something not a structure or union
enc_jpeg_image->outLen, file);
^~
Makefile:67: recipe for target ‘deepstream_app.o’ failed
make: *** [deepstream_app.o] Error 1
make: Leaving directory ‘/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app’

emadomari40 · September 21, 2023, 8:43am

I have added osd_sink_pad_buffer_probe function to deepstream_app.c as below:

static GstPadProbeReturn
osd_sink_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
  gchar *cropped_images_folder_path = "/opt/nvidia/deepstream/deepstream-6.0/DeepStream-Yolo";

  GstBuffer *buf = (GstBuffer *) info->data;
  guint num_rects = 0;
  NvDsObjectMeta *obj_meta = NULL;
  guint vehicle_count = 0;
  guint person_count = 0;
  NvDsMetaList *l_frame = NULL;
  NvDsMetaList *l_obj = NULL;
  NvDsDisplayMeta *display_meta = NULL;
  NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);
  int current_device = -1;
  cudaGetDevice(&current_device);
  struct cudaDeviceProp prop;
  cudaGetDeviceProperties(&prop, current_device);

  for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
    NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
    int offset = 0;
    for (l_obj = frame_meta->obj_meta_list; l_obj != NULL; l_obj = l_obj->next) {
      obj_meta = (NvDsObjectMeta *) (l_obj->data);
      if (obj_meta->class_id == 2) {
        vehicle_count++;
        num_rects++;
      }
      if (obj_meta->class_id == 0) {
        person_count++;
        num_rects++;
      }
      /* To verify  encoded metadata of cropped objects, we iterate through the
       * user metadata of each object and if a metadata of the type
       * 'NVDS_CROP_IMAGE_META' is found then we write that to a file as
       * implemented below.
       */
      char fileNameString[FILE_NAME_SIZE];
      const char *osd_string = "OSD";
      int obj_res_width = (int) obj_meta->rect_params.width;
      int obj_res_height = (int) obj_meta->rect_params.height;
      if(prop.integrated) {
        obj_res_width = GST_ROUND_DOWN_2(obj_res_width);
        obj_res_height = GST_ROUND_DOWN_2(obj_res_height);
      }

      snprintf (fileNameString, FILE_NAME_SIZE, "%s_%d_%d_%d_%s_%dx%d.jpg",
          osd_string, frame_number, frame_meta->source_id, num_rects,
          obj_meta->obj_label, obj_res_width, obj_res_height);
      /* For Demonstration Purposes we are writing metadata to jpeg images of
       * only vehicles for the first 100 frames only.
       * The files generated have a 'OSD' prefix. */
      if (frame_number < 100 && obj_meta->class_id == 2) {
        NvDsUserMetaList *usrMetaList = obj_meta->obj_user_meta_list;
        FILE *file;
        while (usrMetaList != NULL) {
          NvDsUserMeta *usrMetaData = (NvDsUserMeta *) usrMetaList->data;
          if (usrMetaData->base_meta.meta_type == NVDS_CROP_IMAGE_META) {
            NvDsObjEncOutParams *enc_jpeg_image =
                (NvDsObjEncOutParams *) usrMetaData->user_meta_data;
            /* Write to File */
            file = fopen (fileNameString, "wb");
            fwrite (enc_jpeg_image->outBuffer, sizeof (uint8_t),
                enc_jpeg_image->outLen, file);
            fclose (file);
            usrMetaList = NULL;
          } else {
            usrMetaList = usrMetaList->next;
          }
        }
      }
    }
    display_meta = nvds_acquire_display_meta_from_pool (batch_meta);
    NvOSD_TextParams *txt_params = &display_meta->text_params[0];
    txt_params->display_text = g_malloc0 (MAX_DISPLAY_LEN);
    offset =
        snprintf (txt_params->display_text, MAX_DISPLAY_LEN, "Person = %d ",
        person_count);
    offset =
        snprintf (txt_params->display_text + offset, MAX_DISPLAY_LEN,
        "Vehicle = %d ", vehicle_count);

    /* Now set the offsets where the string should appear */
    txt_params->x_offset = 10;
    txt_params->y_offset = 12;

    /* Font , font-color and font-size */
    txt_params->font_params.font_name = "Serif";
    txt_params->font_params.font_size = 10;
    txt_params->font_params.font_color.red = 1.0;
    txt_params->font_params.font_color.green = 1.0;
    txt_params->font_params.font_color.blue = 1.0;
    txt_params->font_params.font_color.alpha = 1.0;

    /* Text background color */
    txt_params->set_bg_clr = 1;
    txt_params->text_bg_clr.red = 0.0;
    txt_params->text_bg_clr.green = 0.0;
    txt_params->text_bg_clr.blue = 0.0;
    txt_params->text_bg_clr.alpha = 1.0;

    nvds_add_display_meta_to_frame (frame_meta, display_meta);
  }
  frame_number++;
  g_print ("Frame Number = %d Number of objects = %d "
      "Vehicle Count = %d Person Count = %d\n",
      frame_number, num_rects, vehicle_count, person_count);
  return GST_PAD_PROBE_OK;
}

and I called osd_sink_pad_buffer_probe inside gie_processing_done_buf_prob as below:

static GstPadProbeReturn
gie_processing_done_buf_prob (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
  GstBuffer *buf = (GstBuffer *) info->data;
  NvDsInstanceBin *bin = (NvDsInstanceBin *) u_data;
  guint index = bin->index;
  AppCtx *appCtx = bin->appCtx;

  if (gst_buffer_is_writable (buf))
    process_buffer (buf, appCtx, index);

  GstPadProbeReturn osd_sink_pad_buffer_probe(GstPad *pad, GstPadProbeInfo *info, void *u_data);


  return GST_PAD_PROBE_OK;
}

The code runs without errors but I am not getting any images on the folder.

yuweiw · September 21, 2023, 10:54am

It’s a API of probe function for Gstreamer. You can’t use it directly like that. We suggest that you run the deepstream-image-meta-test first to familiarize with the usage of the code. Then you need to familiarize with the deepstream-app demo code. Finally, you can add and debug the open source code.

emadomari40 · September 22, 2023, 5:11am

How to call the method in deepstream_app.c? just give me a simple example!

emadomari40 · September 25, 2023, 6:01am

any response?

yuweiw · September 25, 2023, 10:38am

You can just port the code from the osd_sink_pad_buffer_probe to the gie_processing_done_buf_prob . You need to briefly understand the probe mechanism of Gstreamer.

emadomari40 · September 25, 2023, 10:55am

This is what is happening in deepstream_image_meta_test.c, the probes have been added to the pipeline with their functions, but how to call the same functions to deepstream_app.c?

yuweiw · September 26, 2023, 8:04am

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

In our open source, the gie_processing_done_buf_prob function has been added to the piepline too.

opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-app\deepstream_app.c

  if (config->osd_config.enable) {
    NVGSTDS_ELEM_ADD_PROBE (instance_bin->all_bbox_buffer_probe_id,
        instance_bin->osd_bin.nvosd, "sink",
        gie_processing_done_buf_prob, GST_PAD_PROBE_TYPE_BUFFER, instance_bin);
  } else {
    NVGSTDS_ELEM_ADD_PROBE (instance_bin->all_bbox_buffer_probe_id,
        instance_bin->demux_sink_bin.bin, "sink",
        gie_processing_done_buf_prob, GST_PAD_PROBE_TYPE_BUFFER, instance_bin);
  }

system · October 10, 2023, 8:04am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Save image frame with bounding box in C application DeepStream SDK opencv , gstreamer , deepstream	7	677	January 22, 2024
Null value for DeepStream SDK	2	321	October 6, 2023
Issue with Cropping and Saving a Specific Frame Area in .jpg Format DeepStream SDK jetson , deepstream	18	73	March 10, 2025
Unable to add pgie_src_pad from deepstream-image-meta-test to deepstream-app DeepStream SDK	13	1189	December 29, 2021
Saving Full Image Frame (Uncropped) with Bounding Box DeepStream SDK	6	1478	February 22, 2022
Need some tips on debugging deepstream-app-based C++ application DeepStream SDK gstreamer	13	1939	June 1, 2022
Save image from sgie cropped DeepStream SDK gstreamer , deepstream	4	375	February 1, 2024
Cropping a stream and uploading RTP stream DeepStream SDK	4	973	October 12, 2021
Running image save in deepstream5 app issue DeepStream SDK camera , gstreamer	6	1185	October 12, 2021
The whole application is freezed in saving jpg image in deepstream_parallel_inference_app DeepStream SDK deepstream	10	29	March 5, 2025

Crop frame and save it in a path

Related topics