Adding Preprocessing to Frames RTSP

Accelerated Computing Intelligent Video Analytics DeepStream SDK
12

Dear Jason,

Thanks for your suggestion.
You are right, we can find many examples on OpenCV about how to achieve the sepia filtering.
However, I do not understand how to modify the gstdsexample.cpp.

I have added the element in the pipeline as described above. The pipeline is generated correctly.

I believe that cv.transform would do the job of filtering:

cv::Mat kernel =
(cv::Mat_<float>(3, 3)
	<<
	0.272, 0.534, 0.131,
	0.349, 0.686, 0.168,
	0.393, 0.769, 0.189);
[...]
cv::transform(input_img, output_img, kernel);

I tried to do the cv::transform inside the function “get_converted_mat”, just after the color conversion:

  in_mat =
  cv::Mat (dsexample->processing_height, dsexample->processing_width,
  CV_8UC4, dsexample->inter_buf->surfaceList[0].mappedAddr.addr[0],
  dsexample->inter_buf->surfaceList[0].pitch);

 out_mat =
  cv::Mat (dsexample->processing_height, dsexample->processing_width,
  CV_8UC4);

#if (CV_MAJOR_VERSION >= 4)
  cv::cvtColor (in_mat, out_mat, cv::COLOR_RGBA2BGR);
#else
  cv::cvtColor (in_mat, out_mat, CV_RGBA2BGR);
#endif 

  cv::transform(out_mat, *dsexample->cvmat, kernel);

The function get_converted_mat is called by the function gst_dsexample_transform_ip, thus I forced the latter to process the full frame. Below you can see how get_converted_mat is called:

if (true) {
    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next)
    {
      frame_meta = (NvDsFrameMeta *) (l_frame->data);
      NvOSD_RectParams rect_params;

      /* Scale the entire frame to processing resolution */
      rect_params.left = 0;
      rect_params.top = 0;
      rect_params.width = dsexample->video_info.width;
      rect_params.height = dsexample->video_info.height;

      /* Scale and convert the frame */
      if (get_converted_mat (dsexample, surface, i, &rect_params,
            scale_ratio, dsexample->video_info.width,
            dsexample->video_info.height) != GST_FLOW_OK) {
        goto error;
      }

      /* Process to get the output */
      output =
          DsExampleProcess (dsexample->dsexamplelib_ctx,
          dsexample->cvmat->data);
      /* Attach the metadata for the full frame */
      //attach_metadata_full_frame (dsexample, frame_meta, scale_ratio, output, i);
      i++;
      free (output);
    }
  }

Unfortunately, the only result that I obtain from the console when I run the pipeline described in my previous post is the following message after each frame processing:
nvbufsurface: Wrong buffer index (0)

I hope you could help me to understand where to edit the gstdsexample.cpp to correctly apply the cv.transform function and output the processed frames to the next element of the deepstream pipelin.

Thank you very much again!!