Crop and rotate using src pad probe

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Xavier AGX
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) 4.5.1
• TensorRT Version 7.1.3
Following my topic here. I would like to crop and rotate the output of dewarper using src pad probe. My code is as follows:

static GstPadProbeReturn
dewarper_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data)
{
  GstBuffer *buf = (GstBuffer *) info->data;
  GstMapInfo outmap = GST_MAP_INFO_INIT;
  gst_buffer_map (buf, &outmap, GST_MAP_WRITE);
  NvBufSurface*  surface = (NvBufSurface *)outmap.data;

  NvBufSurfTransformRect src_rect, dst_rect;
  src_rect.top   = 0;
  src_rect.left  = 0;
  src_rect.width = (guint) surface->surfaceList[0].width / 2;
  src_rect.height= (guint) surface->surfaceList[0].height;

  dst_rect.top   = 0;
  dst_rect.left  = 0;
  dst_rect.width = 1200;
  dst_rect.height= 300;

  NvBufSurface *dst_surface = NULL;
  NvBufSurfaceCreateParams create_params;

  create_params.gpuId  = surface->gpuId;
  create_params.width  = 1200;
  create_params.height = 300;
  create_params.size = 0;
  create_params.colorFormat = surface->surfaceList[0].colorFormat;
  create_params.layout = surface->surfaceList[0].layout;
  create_params.memType = surface->memType;

  NvBufSurfaceCreate(&dst_surface,1,&create_params);

  NvBufSurfTransformParams transform_params;
  transform_params.src_rect = &src_rect;
  transform_params.dst_rect = &dst_rect;

  NvBufSurfTransformConfigParams transform_config_params;
  NvBufSurfTransform_Error err;

  transform_config_params.compute_mode = NvBufSurfTransformCompute_Default;
  transform_config_params.gpu_id = surface->gpuId;
  transform_config_params.cuda_stream = NULL;
  transform_params.transform_flag = NVBUFSURF_TRANSFORM_CROP_SRC | NVBUFSURF_TRANSFORM_FLIP;
  transform_params.transform_flip = NvBufSurfTransform_Rotate270;
  err = NvBufSurfTransformSetSessionParams (&transform_config_params);
  // crop and rotate 270 to dst surface
  err = NvBufSurfTransform (surface, dst_surface, &transform_params);
  if (err != NvBufSurfTransformError_Success) {
    g_print("NvBufSurfTransform failed with error %d while converting buffer", err);
    return GST_PAD_PROBE_DROP;
  }

  // copy back to original surface
  transform_params.transform_flag = 0;
  err = NvBufSurfTransform (dst_surface, surface, &transform_params);
  if (err != NvBufSurfTransformError_Success) {
    g_print("NvBufSurfTransform failed with error %d while converting buffer", err);
    return GST_PAD_PROBE_DROP;
  }
  surface->surfaceList[0].height = 300;
  surface->surfaceList[0].width = 1200;
  NvBufSurfaceDestroy(dst_surface);

  gst_buffer_unmap (buf, &outmap);
  return GST_PAD_PROBE_OK;
}


It appears to crop and rotate but output image is still 1000x2000 and setting the surfaceList height and width seems to be not enough. How do I properly do this?

3

Whose output? The downstream plugin or the whole pipeline?

4

I just attach the dewarper to the nvoverlaysink or nveglessink and check it. Whole pipeline looks like:

  GstPad *src_pad;
  src_pad = gst_element_get_static_pad (nvdewarper, "src");
  gst_pad_add_probe (src_pad, GST_PAD_PROBE_TYPE_BUFFER,
          dewarper_src_pad_buffer_probe, NULL, NULL);
  
  .....
  
  gst_bin_add_many (GST_BIN (pipeline), source, bayer2rgb, videoconvert,
          nvvideoconvert, nvdewarper, sink, NULL);
  if (!gst_element_link_many (source, bayer2rgb, videoconvert, nvvideoconvert, nvdewarper, sink, NULL)) {
      g_printerr ("Elements could not be linked. Exiting.\n");
      return -1;
  }
5

Got your idea. For the caps in the pipeline has already been decided when the pipeline is built(Caps negotiation), so to replace buffers inside probe is of no use for the downstream elements.

It is better to implement your processing(which needs new buffers and new caps) in a separated plugin.

1 Like