Wrapped NvBufSurface image to VPIImage

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

• Hardware Platform (Jetson / GPU) Jetson AGX Orin
• DeepStream Version 7.0
• JetPack Version (valid for Jetson only) 6.0
• TensorRT Version 8.6.2.3
• Issue Type( questions, new requirements, bugs) Questions

Hi,
I have been working on using VPI3.0 algorithm in deepstream plugin.
Therefore, I need to wrap NvBurSurface to VPIImage.

I have already read this topic, it warp NvBufSurface by using NvBufSurfaceGetMapParams() to get the file descriptor.

However, this topic maybe only one stream in the batch, so he use NvBufSurfaceGetMapParams(surface, 0, &buffer_params), and it works on me when the batch size is one.
I tried multi-stream by changing NvBufSurfaceGetMapParams(surface, frame_meta->source_id, &buffer_params), and it pop out segmentation fault.

Am i using this API correctly? If not please help me to fix my code.

Here is my code:

gst_transform_ip (GstBaseTransform * btrans, GstBuffer * inbuf)
{
  GstFlowReturn flow_ret = GST_FLOW_ERROR;
  GstMapInfo in_map_info;
  NvBufSurface *surface = NULL;
  
  NvDsBatchMeta *batch_meta = NULL;
  NvDsFrameMetaList *l_frame = NULL;

  /* Get input buf */
  memset (&in_map_info, 0, sizeof (in_map_info));
  if (!gst_buffer_map (inbuf, &in_map_info, GST_MAP_READ)) {
    g_print ("Error: Failed to map gst buffer\n");
    goto error;
  }

  surface = (NvBufSurface *) in_map_info.data;

  batch_meta = gst_buffer_get_nvds_batch_meta (inbuf);
  if (batch_meta == nullptr){
    goto error;
  } 

  for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next)
  { 
    NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
    
    NvBufSurfaceMapParams buffer_params;
    VPIImage img_PL;

    // HERE IS THE SEG FAULT //
    NvBufSurfaceGetMapParams(surface, frame_meta->source_id, &buffer_params);

    img_data.bufferType = VPI_IMAGE_BUFFER_NVBUFFER;
    img_data.buffer.fd = buffer_params.fd;

    vpiImageCreateWrapper (&img_data, NULL, VPI_BACKEND_VIC, &img_PL)
    
    /* DO VPI ALGORITHM */
    /* END VPI ALGORITHM */
  }
  
  flow_ret = GST_FLOW_OK;
error:

  gst_buffer_unmap (inbuf, &in_map_info);
  return flow_ret;
}
3

The source id can’t be used as the surface index.

The index of the surface list is the same as the index of frame_meta_list. https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/struct__NvDsBatchMeta.html
https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/structNvBufSurface.html

for (int i=0, l_frame = batch_meta->frame_meta_list; (i<batch_meta->num_frames_in_batch) && (l_frame != NULL); i++ )
{
   NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
    
    NvBufSurfaceMapParams buffer_params;
    VPIImage img_PL;

    // HERE IS THE SEG FAULT //
    NvBufSurfaceGetMapParams(surface, i, &buffer_params);

    ..........
    .........
    l_frame = l_frame->next;
}
4

Thanks for reply. But, this code also cause segmentation fault.

for (int i=0, l_frame = batch_meta->frame_meta_list; (i<batch_meta->num_frames_in_batch) && (l_frame != NULL); i++ )
{
   NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
    
    NvBufSurfaceMapParams buffer_params;
    VPIImage img_PL;

    // HERE IS THE SEG FAULT //
    NvBufSurfaceGetMapParams(surface, i, &buffer_params);

    ..........
    .........
    l_frame = l_frame->next;
}
5

Can you get the call stack of the segment fault with gdb?

6

OK, I’ll try, but It needs a little time for me to learn how to use it.

So, this code works on your computer?

for (int i=0, l_frame = batch_meta->frame_meta_list; (i<batch_meta->num_frames_in_batch) && (l_frame != NULL); i++ )
{
   NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
    
    NvBufSurfaceMapParams buffer_params;
    VPIImage img_PL;

    // HERE IS THE SEG FAULT //
    NvBufSurfaceGetMapParams(surface, i, &buffer_params);

    ..........
    .........
    l_frame = l_frame->next;
}
7

Hi, Fiona
Here is the stack of segment fault signal that gdb shows

Thread 14 "queue2:src" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffff857ac840 (LWP 4133496)]
0x0000ffffedfed4a8 in ?? () from /usr/lib/aarch64-linux-gnu/nvidia/libnvbufsurface.so.1.0.0
(gdb) backtrace
#0  0x0000ffffedfed4a8 in  () at /usr/lib/aarch64-linux-gnu/nvidia/libnvbufsurface.so.1.0.0
#1  0x0000ffffe946419c [PAC] in gst_user_transform_ip(GstBaseTransform*, GstBuffer*) (btrans=<optimized out>, inbuf=0xffff28073240) at gstuser.cpp:470
#2  0x0000fffff723b744 in  () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#3  0x0000fffff723ac5c in  () at /lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#4  0x0000fffff7ec4978 in  () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#5  0x0000fffff7ec7bb8 in  () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#6  0x0000fffff7ec7fe8 in gst_pad_push () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#7  0x0000000000000000 in  ()
(gdb) up
#1  0x0000ffffe946419c [PAC] in gst_user_transform_ip (btrans=<optimized out>, inbuf=0xffff28073240) at gstuser.cpp:470
470	    NvBufSurfaceGetMapParams(surface, i, &buffer_params);
9

HI, any update?

10

I can reproduce the crash. We will investigate and fix the problem.