Yes. And the source code I attached before shows how to get the raw data from the NvbufSuface.
Hi ,I just see some codes elsewhere:
GstMapInfo in_map_info;
NvBufSurface* surface = NULL;memset (&in_map_info, 0, sizeof (in_map_info));
if (!gst_buffer_map (inbuf, &in_map_info, GST_MAP_READ)) {
g_print (“Error: Failed to map gst buffer\n”);
return -1;
}surface = (NvBufSurface *) in_map_info.data;
Do you think I can replace the codes below?
GstMapInfo map;
gst_buffer_map(buf, &map, GST_MAP_READ);
cv::Mat frame(cv::Size(frame_meta->source_frame_width, frame_meta->source_frame_height),
CV_8UC3, (uchar*)map.data, cv::Mat::AUTO_STEP);
gst_buffer_unmap(buf, &map);
So the combination maybe:
GstMapInfo map;
NvBufSurface* surface = NULL;
memset (&in_map_info, 0, sizeof (in_map_info));
if (!gst_buffer_map (inbuf, &map, GST_MAP_READ)) {
g_print ("Error: Failed to map gst buffer\n");
return -1;
}
surface = (NvBufSurface *) map.data;
cv::Mat frame(cv::Size(frame_meta->source_frame_width, frame_meta->source_frame_height),
CV_8UC3, (uchar*)surface, cv::Mat::AUTO_STEP);
gst_buffer_unmap(buf, &map);
So what about the codes above?I am looking forward to you reply!
NO. Please refer to the 142683 to learn how to get the raw data from the NvbufSurface.
Thank you for your apply! I have refered the codes,and add something:
//extra
NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta(buf);
// Get original raw data
GstMapInfo in_map_info;
if (!gst_buffer_map (buf, &in_map_info, GST_MAP_READ)) {
g_print ("Error: Failed to map gst buffer\n");
gst_buffer_unmap (buf, &in_map_info);
return GST_PAD_PROBE_OK;
}
NvBufSurface *surface = (NvBufSurface *)in_map_info.data;
for (NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL;
l_frame = l_frame->next) {
NvDsFrameMeta *frame_meta = l_frame->data;
//TODO for cuda device memory we need to use cudamemcpy
NvBufSurfaceMap (surface, -1, -1, NVBUF_MAP_READ);
/* Cache the mapped data for CPU access */
NvBufSurfaceSyncForCpu (surface, 0, 0); //will do nothing for unified memory type on dGPU
guint height = surface->surfaceList[frame_meta->batch_id].height;
guint width = surface->surfaceList[frame_meta->batch_id].width;
//Create Mat from NvMM memory, refer opencv API for how to create a Mat
Mat nv12_mat = Mat(height*3/2, width, CV_8UC1, surface->surfaceList[frame_meta->batch_id].mappedAddr.addr[0],
surface->surfaceList[frame_meta->batch_id].pitch);
//Convert nv12 to RGBA to apply algo based on RGBA
Mat rgba_mat;
cv::cvtColor(nv12_mat, rgba_mat, CV_YUV2BGRA_NV12);
So does the parameter rgba_mat contains the raw image of RGBA format of each frame?
Yes. This is just to show the scenario that the format of the source is NV12, you need to customize the features you need according to your own image format.
Thank you for your timely answer,making me a little bit more confident . I just check the structure of the pipeline of deepstream_app sample:
If I’m not mistaken,the video decode part of the pipeline uses the Gst-nvvideo4linux2 plugin:
It says
H264 decoder outputs GstBuffer in 8bit semi-planar(NV12) 4:2:0 format.
So if the input is a H264 source,the output will be the NV12 format,right?So maybe there is no need for me to make changes?
You need to verify the actual format with the following parameters in the NvbugSurface.
structNvBufSurface
And if I have car’s masks data ,could you tell me how to paint the masked area with a semi-transparent color in a similar way? thank you ,sir ~
Hi,what do you mean exactly?I have checked the parameter you provide,but find nothing about format.Could you please make it clearer?
I mean about this format issue, you can get that from the structure: NvBufSurfaceColorFormat color_format;.
This structure are open source, you can refer to the link I attached to you before, or directly refer to our source code: sources\includes\nvbufsurface.h.
Do you mean that I can print the NvBufSurfaceColorFormat color_format to see the exact format?
Yes. I have wrote a simple demo based on our sources\apps\sample_apps\deepstream-test1\deepstream_test1_app.c. You can refer to that by reading the probe function osd_sink_pad_buffer_probe.
deepstream_test1_app.c (16.2 KB)
Makefile (1.8 KB)
If it’s any other color format, you need to understand the layout of this format, and then get it from the NvbufSurface.