DeepStream frame get to openCV Mat

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson NX Xavier
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Deepstream_Pose_Estimation sample code : GitHub - NVIDIA-AI-IOT/deepstream_pose_estimation: This is a sample DeepStream application to demonstrate a human pose estimation pipeline.
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I am developing applications using Deepstream Pose estimation Code through Xavier NX.

Under certain conditions, we want to save the frame as a jpg image, and we tried to run it by referring to the code in several topics in forum, but the results were as below.

Currently, I think the step setting value may be the cause of the problem when generating mat of opencv, but I couldn’t find a satisfactory solution, so I ask for your help.


first image is set to frame_step (surface->surfaceList[frame_meta->batch_id].pitch)
second image is set to cv:Mat:AUTO_STEP

below is my code

/* MetaData to handle drawing onto the on-screen-display */
static void create_display_meta(Vec2D &objects, Vec3D &normalized_peaks, NvDsFrameMeta *frame_meta, int frame_width, int frame_height, GstBuffer *buf)
int K = topology.size();
int count = objects.size();
NvDsBatchMeta *bmeta = frame_meta->base_meta.batch_meta;
NvDsDisplayMeta *dmeta = nvds_acquire_display_meta_from_pool(bmeta);
nvds_add_display_meta_to_frame(frame_meta, dmeta);

GstMapInfo map;
memset (&map, 0, sizeof (map));
NvBufSurface *surface = NULL;
cv::Mat frame;

if(gst_buffer_map(buf, &map, GST_MAP_READ)){
surface = (NvBufSurface *);

 NvBufSurfaceMap(surface, -1, -1, NVBUF_MAP_READ);
 NvBufSurfaceSyncForCpu(surface, -1, -1);

gint frame_width = (gint)surface->surfaceList[frame_meta->batch_id].width;
gint frame_height = (gint)surface->surfaceList[frame_meta->batch_id].height;
void *frame_data = surface->surfaceList[frame_meta->batch_id].mappedAddr.addr[0];
size_t frame_step = surface->surfaceList[frame_meta->batch_id].pitch;

frame = cv::Mat(frame_height, frame_width, CV_8UC4, frame_data, cv::Mat::AUTO_STEP);//frame_step-frame_width);

Mat out_mat = cv::Mat (cv::Size(MUXER_OUTPUT_WIDTH, MUXER_OUTPUT_HEIGHT), CV_8UC3);
cv::cvtColor (frame, out_mat, cv::COLOR_RGBA2BGR);
cv::imwrite("../../../ddmMP4/face/test2.jpg", frame);
cv::imwrite("../../../ddmMP4/face/test.jpg", out_mat);


NvBufSurfaceUnMap(surface, -1, -1);
gst_buffer_unmap (buf, &map);

For saving a whole frame to a JPEG file, please refer to this patch:
RTSP camera access frame issue - #19 by DaneLLL

1 Like

solved. Thankyou!