How to use Jetson's unique hardware to save images in probe callback functions (OpenCV format)

[LFYTMLY]

• Hardware Platform (Jetson / GPU)：Jetson orin nano
• DeepStream Version：6.2
• JetPack Version (valid for Jetson only)：5.1.4

My pipeline is nvurisrcbin ->streammux ->nvinfer ->nvvideoconvert (nvbuf mem surface array) ->capsfilter (“video/x-row (memory: NFVM), format=RGBA”) ->fakesink. Add probe callback function to the sinkpad of fakesink and use

 GstMapInfo in_map_info;

if (!gst_buffer_map(buf, &in_map_info, GST_MAP_READ))
{
    return GST_PAD_PROBE_DROP;
}

if (NvBufSurfaceMap(nvbuf_surface, frame_meta->batch_id, 0, NVBUF_MAP_READ) != 0)
{
    continue;
}

if (NvBufSurfaceSyncForCpu(nvbuf_surface, frame_meta->batch_id, 0) != 0)
{
    NvBufSurfaceUnMap(nvbuf_surface, frame_meta->batch_id, 0);
    continue;
}

 cv::Mat img_dest;
 cv::Mat img_rgba = cv::Mat(pipeline_h, pipeline_w, CV_8UC4, 
                            nvbuf_surface->surfaceList[frame_meta->batch_id].mappedAddr.addr[0], 
                            nvbuf_surface->surfaceList[frame_meta->batch_id].pitch);
cv::cvtColor(img_rgba, img_dest, cv::COLOR_RGBA2RGB);

and save the image using the imwrite function again.

However, saving images with imwrite is a bit time-consuming (with a lot of I/O processes), and I ran the compiled code from /usr/src/jetson_maultimedia_api/samples/05_jpeg_decode and found that 05_jpeg_decode used NVJPEG acceleration when saving images, but this code read the image file before saving, and NVUTILS.H functions provided in the library are all implemented based on file streams. I would like to ask if there is any way to hardware accelerate the saving of OpenCV format images or NV12 format images (I can modify my code to not convert to OpenCV format).

[yuweiw]

You can refer to our sample code deepstream-image-meta-test to use the nvds_obj_enc_process to save images with our hardware resource.

[LFYTMLY]

@yuweiw
I have also seen this component, but now I am not using nvosd. Instead, I retrieve bbox, CLS, and conf from the metadata and use OpenCV to draw them. In the probe callback function, I need to save the original image and draw the bbox image, which is a synchronous and time-consuming process. That’s why I need help with hardware saving images.

[yuweiw]

So are you trying to figure out how to save OpenCV NV12 data as jpeg images with hardware acceleration?
You can try researching opencv-cuda to see if it meets your needs.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks