Error when accessing frame in OpenCV in Jetson Nano

Hello, I’ve been trying to access the pipeline frames with OpenCV. I was able to get the frames correctly in my dGPU Desktop, but not in Jetson Nano. The error I’m getting is the following:

 

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.1.1) /home/dlinano/opencv/modules/core/src/cuda/gpu_mat.cu:249: error: (-217:Gpu API call) invalid argument in function 'download'

Aborted (core dumped)

I tested my OpenCV installation and it is not the problem. I’ve read in this link (https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_sample_custom_gstream.html) that “if memory of NvBufSurface is not in CUDA you must convert it to CUDA through CUDA-EGL interop before accessing it in OpenCV.” But I couldn’t find in the sources/gst-plugins/gst-dsexample/gstdsexample.cpp how to do that conversion and I’m not sure if this is my problem. If it is, what should I do? If it’s not what is causing this?

This is the sample code that I’m using inside of the osd_sink_pad_buffer_probe method:


static GstPadProbeReturn osd_sink_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data){
    GstBuffer *buf = (GstBuffer *) info->data;
    NvDsFrameMeta *frame_meta = NULL;
    guint vehicle_count = 0;
    guint person_count = 0;
    gboolean is_first_object = TRUE;
    NvDsMetaList *l_frame, *l_obj;

    NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);
    if (!batch_meta) {
        // No batch meta attached.
        return GST_PAD_PROBE_OK;
    }

    GstMapInfo in_map_info;
    NvBufSurface *surface = NULL;

    memset (&in_map_info, 0, sizeof (in_map_info));

    if (gst_buffer_map (buf, &in_map_info, GST_MAP_READWRITE)) {
        surface = (NvBufSurface *) in_map_info.data;

        NvBufSurfaceMap(surface, -1, -1, NVBUF_MAP_READ_WRITE);
        NvBufSurfaceSyncForCpu(surface, -1, -1);

        batch_meta = gst_buffer_get_nvds_batch_meta(buf);
        for (l_frame = batch_meta->frame_meta_list; l_frame; l_frame = l_frame->next) {
            frame_meta = (NvDsFrameMeta *) l_frame->data;

    #ifdef PLATFORM_TEGRA
            if (NvBufSurfaceMapEglImage(surface, frame_meta->batch_id) != 0)
                cout "ERROR on converting the EGL Buffer.";
    #endif

            gint frame_width = (gint) surface->surfaceList[frame_meta->batch_id].width;
            gint frame_height = (gint) surface->surfaceList[frame_meta->batch_id].height;

            cv::cuda::GpuMat frame;
            frame = cv::cuda::GpuMat(frame_height, frame_width, CV_8UC4,
                                     (void *) surface->surfaceList[frame_meta->batch_id].dataPtr,
                                     surface->surfaceList[frame_meta->batch_id].pitch);

            cv::Mat src_mat_BGRA;
            frame.download(src_mat_BGRA);
            cv::cvtColor(src_mat_BGRA, src_mat_BGRA, COLOR_RGBA2BGR);

     
            string filename = "cv_frames/frame_" + to_string(frame_number) + ".jpg";
            cv::imwrite(filename, src_mat_BGRA);
            ...
}

As you can see above I tried using this part of the code to resolve the issue, but I’ve got the same error I mentioned above.

#ifdef PLATFORM_TEGRA
            if (NvBufSurfaceMapEglImage(surface, frame_meta->batch_id) != 0){
                cout << "ERROR on converting the EGL Buffer." << endl;
            }
#endif

I’m using a modified version of deepstream-test1-app with a segmentation model that I’ve trained. Any help would be appreciated. Thanks in advance.

Can you also provide the application and configuration files you use?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Ok, I’m sorry. I’m using two systems a Jetson Nano and a dGPU Desktop, I’ll provide information about both. Remembering that the code is working fine in dGPU.

Dekstop dGPU Info:
• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 5.0.0
• TensorRT Version: 7.0.0.1
• NVIDIA GPU Driver Version (valid for GPU only): 450.57
• OpenCV Version: 4.4

Jetson Nano Info:
• Hardware Platform (Jetson / GPU): Jetson Nano
• DeepStream Version: 5.0.0
• JetPack Version (valid for Jetson only): 4.4.1
• TensorRT Version: 7.1.3.0
• OpenCV Version: 4.1.1

I’m using the following codes and config files. I’m also using the attached cmake file for compilation.

CMakeLists.txt (2.0 KB) dstest_segmentation_config_semantic.txt (3.6 KB) main.cpp (14.0 KB)

With dGPU, the official opencv in ubuntu does not support CUDA, so your application can not work. Have you rebuilt your own openCV?
This is an openCV issue but not DeepStream issue.

Hi,
Please check if you can run this sample:
How to create opencv gpumat from nvstream?

After the installation, you should see source of dsexample in

/opt/nvidia/deepstream/deepstream-5.0/sources/gst-plugins/gst-dsexample

Thank you @DaneLLL ! With this code sample you provided I was able to get the frames correctly in OpenCV with Jetson Nano. I merged this sample in my code and it worked. Thank you very much! Cheers!

1 Like