How to ensure GPU zero-copy buffer access with GStreamer appsink on Jetson (using NvBufSurface and CUDA)?

[Fiona.Chen]

@talwarmohit2005

Am I correctly achieving zero-copy GPU access using this method? ( I think i am not)

There are lots of customized code, we don’t know what happened. So we don’t know whether your code is “zero-copy GPU access”. But the pipeline nvv4l2camerasrc device=/dev/video0 name=mysource ! video/x-raw(memory:NVMM), width=1920, height=1020, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM), format=RGBA ! tee name=t t. ! queue leaky=2 max-size-buffers=10 ! nvvidconv name=myconv ! video/x-raw(memory:NVMM),format=RGBA ! appsink name=sink emit-signals=true max-buffers=1 drop=true itself is “zero-copy GPU access”.

From your code there is no CPU access until

@talwarmohit2005

    if (!surface->surfaceList[0].mappedAddr.eglImage)
    {
        std::cerr << "EGL Image is null. Skipping frame." << std::endl;
        return GST_PAD_PROBE_DROP;
    }

    status = cuGraphicsEGLRegisterImage(&pResource,
                                        surface->surfaceList[0].mappedAddr.eglImage,
                                        CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE);
    if (status != CUDA_SUCCESS)
    {
        printf("cuGraphicsEGLRegisterImage failed: %d \n", status);
    }
    status = cuGraphicsResourceGetMappedEglFrame(&eglFrame, pResource, 0, 0);

You’ve already got the CUDA array in “eglFrame”. CUDA Driver API :: CUDA Toolkit Documentation. All you succeeded operations should be CUDA operation on the “pArray” in “eglFrame”. Then you can guarantee there is only GPU envolved.
We don’t know what happened in your “ctx->inference->undistort”, “ctx->inference->runInference”,… You need to check by yourself.

If your code “ctx->inference->runInference” is for inferencing, we suggest you to deploy your inferencing model directly with DeepStream APIs.

@talwarmohit2005

Jetson Utils Snippet : I want to do this but directly via GStreamer / Deepstream SDK

#include <jetson-utils/videoSource.h>
#include <jetson-utils/cudaMappedMemory.h>
#include <jetson-utils/cudaNormalize.h>
#include <jetson-utils/cudaUtility.h>
#include <jetson-utils/cudaWarp.h>
#include <jetson-utils/cudaColorspace.h>
#include <jetson-utils/videoOptions.h>

int main(){
    cudaSetDevice(0);
    
    videoOptions opts;
    std::string video_name(argv[1]);
    if (video_name == "rear_cam")
    {
        char *fullpath = realpath("/dev/rear_cam", NULL);
        std::string resolvedStr(fullpath);
        free(fullpath);
        opts.resource = "v4l2://" + resolvedStr;
      
    }

    opts.width = 1920;
    opts.height = 1024;

    // Set framerate
    opts.frameRate = 30;

    // Set number of ring buffers
    opts.numBuffers = 8;

    // Enable zero-copy (recommended for Jetson)
    opts.zeroCopy = true;
    videoSource *video = videoSource::Create(opts);
    if (!video)
    {
        printf("Failed to create video source.\n");
        return -1;
    }

    while (true)
    {
        if (!video->Capture(&img, 1000))
        { // 1000 = timeout in milliseconds
            printf("No frame received, exiting...\n");
            break;
     

.... SEND TO INFERENCE >>>


.....

}
}

Please make sure your model is ONNX model and you know well about your model’s input and output. Then you can follow the instructions DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums to prepare the nvinfer parameters. There are lots of ONNX model deployment samples in DeepStream SDK, please refer to C/C++ Sample Apps Source Details — DeepStream documentation and NVIDIA-AI-IOT/deepstream_reference_apps: Samples for TensorRT/Deepstream for Tesla & Jetson