DeepStream 7.x: How to correctly obtain NvBufSurface from GstBuffer in a custom GStreamer plugin?

[fghoussen]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU from inside docker
• DeepStream Version 7
• TensorRT Version 10
• Issue Type question
• Requirement details

Hello,

I am developing a custom GStreamer element for DeepStream 7.1 (TensorRT 10, NVMM zero-copy pipeline).

My plugin receives video/x-raw(memory:NVMM) buffers (RGBA) after nvstreammux + nvvideoconvert, and I want to run custom CUDA / TensorRT inference in a backend thread.

In DeepStream ≤6.x, examples and posts reference:

• NvDsBatchMeta->batch_buf

• NvDsBatchMeta->surface

• NVDS_USER_META_NVBUF_SURFACE

However, in DeepStream 7.x, none of these exist in the public headers (nvdsmeta.h, gstnvdsmeta.h).

I tried the following approaches, all of which fail or crash:

• gst_buffer_map() → map.data (segfault / invalid)

• iterating batch_user_meta_list

• looking for NVDS_USER_META_NVBUF_SURFACE

• accessing NvDsBatchMeta fields (no surface present)

I found that NVMM buffers are DMA-BUF backed in DS-7, so I attempted this approach:

#include <gst/allocators/gstdmabuf.h>
#include "nvbufsurface.h"

static NvBufSurface *
get_nvbufsurface_from_gstbuffer (GstBuffer *buf)
{
    GstMemory *mem = gst_buffer_peek_memory (buf, 0);
    if (!gst_is_dmabuf_memory (mem))
        return nullptr;

    int fd = gst_dmabuf_memory_get_fd (mem);
    if (fd < 0)
        return nullptr;

    NvBufSurface *surface = nullptr;
    if (NvBufSurfaceFromFd (fd, (void **) &surface) != 0)
        return nullptr;

    return surface;
}

This compiles, but I would like confirmation from NVIDIA:

1. Is DMA-BUF → NvBufSurfaceFromFd() the official and supported way in DeepStream 7.x to retrieve NvBufSurface in custom plugins?

2. Is this how gst-nvinfer internally accesses NVMM buffers in DS-7?

3. Are there any lifetime / synchronization constraints when using this surface in a backend thread (buffer is gst_buffer_ref()’d)?

4. Is there any other recommended public API for this use case?

This information does not seem to be documented clearly in the DeepStream 7 SDK docs, and examples never explicitly retrieve the surface.

Thank you for clarification.

[junshengy]

@fghoussen

Is DMA-BUF → NvBufSurfaceFromFd() the official and supported way in DeepStream 7.x to retrieve NvBufSurface in custom plugins?

Yes.NvBufSurfaceFromFd is offcial supported api.

@fghoussen

• Is this how gst-nvinfer internally accesses NVMM buffers in DS-7?

Function gst_nvinfer_buffer_get_memory in /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvinfer/gstnvinfer_allocator.cpp demostrate how to convert a GstBuffer to a batch buffer.

You can check this flow graph DeepStream SDK FAQ

Within the NvBufSurface structure, surfaceList holds a pointer to an array of batched buffers. Its type is NvBufSurfaceParams

@fghoussen

Are there any lifetime / synchronization constraints when using this surface in a backend thread (buffer is gst_buffer_ref()’d)?

The lifecycle of NvBufSurface is consistent with that of its parent GstBuffer instance.

@fghoussen

Is there any other recommended public API for this use case?

There is a public api documentation, please refer to this link

NVIDIA DeepStream SDK API Reference: NvBufSurface Types and Functions |...

[fghoussen]

Yes.NvBufSurfaceFromFd is offcial supported api.

OK. How to build / instanciate the fd argument to pass to NvBufSurfaceFromFd?

Is the code above get_nvbufsurface_from_gstbuffer (posted in the first message) correct?
When I use it, the returned surface is always surface = nullptr.

I retrieve the GstBuffer *buf provided by the GStreamer “transform_ip” callback, then I gst_buffer_ref the buffer (to make sure the buffer is still valid when I’ll need it later), then I push the buffer in a queue, then I pop this buffer back from the queue and pass it as argument of get_nvbufsurface_from_gstbuffer.

How to build arguments for NvBufSurfaceFromFd? I found no example of use:

$ grep -rni NvBufSurfaceFromFd /opt/nvidia/deepstream/deepstream-7.1/sources

/opt/nvidia/deepstream/deepstream-7.1/sources/includes/nvbufsurface.h:790:int NvBufSurfaceFromFd (int dmabuf_fd, void **buffer);

[junshengy]

gstnvinfer_allocator.cpp is the recommended approach, as it extracts the CUDA buffer from nvbufsurface.

If you insist on using NvBufSurfaceFromFd and have correctly flashed Jetpack, you can find the NvBufSurfaceFromFd sample code in the /usr/src/jetson_multimedia_api directory.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.