Creating GstBuffers using NvBufSurfaceCreate in app source

[abdo.babukr1]

Hardware Platform (Jetson / GPU): Quadro RTX4000
DeepStream Version: Deepstream 6.0
TensorRT Version: TensorRT 8.0.1
NVIDIA GPU Driver Version: 470.63.01

An example is shown here on how to create NVMM buffers on the jetson using NvBufferCreate():

https://forums.developer.nvidia.com/uploads/short-url/7Jw7yIt3zu4n6vIYXmCKTa2LomX.zip

How would we implement this on a discrete GPU using NvBufSurfaceCreate()? gst_nvds_buffer_pool_new() can create a pool of buffers, however it is causes jitter issues on aother running process when acquiring the buffer. I am specifically looking to generate NVMM buffers using NvBufSurfaceCreate() so I can have control over the allocation of the GPU memory.

[Fiona.Chen]

There is sample of how to use NvBufSurfaceCreate() in dsexample codes. Please refer to /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-dsexample

[abdo.babukr1]

@Fiona.Chen
Although NvBufSurfaceCreate() is used in the dsexample codes, it is not combined with any gstreamer functions that wrap the GPU buffer onto a GstBuffer. The example is for a transform plugin where a transformation is performed on the buffer created with NvBufSurfaceCreate() as opposed to a source plugin which generates the GstBuffer like the appsrc example in the zip file.

[Fiona.Chen]

Please refer to /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-nvdsvideotemplate for creating new NVBufSurfae with GstBufferPool

[abdo.babukr1]

Yes, the gst-nvdsvideotemplate example uses gst_nvds_buffer_pool_new() to generate a GstBuffer, I mentioned it is causing jitter on another time sensitive process which is ran persistently. The jitter is caused when the call is made to acquire the buffer using gst_buffer_pool_acquire_buffer. This method is timed to last over 100us and causes jitter up to 400us on another process. The reason why I am specifically looking to use NvBufSurfaceCreate() is because I would have control over the allocation of the memory which I’m suspecting is done repetitively with gst_nvds_buffer_pool_new().

[Fiona.Chen]

gst_nvds_buffer_pool_new() generates GstBuffers with NvBufSurface, and the GstBuffers can be used repeatedly in the pipeline. I don’t understand your description of the jitter issue, the buffers will be used in loop. You just need to create them once the pipeline is initialized.

@abdo.babukr1

it is causes jitter issues on aother running process when acquiring the buffer

[abdo.babukr1]

A separate process that’s not using deepstream has processing time within 1.9 to 2.1 ms. When running deepstream with the gst_nvds_buffer_pool_new() and gst_pool_acquire_buffer() the timing of the other mentioned process takes between 1.6 to 2.4 ms. So before the jitter was about ±100us and after it became ±400us.

I’m not sure if this is going to fix the jitter issue but I figured out how to create the GstBuffers without gst_nvds_buffer_pool_new() and gst_buffer_acquire_buffer and instead using NvBufSurfaceCreate():

On main:

    NvBufSurfaceCreateParams params = {0};
    params.gpuId = 0;
    params.width = 3840;
    params.height = 2160;
    params.isContiguous = false;
    params.layout = NVBUF_LAYOUT_PITCH;
    params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
    params.memType = NVBUF_MEM_DEFAULT;

    if(NvBufSurfaceCreate(&surf, 1, &params)==0)
        printf("success creating buffer surface\n");

    for (int i=0; i<600; i++) {
        pval += 32;
        feed_function(surf);
        usleep(16666);
    }

In feed_function:

static gboolean feed_function(NvBufSurface *surf) {

    GstBuffer *buffer;
    GstFlowReturn ret;
    GstMapInfo map = {0};
    GstMemoryFlags flags = (GstMemoryFlags)0;
    gsize memsize = sizeof(NvBufSurface);
    gpointer data = NULL;

    /* cuda goes here */    
    
    /* creating GstBuffer container */
    data = g_malloc(sizeof(NvBufSurface));
    buffer = gst_buffer_new_wrapped_full (flags, data, memsize, 0, memsize, gpointer(surf), notify_to_destroy);

    /* modifying surface */
    gst_buffer_map (buffer, &map, GST_MAP_READWRITE);
    memcpy(map.data, surf, memsize);
    /* release memory */
    gst_buffer_unmap(buffer, &map);

    g_signal_emit_by_name (appsrc_, "push-buffer", buffer, &ret);
    gst_buffer_unref(buffer);

    timestamp += 16666;
    return G_SOURCE_CONTINUE;
}