How to extract RGB data from a GstBuffer in multi streams？

[dao027]

DeepStreamSDK 6.3.0
c++
Jetson Xavier NX Jetpack 5.1.2 [L4T 35.4.1]

in my pipeline, i have 2 streams in nvstreammux, after nvtracker i want to extract rgb data of the frames, i have added 2 nvvideoconvert and 2 capsfilter in pipeline, the first converts from yuv to rgb:

GstCaps *caps_filter = gst_caps_from_string("video/x-raw(memory:NVMM), format=RGB, width=1920, height=1080");
    g_object_set(G_OBJECT(caps), "caps", caps_filter, NULL);
    gst_caps_unref(caps_filter);

the second converts from gpu to cpu:

GstCaps *caps_filter_d2h = gst_caps_from_string("video/x-raw, format=RGB, width=1920, height=1080");
    g_object_set(G_OBJECT(caps_d2h), "caps", caps_filter_d2h, NULL);
    gst_caps_unref(caps_filter_d2h);

after second convertor(caps filter), i add a probe function caps_src_pad_buffer_probe:

GstBuffer *buf = (GstBuffer *) info->data;
GstMapInfo map;
if (!gst_buffer_map(buf, &map, GST_MAP_READ)) {
    g_printerr("Failed to map buffer\n");
    gst_caps_unref(caps);
    return GST_PAD_PROBE_OK;
}
gsize size = map.size;
g_print("Mapped buffer size: %lu\n", size);

the size is 1920x1080x3 bytes, just 1 image , it should be 2 times bytes, don’t know y? how can i extract the 2 frames RGB data in cpu?
after i pass map.data to cv::Mat, i can save the image, .jpg file is ok

[yuweiw]

You can refer to our NvbufSurface(sources\includes\nvbufsurface.h). All the frames are stored in this structure. If there are more than 1 stream, there will be more than 1 surface in the NvBufSurfaceParams *surfaceList;.

[dao027]

ok，many thanks