Custom pipeline Error(batch size Not 1)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi
I have a question

I made a gstreamer pipeline with cbao’s guide

Blockquote

                        |-----------infer valve ----tiler---- osd ---------| 
 two srcs--mux--infer --tee                                             funnel-overlaysink
                        |------------menuUI valve----- dsexample-----------|

Blockquote

[setting 1]
batchsize = 2
infervalve’s drop = FALSE
menuUIvalve’s drop = TRUE
tiler = 1rows, 2column
→ looks good

[setting 2]
batchsize= 2
infervalve’s drop = TRUE
menuUIvalve’s drop = FALSE
->throw error in (ERROR: batch size not 1)**

why throw this error?

help me.

thanks…

+++++++++++Add
I couldn’t find nvoverlaysink’s info in Nvidia Deepstream Details Doc
but i think nvoverlaysink’s input must contained only Single frame buffer
and think The reason of the Deepstream’s error is because the Dsexample’s output is ‘N batched data’

i want to know how to processing(N batched buffer → dsexample’s transform_ip Process → single frame buffer)
Like nvmultistreamtiler or compositor

transform_ip (GstBaseTransform * btrans, GstBuffer * inbuf)

can i change over from inbuf to my custom singleframe buffer(src+menu ui)?

3

Could you post the previous topic here, and also is it possible to share your code with us.

4

Hi @bcao

this is last topic

i made a pipeline follow your suggest

pipeline
pipeline752×190 2.5 KB

what I want to make is change over from streammux’s src data(N Batched data) to single batched data(composited) after processing my customplugin’s transform function

like i wrote before i think overlaysink has to contain only one batched data

i found function called “transform” in GstBaseTransform API Reference doc, i think if use it i can make i want

i tried.(referred to topic of similar problem)
my transform code is below, but something seems to be wrong.

static GstFlowReturn
gst_menudraw_transform (GstBaseTransform * btrans, GstBuffer * inbuf, GstBuffer * outbuf)
{
GstMenuDraw *menudraw = GST_MENUDRAW (btrans);
GstFlowReturn flow_ret = GST_FLOW_ERROR;
GstMapInfo in_map_info;
GstMapInfo out_map_info;
NvBufSurface *insurface = NULL;
NvBufSurface *outsurface = NULL;
NvDsBatchMeta *batch_meta = NULL;
NvDsMeta *meta = NULL;
NvDsFrameMeta *frame_meta = NULL;
g_print (“TransForm \n”);

if (!gst_buffer_map (inbuf, &in_map_info, GST_MAP_READWRITE)) {
g_print (“Error: Failed to in map gst buffer\n”);
goto error;
}
if (!gst_buffer_map (outbuf, &out_map_info, GST_MAP_READWRITE)) {
g_print (“Error: Failed to out map gst buffer\n”);
goto error;
}
outsurface = (NvBufSurface *)out_map_info.data;

batch_meta = nvds_create_batch_meta(1);
meta = gst_buffer_add_nvds_meta (outbuf , batch_meta, NULL, copy_user_meta, release_user_meta);
meta->meta_type = NVDS_BATCH_GST_META;
batch_meta->base_meta.batch_meta = batch_meta;
batch_meta->base_meta.copy_func = copy_user_meta;
batch_meta->base_meta.release_func = release_user_meta;
batch_meta->max_frames_in_batch = 1;
frame_meta = nvds_acquire_frame_meta_from_pool(batch_meta);
nvds_add_frame_meta_to_batch(batch_meta, frame_meta);
frame_meta->pad_index = 0;
frame_meta->source_id = 0;
frame_meta->frame_num = 0;
frame_meta->batch_id = 0;
frame_meta->source_frame_width = 1024;
frame_meta->source_frame_height = 600;
frame_meta->num_surfaces_per_frame = 1 ;

g_print(“out buf Patch size %u\n”,batch_meta->num_frames_in_batch);
Process_Draw_Menu(insurface, outsurface, outbuf);
flow_ret = GST_FLOW_OK;
g_print (“TransForm OK\n”);

error:
gst_buffer_unmap (inbuf, &in_map_info);
gst_buffer_unmap (outbuf, &out_map_info);
return flow_ret;
}

function of the Process_Draw_Menu

  1. Draw menu UI
  2. Draw overlay N camera sources a fixed area on 1.
  3. Finally i want to output buf has to 1 batch data

so i want to make to my plugin similar like nvmultistreamtiler(N batch data input → process → one batch data output)

I don’t know if the way I try is right.

Thanks.

5

++++++Add++++++++
Hi @bcao

I found some functions from gstbasetransform API for modifying output buffer, their names are “prepare_output_buffer” and “transform”

I added a GstBuffer to struct of _GstMenuDraw (The name is different from Dsexample and the structure is the same. ), and i made new GstBuffer on gst_menudraw_start().

code is below (name of the added gstbuffer is “outBuff”)

static gboolean
gst_menudraw_start (GstBaseTransform * btrans)
{
GstMenuDraw * menudraw = GST_MENUDRAW (btrans);
NvBufSurfaceCreateParams create_params;
MenuDrawInitParams init_params = { menudraw->processing_width, menudraw->processing_height,
menudraw->process_full_frame };
NvDsBatchMeta *batch_meta = NULL;
NvDsFrameMeta *frame_meta = NULL;
NvDsMeta *meta = NULL;
GstQuery *queryparams = NULL;
guint batch_size = 1;

/* Algorithm specific initializations and resource allocation. */
menudraw->menudrawlib_ctx = MenuDrawCtxInit (&init_params);

GST_DEBUG_OBJECT (menudraw, “ctx lib %p \n”, menudraw->menudrawlib_ctx);

CHECK_CUDA_STATUS (cudaSetDevice (menudraw->gpu_id),
“Unable to set cuda device”);

menudraw->batch_size = 1;
queryparams = gst_nvquery_batch_size_new ();
if (gst_pad_peer_query (GST_BASE_TRANSFORM_SINK_PAD (btrans), queryparams)
|| gst_pad_peer_query (GST_BASE_TRANSFORM_SRC_PAD (btrans), queryparams)) {
if (gst_nvquery_batch_size_parse (queryparams, &batch_size)) {
g_print(“parsed batch size = %u\n”,batch_size);
menudraw->batch_size = batch_size;
}
}
GST_DEBUG_OBJECT (menudraw, “Setting batch-size %d \n”,
menudraw->batch_size);
gst_query_unref (queryparams);

CHECK_CUDA_STATUS (cudaStreamCreate (&menudraw->cuda_stream),
“Could not create cuda stream”);

if (menudraw->inter_buf)
NvBufSurfaceDestroy (menudraw->inter_buf);
menudraw->inter_buf = NULL;
g_print(“parsed process w%u,h%d\n”,menudraw->processing_width, menudraw->processing_height);

/* An intermediate buffer for NV12/RGBA to BGR conversion will be

  • required. Can be skipped if custom algorithm can work directly on NV12/RGBA. */
    create_params.gpuId = menudraw->gpu_id;
    create_params.width = menudraw->processing_width;
    create_params.height = menudraw->processing_height;
    create_params.size = 0;
    create_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
    create_params.layout = NVBUF_LAYOUT_PITCH;
    ifdef aarch64
    create_params.memType = NVBUF_MEM_DEFAULT;
    else
    create_params.memType = NVBUF_MEM_CUDA_UNIFIED;
    endif
    //Make GST Buffer
    if (NvBufSurfaceCreate (&menudraw->inter_buf, 1,
    &create_params) != 0) {
    GST_ERROR (“Error: Could not allocate internal buffer for menudraw”);
    goto error;
    }

menudraw->outBuff = gst_buffer_new_wrapped_full (GST_MEMORY_FLAG_ZERO_PREFIXED, menudraw->inter_buf, sizeof(NvBufSurface), 0, sizeof(NvBufSurface), NULL, NULL);
batch_meta = nvds_create_batch_meta(1);
meta = gst_buffer_add_nvds_meta (menudraw->outBuff , batch_meta, NULL, copy_user_meta, release_user_meta);
meta->meta_type = NVDS_BATCH_GST_META;
batch_meta->base_meta.batch_meta = batch_meta;
batch_meta->base_meta.copy_func = copy_user_meta;
batch_meta->base_meta.release_func = release_user_meta;
batch_meta->max_frames_in_batch = 1;
frame_meta = nvds_acquire_frame_meta_from_pool(batch_meta);
nvds_add_frame_meta_to_batch(batch_meta, frame_meta);
// Some important parameters to fill
frame_meta->pad_index = 0;
frame_meta->source_id = 0;
frame_meta->buf_pts = 0;
frame_meta->ntp_timestamp = 0;
frame_meta->frame_num = 0;
frame_meta->batch_id = 0;
frame_meta->source_frame_width = 1024;
frame_meta->source_frame_height = 600;
frame_meta->num_surfaces_per_frame = 1 ;

// Create host memory for storing converted/scaled interleaved RGB data
CHECK_CUDA_STATUS (cudaMallocHost (&menudraw->host_rgb_buf,
menudraw->processing_width * menudraw->processing_height *
RGBA_BYTES_PER_PIXEL), “Could not allocate cuda host buffer”);

GST_DEBUG_OBJECT (menudraw, “allocated cuda buffer %p \n”,
menudraw->host_rgb_buf);

// CV Mat containing interleaved RGB data. This call does not allocate memory.
// It uses host_rgb_buf as data.
menudraw->cvmat =
new cv::Mat (menudraw->processing_height, menudraw->processing_width,
CV_8UC3, menudraw->host_rgb_buf,
menudraw->processing_width * RGBA_BYTES_PER_PIXEL);

if (!menudraw->cvmat)
goto error;

GST_DEBUG_OBJECT (menudraw, “created CV Mat\n”);

return TRUE;
error:
if (menudraw->host_rgb_buf) {
cudaFreeHost (menudraw->host_rgb_buf);
menudraw->host_rgb_buf = NULL;
}

if (menudraw->cuda_stream) {
cudaStreamDestroy (menudraw->cuda_stream);
menudraw->cuda_stream = NULL;
}
if (menudraw->menudrawlib_ctx)
MenuDrawCtxDeinit (menudraw->menudrawlib_ctx);
return FALSE;
}

next i made prepare_output_buffer() function

it does some processing ex : outBuff map & modity to surface data & outBuff unmap

below too

static GstFlowReturn
gst_menudraw_prepare_output_buffer(GstBaseTransform * btrans, GstBuffer * inbuf, GstBuffer ** outbuf)
{
GstMenuDraw *menudraw = GST_MENUDRAW (btrans);
GstFlowReturn flow_ret = GST_FLOW_ERROR;
GstMapInfo out_map_info;
NvBufSurface *outsurface = NULL;
NvDsBatchMeta *batch_meta = NULL;
g_print (“Prepare \n”);

CHECK_CUDA_STATUS (cudaSetDevice (menudraw->gpu_id),
“Unable to set cuda device”);

memset (&out_map_info, 0, sizeof (out_map_info));

if (!gst_buffer_map (menudraw->outBuff, &out_map_info, GST_MAP_WRITE)) {
g_print (“Error: Failed to out map gst buffer\n”);
goto error;
}

outsurface = (NvBufSurface *) out_map_info.data;
GST_DEBUG_OBJECT (menudraw,“Processing Frame Surface %p\n”,outsurface);
if (NvBufSurfaceMap (outsurface, -1, -1, NVBUF_MAP_READ_WRITE) != 0){
goto error;
}
NvBufSurfaceSyncForCpu(outsurface, -1, -1);
if (CHECK_NVDS_MEMORY_AND_GPUID (menudraw, outsurface))
{
g_print (“Error: Check NVDS Memory And Gpu id\n”);
goto error;
}

Process_Draw_Menu(outsurface);
outsurface->numFilled = 1;

NvBufSurfaceSyncForDevice(outsurface, -1, -1);
if (NvBufSurfaceUnMap (outsurface, -1, -1)){
goto error;
}
*outbuf = menudraw->outBuff;
flow_ret = GST_FLOW_OK;
g_print (“Prepare OK\n”);
error:
if(flow_ret == GST_FLOW_ERROR)
{
*outbuf = inbuf;
}
gst_buffer_unmap (menudraw->outBuff, &out_map_info);
g_print (“Prepare unmap OK\n”);
return flow_ret;
}

first time it has no problem, but it throw the map error after first

error code : “write map requested on non_writable buffer”

from this code

if (!gst_buffer_map (menudraw->outBuff, &out_map_info, GST_MAP_WRITE)) {
g_print (“Error: Failed to out map gst buffer\n”);
goto error;
}

and transform code(this function does nothing);

static GstFlowReturn
gst_menudraw_transform (GstBaseTransform * btrans, GstBuffer * inbuf, GstBuffer * outbuf)
{
GstMenuDraw *menudraw = GST_MENUDRAW (btrans);
GstFlowReturn flow_ret = GST_FLOW_ERROR;

flow_ret = GST_FLOW_OK;
g_print (“TransForm OK\n”);

error:

return flow_ret;
}

i think i do unmap clearly, but i don’t know why throw this errorcode from the second time

I wonder if I missed anything.

6

Up

7

Sry for the late reply, but could you share us a sample code which can be compiled and run well, so we can accelerate rooting cause the issue.

8

@bcao
How do i send you my code?

Just Upload ?

9

You can send me a private message if you have any concern.