How to manually form a batch

Accelerated Computing Intelligent Video Analytics DeepStream SDK
22

I have mentioned “The batch meta should be align to your stream. You should split one stream to four streams if you modify batch meta in this way.”

23

Your transform plugin should be one sink pad and four src pads plugin if you want to generate such batch meta.

24

Or else you need to implement the surface list like what is done inside nvstreammux.

25

I am trying to implement it similar to nvstreammux but as it is closed-source I am not sure how. Now my prepare_output_buffer allocates output buffer as follows:

    if (NvBufSurfaceCreate (&batcher->inter_buf, 4,
				&create_params) != 0) {
		GST_ERROR ("Error: Could not allocate internal buffer for batcher");
		goto error;
	}

	*outbuf = NULL;
	*outbuf = gst_buffer_new_wrapped_full (GST_MEMORY_FLAG_ZERO_PREFIXED, batcher->inter_buf,4 * sizeof(NvBufSurface), 0,4 * sizeof(NvBufSurface), NULL, NULL);
26

We do not open source it because it is not encouraged to generate new batch without nvstreammux.

What I can tell you is that you need to generate new BufSurface and new surfaceList but not just use as it is.

27

I newly create inter_buf surface inside the prepare_output_buffer which should generate new surface everytime. Is it not enough?

28

You need new surfaceList to form the new batch.

This method also need new buffers and buffer copying, it will not improve performance compare to your “Using tee, nvvideoconvert, streammux” method.

29

To anyone who may have same problem, I have successfully formed a batch manually and visualized on the tiler by setting the frame_meta->pad_index same as batch_id. It is definitely faster than using tee on the original large source.

5 Likes
30

I was working on the same problem as you, looks like we both solved it approximately at the same time. I do still get a segfault if I try to demux the batched buffers with nvstreamdemux. Have you tested if your code works with that element by any chance? If it does, that would be some positive feedback and I just need to keep debugging for a bit.

Thanks in advance!

31

I am not sure about demux but I couldn’t apply nvinfer so I am trying to fix it. Does tiler also work on your batch? I would appreciate if you let me know when you debug it. Error I am getting on nvinfer + tiler is:

Running...
0:00:05.051540393 12808   0x55687a2590 WARN                 nvinfer gstnvinfer.cpp:1984:gst_nvinfer_output_loop:<nvinfer0> error: Internal data stream error.
0:00:05.052303462 12808   0x55687a2590 WARN                 nvinfer gstnvinfer.cpp:1984:gst_nvinfer_output_loop:<nvinfer0> error: streaming stopped, reason error (-5)
ERROR from element nvtiler: Memory Compatibility Error:Input surface gpu-id doesnt match with configured gpu-id for element, please allocate input using unified memory, or use same gpu-ids OR, if same gpu-ids are used ensure appropriate Cuda memories are used
Error details: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvtiler/gstnvtiler.cpp(639): gst_nvmultistreamtiler_transform (): /GstPipeline:object-tracker-pipeline/GstNvMultiStreamTiler:nvtiler:
surface-gpu-id=941047024,nvtiler-gpu-id=0
Returned, stopping playback
Deleting bus
Deleting pipeline
32

The tiler does work with my manual batching! I think you might want to double check the gpuId on the NvBufSurface?

33

Your issue might be related to this topic. I checked gpuId multiple times but the error seems to be inconsistent and possibly due to some race conditions inside the plugins. I will open a new issue with it.

34

@pesirnl I have found out that destroying previous inter_buf in prepare_output_buffer is causing segmentation fault in high load environment as the next element didn’t finish processing the contents of the inter_buf. Not destroying the inter_buf and reusing the old one is running well in my case.

1 Like
36

Hey there,
Can you by any chance share your code? or just the main part for creating the batch.

Best.

1 Like