The difference of Multimedia API for capture

In samples,
iCaptureSession->createOutputStreamSettings has two types
STREAM_TYPE_BUFFER and STREAM_TYPE_EGL

By EGL mode , we got a stream , then get EGLImage from the stream.
By BUFFER mode, we create some Native Buffer, then get the EGLImage from Native Buffer.

Quetions are:

  1. What’s the difference between two modes? Do they have same latency?
  2. What is the type of the buffer created by NvBufferCreateEx ? It seem can be access by GPU, related by cudaMallocPitch. The buffer memory is device memory , Pinned host memory, or managed memory?
  3. The buffer name is “DMABuffer” in samples. Is it mean the buffer receive the CSI data by DMA mode? And the format is NV12 only?

Thank you for help!!

Hi,

The two modes are identical and you can choose one according to the usecase. If you only do GL/EGL function calls, you can set EGL mode. If you need hardware video encoding or format conversion, you can set BUFFER mode.

It is more like device memory and can be accessed by GPU. Please refer to HandleEGLImage().

Yes, after debayering through hardware ISP engine, the buffer format is NV12.

Thanks !

We concerned about two factors.

1.Memory copy, It’s really slow in CPU. The EGLImage based the buffer created by NvBufferCreateEx. It’s indeed can be accessed by GPU or coder/decoder by some convert method. Does memory copy involved in these converting?

2.Cache, Does the buffer has cache in cpu and gpu side both?

If you need hardware video encoding or format conversion, you can set BUFFER mode

We can get frame from stream in EGL mode, then encode it. What is the benefit for using Buffer mode?
Buffer mode using DMA , Does it mean less latency?

Hi,

For copying NvBuffer, you can call NvBufferTransform() which can do sclaing/cropping/format conversion.

Yes, need to synchronize the buffers by calling NvBufferMemSyncForCpu() or NvBufferMemSyncForDevice().

You can run sudo tegrastats to check if hardware encoder is enabled in your implementation. If it is enabled, it shall be good and you may not need to use NvBuffer.

Thanks!!

There is no memory copy in NvBuffer except explicit call NvBufferTransform or NvBuffer2Raw?

The Buffer’s behavior like "Managed Memory” but it not?

NvVideoEncoder is the api For hardware encoder ? It still need input data, Why Nvbuffer not to use?

Hi,

NvBufferTransform() is to copy NvBuffer to NvBuffer. It is hardware memory copy. NvBuffer2Raw() is to copy NvBuffer to CPU buffer. It is software memory copy.

It may not have exact same behavior as managed memory allocated through CUDA APIs. For accessing NvBuffer through CPU, need to map/unmap the buffer.

NvVideoEncoder is hardware encoder and input buffer is NvBuffer. Please refer to the sample in

/usr/src/jetson_multimedia_api/samples/01_video_encode

Document:
https://docs.nvidia.com/jetson/l4t-multimedia/index.html