Extracting DMA buffer information from NvBuffer

I’m trying to pass the output of NvBufferTransform to /dev/nvhost-msenc via libv4l2.

I am using cgo but the general flow is I create an NvBuffer to output to:

	frame:= &NvBuffer{}
	if ret := C.NvBufferCreateEx(&frame.fd, &params); ret != 0 {
		return nil, errors.New("error creating render buffer")
	}
	if ret := C.NvBufferMemMap(frame.fd, 0, C.NvBufferMem_Read_Write, &frame.start); ret != 0 {
		return nil, errors.New("error mapping render buffer")
	}

then I pass this buffer to NvBuffer to NvBufferTransform:

	if ret := C.NvBufferTransform(w.dmaBufs[buf.index].fd, frame.fd, &w.transformParams); ret != 0 {
		return errors.New("error transforming buffer")
	}

This works great, I can run NvBuffer2Raw on each of the Y U V planes and extract the raw image correctly. I then try to send this to an encoder:

	var params C.NvBufferParams
	if ret, err := C.NvBufferGetParams(frame.fd, &params); ret < 0 {
		return fmt.Errorf("failed to get buffer params: %w", err)
	}

	var buf C.struct_v4l2_buffer
	planes := (*[3]C.struct_v4l2_plane)(C.malloc(3 * C.sizeof_struct_v4l2_plane))
	defer C.free(unsafe.Pointer(planes))

	buf._type = C.V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
	buf.memory = C.V4L2_MEMORY_DMABUF
	buf.length = params.num_planes
	// set the planes
	*(*uintptr)(unsafe.Pointer(&buf.m)) = uintptr(unsafe.Pointer(planes))

	if e.activeOutputBuffers < e.numOutputBuffers {
		buf.index = e.activeOutputBuffers
		e.activeOutputBuffers++
	} else {
		// dequeue a buffer
		if ret, err := C.v4l2_ioctl(e.fd, C.VIDIOC_DQBUF, unsafe.Pointer(&buf)); ret < 0 {
			return fmt.Errorf("failed to dequeue output buffer: %w", err)
		}
	}

	log.Printf("%#v", params)
	for i := 0; i < int(params.num_planes); i++ {
		*(*C.uint)(unsafe.Pointer(&planes[i].m)) = params.dmabuf_fd
		planes[i].bytesused = params.psize[i]
		planes[i].data_offset = params.offset[i]
	}

	buf.flags |= C.V4L2_BUF_FLAG_TIMESTAMP_COPY
	buf.timestamp.tv_sec = C.long(frame.PTS.Microseconds() / 1000000)
	buf.timestamp.tv_usec = C.long(frame.PTS.Microseconds() % 1000000)

	if ret, err := C.v4l2_ioctl(e.fd, C.VIDIOC_QBUF, unsafe.Pointer(&buf)); ret < 0 {
		return fmt.Errorf("failed to enqueue output buffer: %w", err)
	}

However, the encoded bitstream on the capture plane terminates with a very large number of zeros in an NALU:

0000000140010c01ffff01400000030000030000030000030078ac090000000142010101400000030000030000030000030078a003c0801107cb96bb908444b820000000014401c077c0cc90000000012601ac461f59fffa5d244197f5c2e149bc434d0000030000030000030001a2a8f69ed0d3f68eef17a3c6a36048c7f043b7e6164c6d84557d...00000301f231785e4768e58905c347ae9086cf80f8753c2294c7b4000003000003003ae23809affeffdb33f93c7ef6e39a31133ed3912ef6bc000003000197f81a2db2906df719070b58208116164af289a040890d9ab088a0286bcb392f74b296c036bc87fba236bd8000025a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

When I play this back, I end up with a video that has a trailing black color:

image

What could be causing the encoder to spit out these incorrect NALUs? Because the NALU parses correctly, I’m led to believe that the 0’s are actually “correct” and the encoder is receiving bad input.

Is my conversion from NvBuffer to v4l2_buffer correct? I was not able to find an example that does this with a DMA buffer instead of an MMAP buffer.

Hi,
Please dump w.dmaBufs[buf.index].fd and check if the frame data is correct. And you can refer to this patch fo12_camera_v4l2_cuda + NvVideoEncoder:
TX2 Camera convert/encode using Multimedia API issue - #17 by DaneLLL

I exported the frame data, it renders the full image as expected. I suspect there is something in the translation between NvBuffer and v4l2_buffer? I noticed in the patch that there is this line:

enc_buf.m.planes[0].bytesused = 1; // byteused must be non-zero

Why is bytesused 1? I will try to reproduce the linked code a little more closely as well, thanks!

Hi,
It is a dummy value to let encoder know this buffer has frame data. Please do the same in your code.

I figured out what was wrong. In my buffer preparation, I didn’t export all the buffers because I forgot to set index on v4l2_exportbuffer.

Thanks @DaneLLL for the pointer to the updated sample, it eventually led me to discover what was different.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.