I’m trying to pass the output of NvBufferTransform to /dev/nvhost-msenc via libv4l2.
I am using cgo but the general flow is I create an NvBuffer to output to:
frame:= &NvBuffer{}
if ret := C.NvBufferCreateEx(&frame.fd, ¶ms); ret != 0 {
return nil, errors.New("error creating render buffer")
}
if ret := C.NvBufferMemMap(frame.fd, 0, C.NvBufferMem_Read_Write, &frame.start); ret != 0 {
return nil, errors.New("error mapping render buffer")
}
then I pass this buffer to NvBuffer to NvBufferTransform:
if ret := C.NvBufferTransform(w.dmaBufs[buf.index].fd, frame.fd, &w.transformParams); ret != 0 {
return errors.New("error transforming buffer")
}
This works great, I can run NvBuffer2Raw on each of the Y U V planes and extract the raw image correctly. I then try to send this to an encoder:
var params C.NvBufferParams
if ret, err := C.NvBufferGetParams(frame.fd, ¶ms); ret < 0 {
return fmt.Errorf("failed to get buffer params: %w", err)
}
var buf C.struct_v4l2_buffer
planes := (*[3]C.struct_v4l2_plane)(C.malloc(3 * C.sizeof_struct_v4l2_plane))
defer C.free(unsafe.Pointer(planes))
buf._type = C.V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
buf.memory = C.V4L2_MEMORY_DMABUF
buf.length = params.num_planes
// set the planes
*(*uintptr)(unsafe.Pointer(&buf.m)) = uintptr(unsafe.Pointer(planes))
if e.activeOutputBuffers < e.numOutputBuffers {
buf.index = e.activeOutputBuffers
e.activeOutputBuffers++
} else {
// dequeue a buffer
if ret, err := C.v4l2_ioctl(e.fd, C.VIDIOC_DQBUF, unsafe.Pointer(&buf)); ret < 0 {
return fmt.Errorf("failed to dequeue output buffer: %w", err)
}
}
log.Printf("%#v", params)
for i := 0; i < int(params.num_planes); i++ {
*(*C.uint)(unsafe.Pointer(&planes[i].m)) = params.dmabuf_fd
planes[i].bytesused = params.psize[i]
planes[i].data_offset = params.offset[i]
}
buf.flags |= C.V4L2_BUF_FLAG_TIMESTAMP_COPY
buf.timestamp.tv_sec = C.long(frame.PTS.Microseconds() / 1000000)
buf.timestamp.tv_usec = C.long(frame.PTS.Microseconds() % 1000000)
if ret, err := C.v4l2_ioctl(e.fd, C.VIDIOC_QBUF, unsafe.Pointer(&buf)); ret < 0 {
return fmt.Errorf("failed to enqueue output buffer: %w", err)
}
However, the encoded bitstream on the capture plane terminates with a very large number of zeros in an NALU:
0000000140010c01ffff01400000030000030000030000030078ac090000000142010101400000030000030000030000030078a003c0801107cb96bb908444b820000000014401c077c0cc90000000012601ac461f59fffa5d244197f5c2e149bc434d0000030000030000030001a2a8f69ed0d3f68eef17a3c6a36048c7f043b7e6164c6d84557d...00000301f231785e4768e58905c347ae9086cf80f8753c2294c7b4000003000003003ae23809affeffdb33f93c7ef6e39a31133ed3912ef6bc000003000197f81a2db2906df719070b58208116164af289a040890d9ab088a0286bcb392f74b296c036bc87fba236bd8000025a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
When I play this back, I end up with a video that has a trailing black color:
What could be causing the encoder to spit out these incorrect NALUs? Because the NALU parses correctly, I’m led to believe that the 0’s are actually “correct” and the encoder is receiving bad input.
Is my conversion from NvBuffer to v4l2_buffer correct? I was not able to find an example that does this with a DMA buffer instead of an MMAP buffer.