[Jetson] Sharing Vulkan buffer data to NVidia video encoder (V4L2)

Hello,

I have been looking for samples/examples showing how to transfer data from a Vulkan context to V4L2 (NvBuffer) without passing by the CPU, but couldn’t find anything.

I am trying to encode frames rendered in Vulkan using the V4L2 Video encoder API on the Jetson.
I managed to generate a valid video by copying frame data from the vulkan context to the CPU and then from the CPU to NvBuffers, but for obvious performance reasons I would like to keep data on the GPU and copy it directly from the Vulkan context to NvBuffers.

I tried 2 different ways so far without success:

1# Create a NvBuffer using NvBufferCreateEx and import it in Vulkan as an external memory

int dmabuff_fd;
NvBufferCreateParams cParams;
cParams.width = ctx.width;
cParams.height = ctx.height;
cParams.layout = NvBufferLayout_Pitch;
cParams.colorFormat = NvBufferColorFormat_YUV444;
cParams.nvbuf_tag = NvBufferTag_VIDEO_ENC;
cParams.payloadType = NvBufferPayload_SurfArray;
ret = NvBufferCreateEx(&dmabuff_fd, &cParams);

...

vk::ImportMemoryFdInfoKHR importMemInfo{};
importMemInfo.handleType = vk::ExternalMemoryHandleTypeFlagBits::eOpaqueFd;
importMemInfo.fd = dmabuff_fd;

vk::MemoryAllocateInfo allocInfo = {};
allocInfo.pNext = &importMemInfo;
allocInfo.allocationSize = memReqs.size;
allocInfo.memoryTypeIndex = Vulkan::findMemoryTypeIndex(memReqs, memProps, vk::MemoryPropertyFlagBits::eDeviceLocal);
auto bufferMemory = device.allocateMemoryUnique(allocInfo);

But this fails with the following error during the call to allocateMemoryUnique:

PosixMemGetSize: lseek failed: Bad file descriptor
NVMAP_IOC_FREE failed: Bad file descriptor
terminate called after throwing an instance of 'vk::OutOfDeviceMemoryError'
what():  vk::Device::allocateMemoryUnique: ErrorOutOfDeviceMemory

I also tried to use vk::ExternalMemoryHandleTypeFlagBits::eDmaBufEXT for the handle type,
but physicalDevice.getExternalBufferProperties indicated that no valid usage was possible.

2# Create an exportable buffer in Vulkan and pass its file descriptor to V4L2

vk::ExportMemoryAllocateInfo exportMemInfo{};
exportMemInfo.handleTypes = vk::ExternalMemoryHandleTypeFlagBits::eOpaqueFd;

vk::MemoryAllocateInfo allocInfo = {};
allocInfo.pNext = &exportMemInfo;
allocInfo.allocationSize = memReqs.size;
allocInfo.memoryTypeIndex = Vulkan::findMemoryTypeIndex(memReqs, memProps, vk::MemoryPropertyFlagBits::eDeviceLocal);
auto deviceMemory = device.allocateMemoryUnique(allocInfo));

vk::MemoryGetFdInfoKHR fdInfo{};
fdInfo.memory = deviceMemory.get();
fdInfo.handleType = vk::ExternalMemoryHandleTypeFlagBits::eOpaqueFd;
dmabuff_fd = vulkanBackend.device().getMemoryFdKHR(fdInfo);
	
...
v4l2_buf.m.planes[j].m.fd = dmabuff_fd;
...
	
ret = ctx.enc->output_plane.qBuffer(v4l2_buf, NULL);

This fails with the following error message:

nvbuf_utils: dmabuf_fd 1140 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...

Could you give me any hint on how to manage this?

Thank you in advance.
Pierre.

Hi,
This use-case is not supported. You may try to use OpenGL/EGL functions like:
Trying to process with OpenGL an EGLImage created from a dmabuf_fd - #9 by DaneLLL

Hi,
Thanks for the answer.
Understood, I will go through OpenGL/EGL.
Would be nice to be doing this directly from Vulkan in the future though.
Best,

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.