FFMPEG (LibAV) using AV_PIX_FMT_D3D11 gives "Error registering the input resource" from NVENC

Gaming and Visualization Technologies General Topics and Other SDKs
, ,
1

I am capturing the screen using DXGI and obtaining a ID3D11Texture2D if I download that to CPU and use av_hwframe_transfer_data, the encoder works fine.

But I am trying to remove that step of downloading the texture to CPU and trying to pass it directly, trying to send the GPU memory pointe directly into the encoder. But when I try to do that I am getting the following error:

[h264_nvenc @ 00000191233e1bc0] Error registering an input resource: invalid call (9):
[h264_nvenc @ 00000191233e1bc0] Could not register an input HW frame
Error sending a frame to the encoder: Unknown error occurred

I am basically doing same that this user of stackoverflow → c++ - FFMPEG using AV_PIX_FMT_D3D11 gives “Error registering the input resource” from NVENC - Stack Overflow with similar results…

And I am using latest version of ffmpeg at the minute, v6.0.

Any tips on how to proceed with this ?

2

Hello @forlayo, welcome to the NVIDIA developer forums!

I am sorry that there was no reply so far. I tried to reach out and see if I can find someone internal who could help, might still take a while.

I am not certain that what you have in mind would actually work with DX at all. I am not the expert here, but my impression is that you would need to go through Vulkan and/or CUDA to achieve this.