Is it currently possible to transfer vulkan images to NVEnc? If so, how would you go about doing that? The latest Video Codec SDK (8.2.16) have samples for OpenGL, Direct3D and Cuda, but no sample for Vulkan.
You need to use the cuda 10 external resource interop functions to import your Vulkan data. You can definitely import a VkImage, which gets mapped to a cuda mipmapped array. However, it’s unclear if nvenc supports using cuda arrays as an input resource type. There is an array resource type but the documentation for registering resources doesn’t list it as supported. It’s also unclear how you’d use it, because nvenc only accepts one pointer/handle for an input resource and a multiplane format will be represented by multiple VkImages (and if you use a multi-plane VkImage format, it’s unclear whether that can even be imported by cuda as a mipmapped array cannot express it.
The punchline is you probably have to do a copy, either copying to a VkBuffer on the vulkan side and importing that, or doing a cuda memcpy on the cuda side after importing. It’s unclear to me which is easier.
There are no official examples for this kind of usage but you can look at my work-in-process efforts to do decoding to vulkan. It’s going in the other direction but the basic principles are the same.
https://github.com/philipl/mpv/tree/libplacebo (copying to a buffer and then copying to image)
https://github.com/philipl/mpv/tree/libplacebo-vkmimage (copying to the image directly)
You also would need to look at libplacebo itself to see all the subtleties around managing the state of the images and ensuring all the required queue, layout, and access transitions take place.
Thank you so much! Very helpful. I don’t think doing a copy is any problem as long as I don’t have to go through host memory.