Hello everyone,
We are developing VR & AR applications to operate our robots. In our framework, we pre-process our real-time image streams in the GPU memory using Nvidia Cuda and some existing Nvidia SDKs. To augment our real-time video feeds together with UE objects in the scene, we developed a direct GPU copy of video feed on 2D textures rendered in UE in Windows using D3D11. (GitHub - leggedrobotics/cuda_ue4_linux
The feature/windows branch is the working version for Windows platforms. If you request, I can also share the sample raw video dataset with you as well.)
Similarly, I would like to access the captured game scene in the GPU through a camera actor. Then I could pass it to perform further processing using CUDA & Nvidia SDKs. Such as a scene capture application into GPU memory instead of file copy: https://github.com/TimmHess/UnrealImageCapture
However, we would like to achieve this direct GPU copy operation on Linux platforms using VulkanRHI as well. Unfortunately, I couldn’t find a working example for this type of operation.
It would be great if it is possible to solve this challenge together with the Nvidia experts and its community.
Best,
Burak