I couldn’t find anything in the VPI docs or from searching the forums.
VPI claims that other backends like PVA or VIC can be utilized for some image processing and CV algorithms which leaves the GPU free to do something else like neural net inference.
I’d like to be able to run TRT inference with an engine file on a VPI image that’s been transferred to the GPU. I want to keep it as a VPI image so that I can use the PVA backend in conjunction with inference on the GPU. Is this possible? Are there any examples? Since the VPI image is in CUDA memory, can I make an inference call like it’s stated here: Developer Guide :: NVIDIA Deep Learning TensorRT Documentation
Thanks for sharing. From my understanding, using VPI would only eliminate the copying of images from CPU to GPU and then copying results back to CPU. So I wouldn’t have to move data using cudaMemcpyAsync()?
I’d be able to directly run inference using context->enqueueV3(stream); I’d still also need to setup the execution context and deserialize the TRT engine file since that’s separate from VPI. Is that correct?
My question is when might I need to use the vpiImageCreateCUDAMemWrapper() ? Would it function the same as vpiImageCreateWrapperOpenCVMat() after I move the image to the GPU? Would it be better to wrap it into CUDAMem to make it better for TRT inference?