Hi,
I am trying to read an mp4 file in Python using OpenCV CUDA and GStreamer, and use the captured frame in a VPI pipeline. Is there a way to achieve zero mem copy in the process of capturing the frame and sending it to the VPI pipeline?
Here is my code -
cap_front = cv2.VideoCapture('filesrc location=front_video.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! queue ! videoconvert ! queue ! video/x-raw, format=BGR ! appsink ', cv2.CAP_GSTREAMER)
out = cv2.VideoWriter('appsrc ! queue ! videoconvert ! video/x-raw,format=RGBA ! nvvidconv ! nvegltransform ! nveglglessink', cv2.CAP_GSTREAMER, 0, 30.0, (1280,720))
while True:
ret_front, frame_front = cap_front.read()
with vpi.Backend.CUDA:
distortion_corrected_front = vpi.asimage(frame_front)\
.convert(vpi.Format.NV12_ER)\
.remap(warpmap_distortion_correction, interp=vpi.Interp.LINEAR)\
.convert(vpi.Format.RGB8)
out.write(distortion_corrected_front.cpu())
I wanted to know if I can copy frame_front
directly into a VPI buffer or other device memory to achieve zero mem copy. Also is there a way to avoid copying distortion_corrected_front
into host for display.
Thanks
Hi,
VPI can wrap images from a preallocated buffer.
If you can read an image to a GPU-accessible buffer, you can create a VPI image with the below API:
https://docs.nvidia.com/vpi/group__VPI__Image.html#ga3e7cf2520dd568a7e7a9a6876ea7995c
Thanks.
Thanks for the reply @AastaLLL
I actually found another way of doing it using jetson_utils library. I am using the following code -
from jetson_utils import videoSource, videoOutput
import pycuda.driver as drv
import pycuda.autoprimaryctx
from pycuda.compiler import SourceModule
from extract_system_calibration import EXTRACT_SYSTEM_CALIBRATION
from exception_message import ExceptionMessage
import numpy
import cv2
import vpi
if __name__ == '__main__':
param_videoOutput = []
param_videoOutput.append("--width=" + str(1280))
param_videoOutput.append(f"--height=" + str(720))
input = videoSource('file://front_video.mp4')
output = videoOutput('display://0', argv=param_videoOutput)
while True:
image = input.Capture(format='rgb8')
if image is None:
continue
with vpi.Backend.CUDA:
distortion_corrected_front = vpi.asimage(image)\
.convert(vpi.Format.NV12_ER)\
.remap(warpmap_distortion_correction, interp=vpi.Interp.LINEAR)\
.convert(vpi.Format.RGB8)
output.Render(distortion_corrected_front)
if not input.IsStreaming() or not output.IsStreaming():
break
And the error I get is -
Traceback (most recent call last):
File "test_script2.py", line 189, in <module>
output.Render(distortion_corrected_front)
Exception: jetson.utils -- videoOutput.Render() failed to get image pointer from first arg (should be cudaImage)
[gstreamer] gstreamer mysink taglist, video-codec=(string)"H.264\ \(Main\ Profile\)", bitrate=(uint)4686376, minimum-bitrate=(uint)3471360, maximum-bitrate=(uint)3540480;
[gstreamer] gstDecoder -- stopping pipeline, transitioning to GST_STATE_NULL
[gstreamer] gstDecoder -- onPreroll()
Is there any conversion I need to perform on the VPI image to pass it to the output streaming pipeline?
Thanks
Hi,
Is this topic duplicated to 249136?
Is Python essential for you?
With the C++ interface, it will be easier to get the GPU buffer pointer.
Thanks.
Hi @AastaLLL ,
Thank you for the reply. I was able to resolve it in Python by using jetson_utils library available at https://github.com/dusty-nv/jetson-utils . In jetson_utils, I used VideoSource and VideoOutput to capture and render video frames. I was able to pass the output of VideoSource capture, which is in cudaImage format, to the vpi pipeline and achieve zero mem copy.
Suraj
system
Closed
May 26, 2023, 4:48pm
8
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.