How to pass a numba cuda DeviceNDArray or GPUArray to VideoCapture render method in jetson_utils to display streaming video

Hi,

I am currently using jetson_utils.VideoSource Capture method to read frames from an mp4 file, perform some operations on those frames using vpi, PyCUDA and/or Numba Cuda, and ultimately pass those frames to jetson_utils.VideoOutput Render method to display the output video.
However, I am unable to pass data in the formats of vpi.Image, DeviceNDArray and/or GPUArray formats to the Render method of VideoOutput. The Render method is expecting data in the format cudaImage. How do I convert the the aforementioned data formats to cudaImage format for compatibility with Render method of VideoOutput?
I believe they all have the cuda_array_interface property. But I am trying to figure out a way to use it for interoperability between libraries.

Thanks

Hi @surajgampa95, cudaImage also implements __cuda_array_interface__ so you can pass cudaImage’s to those libraries like shown here:

https://github.com/dusty-nv/jetson-utils/blob/b997c37cbf50209d84fc32c130da6406228e97c0/python/examples/cuda-array-interface.py

However I believe you are correct that the reverse direction isn’t explicitly implemented - I will add this to my todo list to investigate. Could you use a cudaImage as the output of PyCUDA/Numba/ect, so that those results are already stored in the cudaImage?

@surajgampa95 there is now an example of mapping externally-allocated GPU memory into a cudaImage (without memory copies) here:

There was also documentation added on the support for __cuda_array_interface__ in cudaImage here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-image.md#cuda-array-interface

Hi @dusty_nv, thank you for the response. However, I have tried the following code -

        try:
            while True:
                frame_left = __cap_front_stream__.Capture(format='rgb8')
                if frame_left is None:
                    continue

                with vpi.Backend.CUDA:
                    distortion_corrected_left = vpi.asimage(frame_left)\
                                                .convert(vpi.Format.NV12_ER)\
                                                .remap(warpmap_distortion_correction, interp=vpi.Interp.LINEAR)\
                                                .convert(vpi.Format.RGB8)
                
                with distortion_corrected_left.rlock_cuda() as cudaBuffer:
                    device_pointer = cudaBuffer.__cuda_array_interface__['data'][0]
                distortion_corrected_left_cudaImg = cudaImage(ptr = device_pointer, width=1280, height=720, format='rgb8')
                __output_stream__.Render(distortion_corrected_left_cudaImg)

                if not __cap_front_stream__.IsStreaming() or not __output_stream__.IsStreaming():
                    break
        except Exception as exception:
            print(str(exception))

And the error I get is -

'ptr' is an invalid keyword argument for this function

Do I need to update my jetson_utils library to your latest commit? Because I am of the assumption that you just added an example to your latest commit. Let me know otherwise.

Thanks

Yes, do a git pull --recurse-submodules from your jetson-inference directory and rebuild/reinstall:

cd jetson-inference
git pull --recurse-submodules
cd build
cmake ../
make
sudo make install

Let me know if that works for you.

@dusty_nv thank you for the reply. It works now.

Thanks again

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.