VPI library with Jetsons?

[easybob]

Hello,

i try to use VPI with the Jetson AGX Orin without success. I would like to bring back VPI routine result in a numpy array and things goes wrong.

I asked for things here but it seems this topic section is a kind of ghost section :

VPI library : request for new functions

Hello, i have started to use VPI with Jetson AGX Orin. This library is really cool and performs great. For information, my work : I really hope you will add new functions in VPI. Here are my requests : Color conversion : add the possibility to debayer bayer image (4 main bayer patterns at least) Color image channel split and merge like Python cv2.split() and cv2.merge(). For now, VPI channel mixer returns 3 color images (R, G, B). It would be better to get R, G, B mono channels It would…

I still don’t understand why VPI has been related to PIL library which is the slowest library we can found for image processing. And no link with numpy ? Can’t understand.

Some clear help would be appreciate.

Alain

[AastaLLL]

Hi,

VPI uses PIL for reading/writing image and also use NumPy for its CPU buffer.

You can get the NumPy output like the below:
Ex. 01-convolve_2d

...
# Using the chosen backend,
with backend:
    # Run input through the convolution filter
    output = input.convolution(kernel, border=vpi.Border.ZERO)

numpy_output = output.cpu()
print(type(numpy_output))

<class ‘numpy.ndarray’>

Thanks.

[easybob]

Hello AsstaLLL,

unfortunately, this do not work. My code :

    backend = vpi.Backend.CUDA
    image_vpi = vpi.asimage(image_mono)
    image_vpi = image_vpi.convert(vpi.Format.U8, backend=vpi.Backend.CUDA)
    with backend:
        gaussian = image_vpi.gaussian_filter(niveau_blur, 1.7, border=vpi.Border.ZERO)
    image_gaussian_blur_mono = gaussian.cpu()

I tried many things but i always get the same error message :
image_gaussian_blur_mono = gaussian.cpu()
RuntimeError: VPI_ERROR_INTERNAL: Can’t perform shared mapping

I must say that i use PyCuda in my program and maybe this is a problem. When i use Cupy for eaxample, there is a problem with PyCuda. Cupy alone is OK but Cupy with PyCuda do not work.

Any suggestion ?

Alain

[easybob]

In my software (JetsonSky), i use PyCuda, OpenCV and Numpy (mainly).

I would like to use Cupy (instead of numpy) and VPI (instead of some OpenCV functions).

I made some tests with Cupy. as i said previously, Cupy alone is OK. Cupy with Pycuda generate an error with PyCuda.

Maybe it is the same thing with VPI + PyCuda.

Concerning initialization of PyCuda, i do nothing special except pycuda.autoinit. This autoinit fix the GPU context.

When i use Cupy, it seems PyCuda has lost its context.

Do i need to set a different context for Cupy in order to preserve Pycuda context ? If yes, how can i do that ?

Is VPI needs the same thing ?

I am not sure i am very clear in my explanations.

Alain

[AastaLLL]

Hi,

A possible reason is that the image is originally stored in a GPU buffer.
VPI doesn’t map it to the CPU automatically for performance and causes the error.
(But VPI can do this if the corresponding flag is set.)

Do you have a complete source that could reproduce this issue?
We want to check it further to see how to fix this.

Thanks.

[easybob]

Hello,

I made a small test program :

import numpy as np
import cv2
import vpi

Image_Test = ‘/home/alain/Work/Python/Pytorch/Images/4K_base.tif’
Save_image_vpi = ‘/home/alain/Work/Python/Pytorch/Images/4K_result_VPI.jpg’

img_OpenCV = cv2.imread(Image_Test,cv2.IMREAD_COLOR)
img_OpenCV_mono = cv2.cvtColor(img_OpenCV,cv2.COLOR_BGR2GRAY)

niveau_blur = 11
backend = vpi.Backend.CUDA

with backend:
image_vpi = vpi.asimage(img_OpenCV_mono)
gaussian = image_vpi.gaussian_filter(niveau_blur, 1.7, border=vpi.Border.ZERO)

image_gaussian_blur_mono = gaussian.cpu()

cv2.imwrite(Save_image_vpi, image_gaussian_blur_mono, [int(cv2.IMWRITE_JPEG_QUALITY), 95])

In that case (i only import VPI library), the test is OK.

I made an other test : this time, i also import Pycuda to be more close than my software JetsonSky.

import numpy as np
import cv2
import vpi
import pycuda.driver as drv
import pycuda.autoinit
from pycuda.compiler import SourceModule
import pycuda.cumath

Image_Test = ‘/home/alain/Work/Python/Pytorch/Images/4K_base.tif’
Save_image_vpi = ‘/home/alain/Work/Python/Pytorch/Images/4K_result_VPI.jpg’

img_OpenCV = cv2.imread(Image_Test,cv2.IMREAD_COLOR)
img_OpenCV_mono = cv2.cvtColor(img_OpenCV,cv2.COLOR_BGR2GRAY)

niveau_blur = 11

backend = vpi.Backend.CUDA

with backend:
image_vpi = vpi.asimage(img_OpenCV_mono)
gaussian = image_vpi.gaussian_filter(niveau_blur, 1.7, border=vpi.Border.ZERO)

image_gaussian_blur_mono = gaussian.cpu()

cv2.imwrite(Save_image_vpi, image_gaussian_blur_mono, [int(cv2.IMWRITE_JPEG_QUALITY), 95])

In that case, i have this error :

Traceback (most recent call last):
File “/usr/lib/python3.8/idlelib/run.py”, line 559, in runcode
exec(code, self.locals)
File “/home/alain/Work/Python/Pytorch/testVPI.py”, line 23, in
image_gaussian_blur_mono = gaussian.cpu()
RuntimeError: VPI_ERROR_INTERNAL: (cudaErrorDeviceUninitialized)

In my software JetsonSky, here is my code for GaussianBlur_Mono (it’s a function) :

def gaussianblur_mono(image_mono,niveau_blur):
global compute_library

image_gaussian_blur_mono = image_mono
if compute_library == "opencv" :
    image_gaussian_blur_mono = cv2.GaussianBlur(image_mono,(niveau_blur,niveau_blur),cv2.BORDER_DEFAULT)
    
if compute_library == "vpi" :
    if niveau_blur > 11 :
        niveau_blur = 11
    backend = vpi.Backend.CUDA
    with backend:
        image_vpi = vpi.asimage(image_mono)
        gaussian = image_vpi.gaussian_filter(niveau_blur, 1.7, border=vpi.Border.ZERO)
    image_gaussian_blur_mono = gaussian.cpu()

return image_gaussian_blur_mono

When i call this function, i get this error :

Exception in Tkinter callback
Traceback (most recent call last):
File “/usr/lib/python3.8/tkinter/init.py”, line 1892, in call
return self.func(*args)
File “/usr/lib/python3.8/tkinter/init.py”, line 814, in callit
func(*args)
File “/home/alain/Work/Python/JetsonSky/JetsonSky_V16_03Base.py”, line 2270, in refresh
application_filtrage()
File “/home/alain/Work/Python/JetsonSky/JetsonSky_V16_03Base.py”, line 3299, in application_filtrage
fd_b=gaussianblur_mono(fd_b,niveau_blur)
File “/home/alain/Work/Python/JetsonSky/JetsonSky_V16_03Base.py”, line 2540, in gaussianblur_mono
image_gaussian_blur_mono = gaussian.cpu()
RuntimeError: VPI_ERROR_INTERNAL: Can’t perform shared mapping

It seems VPI doesn’t like Pycuda very much.

It would be really useful if we could use VPI with Pycuda (same thing with Cupy and Pycuda).

Alain

[Honey_Patouceul]

Not sure if this is possible for your software, but for your example it works when changing:

import pycuda.autoinit

into

import pycuda.autoprimaryctx

[easybob]

Hello Honey_Patouceul,

well, i tried import pycuda.autoprimaryctx and … it seems it works ! No error message any more when i use Pycuda and Cupy toghether ! Tada !

I will test this with Pycuda and VPI to see if this solve the issue.

If it is true that Pycuda can work with Cupy and VPI, this will allow better GPU use and faster treatments. This is great especially for high frame rate with planetary treatments.

Well, you are the man of the year ! Nvidia could give you an AGX Orin to thanks you !

I will tell you if everything is ok.

Many thanks man !

Alain

[easybob]

Hello,

i tested import pycuda.autoprimaryctx with VPI and it works !

It seems my problem is solved.

Thanks for all.

Alain