CUDA disabled on Orin after restart

[christina.malliou]

Hi! I’ve started working with a Jetson AGX Orin and I’ve noticed that CUDA was not enabled. I’ve built OpenCV 4.7 from source in virtual environment with the following parameters for the cmake to make sure cuda was enabled and the python version for build would be python 3.

cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=ON \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUBLAS=1 \
-D WITH_CUDA=ON \
-D BUILD_opencv_cudacodec=OFF \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D WITH_V4L=ON \
-D WITH_QT=OFF \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_PC_FILE_NAME=opencv.pc \
-D OPENCV_ENABLE_NONFREE=ON \
-D OPENCV_PYTHON3_INSTALL_PATH=~/lib/python3.8/site-packages \
-D PYTHON_INCLUDE_DIR=/usr/include/python3.8 \
-D OPENCV_EXTRA_MODULES_PATH=/home/seems/opencv_contrib/modules \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D WITH_LIBV4L=ON \
-D CUDNN_INCLUDE_DIR=/usr/include \
-D BUILD_opencv_python3=ON \
-D HAVE_opencv_python3=ON \
-D BUILD_opencv_python2=OFF \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D PYTHON3_EXECUTABLE=~/.virtualenvs/opencv_dnn_cuda/bin/python3 \
-D PYTHON_DEFAULT_EXECUTABLE=~/.virtualenvs/opencv_dnn_cuda/bin/python3 \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_EXAMPLES=OFF ..

At first, CUDA was enabled and the when I was printing in the virtual environment the number of CUDA enabled devices it returned 1.

After some days, I’ve noticed that although I was able to access CUDA, a yolov4 model that I was trying to run had low performance and using jtop from the terminal I saw that the GPU was not reaching more than 15% when running the model. When printing the CUDA enabled device count, it started return 0. The info on jtop shows the correct openCV version (4.7.0) with CUDA: YES

Is there something that could have caused that? Apart from rebuilding the OpenCV from source is there any other way to solve this issue?

Thanks!

[AastaLLL]

Hi,

Could you try the CUDA device query sample to check the GPU status in the virtual environment?
This can help us to figure out whether the issue comes from GPU or OpenCV library.

$ git clone --depth 1 --branch v11.4 https://github.com/NVIDIA/cuda-samples.git
$ cd cuda-samples/Samples/deviceQuery/
$ make
$ ./deviceQuery 

Thanks.

[christina.malliou]

Thanks for the reply!

I found from another thread, a script for Xavier to install OpenCV that seems to be working.

The result from the device query is the attached.

However, another issue has been raised. OpenCV doesn’t appear on the list of packages even though I seem to be able to import it.

[AastaLLL]

Hi,

It looks like you are using a virtual environment.
Is the OpenCV installed within the env or outside of the env?

Thanks.

[christina.malliou]

Hi! OpenCV was installed both in the virtual environment and the base environment.
It seems to work, but I can’t seem to find the OpenCV in any of the environments, while at the same time the info at the jtop show OpenCV 4.6 with CUDA…

[AastaLLL]

Hi,

For the OpenCV that is installed on the base environment.
Do you rebuild it with CUDA enabled? If yes, does it work normally?

In the last figure you shared, it looks like you are using OpenCV 4.6.
But the version built with CUDA support should be version 4.7?

Guess that there are multiple OpenCV exist in your virtual environment which causes the confusion.
OpenCV 3.6 might be installed automatically when any package is dependent on OpenCV.

Thanks.