Error compiling for opencv4.5.5 with cuda acceleration on JetsonNano (cudaVideoCodec_AV1 was not declared)

shenshen927 · January 14, 2022, 4:44am

I need to compile opencV with CUDA acceleration, using OpencV version 4.5.5, and compiled according to the following steps.

Install nV-codec-headers SDK-10.0 and make install.
Compile FFMPEG with nvcuvid.
Download and install the Video Codec SDK 10.0.26, and copy libnvcuvid.so and the associated header file to /usr/local/cuda/lib64 (/include).
Compile the opencv.

The opencV cmake compilation options used are as follows:

cmake -D WITH_CUDA=ON  \
      -D WITH_CUDNN=ON  \
      -D WITH_CUBLAS=ON  \
      -D CUDA_ARCH_BIN="5.3"  \
      -D CUDA_ARCH_PTX=""  \
      -D BUILD_CUDA_STUBS=ON  \
      -D ENABLE_FAST_MATH=ON  \
      -D CUDA_FAST_MATH=ON  \
      -D OPENCV_DNN_CUDA=ON  \
      -D OPENCV_ENABLE_NONFREE=ON  \
      -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.5.5/modules  \
      -D WITH_GSTREAMER=ON  \
      -D WITH_LIBV4L=ON  \
      -D WITH_NVCUVID=ON  \
      -D BUILD_opencv_cudacodec=ON  \
      -D WITH_FFMPEG=ON  \
      -D WITH_CUFFT=ON  \
      -D OPENCV_GENERATE_PKGCONFIG=ON  \
      -D BUILD_opencv_python2=OFF  \
      -D BUILD_opencv_python3=ON  \
      -D BUILD_TESTS=OFF  \
      -D BUILD_PERF_TESTS=OFF  \
      -D BUILD_EXAMPLES=OFF  \
      -D CMAKE_BUILD_TYPE=RELEASE  \
      -D CMAKE_INSTALL_PREFIX=/usr/local  \
      -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.2  \
      -D CUDA_nvcuvid_LIBRARY=/usr/local/cuda-10.2/lib64/libnvcuvid.so  \
      ../opencv-4.5.5

The following error occurred while making.

[ 55%] Building NVCC (Device) object modules/cudacodec/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nv12_to_rgb.cu.o
[ 55%] Building CXX object modules/highgui/CMakeFiles/opencv_highgui.dir/src/backend.cpp.o
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/cuvid_video_source.cpp.o
In file included from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core.hpp:53:0,
                 from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/cuda.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/include/opencv2/cudacodec.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/precomp.hpp:54,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/cuvid_video_source.cpp:44:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/cuvid_video_source.cpp: In constructor ‘cv::cudacodec::detail::CuvidVideoSource::CuvidVideoSource(const String&)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/cuvid_video_source.cpp:71:51: warning: comparison between ‘enum cv::cudacodec::Codec’ and ‘enum cudaVideoCodec_enum’ [-Wenum-compare]
     CV_Assert(Codec::NumCodecs == cudaVideoCodec::cudaVideoCodec_NumCodecs);
                                                   ^
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/ffmpeg_video_source.cpp.o
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp: In function ‘int StartCodeLen(unsigned char*, int)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp:116:5: warning: no previous declaration for ‘int StartCodeLen(unsigned char*, int)’ [-Wmissing-declarations]
 int StartCodeLen(unsigned char* data, const int sz) {
     ^~~~~~~~~~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp: In function ‘bool ParamSetsExist(unsigned char*, int, unsigned char*, int)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp:125:6: warning: no previous declaration for ‘bool ParamSetsExist(unsigned char*, int, unsigned char*, int)’ [-Wmissing-declarations]
 bool ParamSetsExist(unsigned char* parameterSets, const int szParameterSets, unsigned char* data, const int szData) {
      ^~~~~~~~~~~~~~
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/frame_queue.cpp.o
[ 55%] Building CXX object modules/highgui/CMakeFiles/opencv_highgui.dir/src/window.cpp.o
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/thread.cpp.o
[ 55%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/op_halide.cpp.o
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/video_decoder.cpp.o
[ 55%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/op_inf_engine.cpp.o
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp: In member function ‘void cv::cudacodec::detail::VideoDecoder::create(const cv::cudacodec::FormatInfo&)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:81:29: error: ‘cudaVideoCodec_AV1’ was not declared in this scope
                             cudaVideoCodec_AV1      == _codec ||
                             ^~~~~~~~~~~~~~~~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:81:29: note: suggested alternative: ‘cudaVideoCodec_NV12’
                             cudaVideoCodec_AV1      == _codec ||
                             ^~~~~~~~~~~~~~~~~~
                             cudaVideoCodec_NV12
In file included from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core.hpp:53:0,
                 from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/cuda.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/include/opencv2/cudacodec.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/precomp.hpp:54,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:44:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:106:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         videoFormat.ulWidth <= decodeCaps.nMaxWidth &&
         ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:107:30: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         videoFormat.ulHeight <= decodeCaps.nMaxHeight);
         ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:109:67: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     CV_Assert((videoFormat.width >> 4)* (videoFormat.height >> 4) <= decodeCaps.nMaxMBCount);
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/build.make:805: recipe for target 'modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/video_decoder.cpp.o' failed
make[2]: *** [modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/video_decoder.cpp.o] Error 1
CMakeFiles/Makefile2:4873: recipe for target 'modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/all' failed
make[1]: *** [modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

What is the reason for this? Please help me, thanks.

shenshen927 · January 16, 2022, 4:17am

I compiled opencv4.5.5 on jetson nano by following these steps.

Install nV-codec-headers SDK-10.0 and make install.
Compile FFMPEG with nvcuvid.
Download and install the Video Codec SDK 10.0.26, and copy libnvcuvid.so and the associated header file to /usr/local/cuda/lib64 (/include).
Compile the opencv and opencv_contrib.

The compilation environment and associated dependent libraries are shown below:
Jetpack 4.6 [L4T 32.6.1]
CUDA: 10.2.300
cuDNN: 8.2.1.32
nv-codec-headers-sdk: 10.0.26.2
Video_Codec_SDK: 10.0.26
ffmpeg: 4.4.1

I found this was probably a bug in opencv_contrib or in Video_Codec_SDK. There is no definition for cudaVideoCodec_AV1 in Opencv_contrib, nor in nV-codec-headers - SDK-10.0 or video_codec_sdk_10.0.26. CudaVideoCodec_AV1 is defined in Video_Codec_SDK_11. However, in NVIDIA’s official instructions, Video_Codec_SDK_11 relies on CUDA11, but CUDA10.2 is used In JetPack 4.6. This seems to be an insoluble contradiction at present.

For now, I’ve fixed the problem for the time being by comment out lines 81 in video_decoder.cpp and 132 through 137 in video_parser.cpp. But the solution is not so elegant, so what is the elegant solution?

The commented out code is shown below:

video_decorder.cpp

#if defined (HAVE_CUDA)
#if (CUDART_VERSION >= 6500)
    codecSupported |=       cudaVideoCodec_HEVC     == _codec;
#endif
#if  ((CUDART_VERSION == 7500) || (CUDART_VERSION >= 9000))
    codecSupported |=       cudaVideoCodec_VP8      == _codec ||
                            cudaVideoCodec_VP9      == _codec ||
                            /*cudaVideoCodec_AV1      == _codec ||*/
                            cudaVideoCodec_YUV420   == _codec;
#endif
#endif

vedeo_parser.cpp

    if (format->codec         != thiz->videoDecoder_->codec()       ||
        format->coded_width   != thiz->videoDecoder_->frameWidth()  ||
        format->coded_height  != thiz->videoDecoder_->frameHeight() ||
        format->chroma_format != thiz->videoDecoder_->chromaFormat()||
        format->bit_depth_luma_minus8 != thiz->videoDecoder_->nBitDepthMinus8() ||
        format->min_num_decode_surfaces != thiz->videoDecoder_->maxDecodeSurfaces())
    {
        FormatInfo newFormat;
        newFormat.codec = static_cast<Codec>(format->codec);
        newFormat.chromaFormat = static_cast<ChromaFormat>(format->chroma_format);
        newFormat.nBitDepthMinus8 = format->bit_depth_luma_minus8;
        newFormat.ulWidth = format->coded_width;
        newFormat.ulHeight = format->coded_height;
        newFormat.width = format->coded_width;
        newFormat.height = format->coded_height;
        newFormat.displayArea = Rect(Point(format->display_area.left, format->display_area.top), Point(format->display_area.right, format->display_area.bottom));
        newFormat.fps = format->frame_rate.numerator / static_cast<float>(format->frame_rate.denominator);
        newFormat.ulNumDecodeSurfaces = format->min_num_decode_surfaces;
        if (format->progressive_sequence)
            newFormat.deinterlaceMode = Weave;
        else
            newFormat.deinterlaceMode = Adaptive;
        int maxW = 0, maxH = 0;
        // AV1 has max width/height of sequence in sequence header
        /*if (format->codec == cudaVideoCodec_AV1 && format->seqhdr_data_length > 0)
        {
            CUVIDEOFORMATEX* vidFormatEx = (CUVIDEOFORMATEX*)format;
            maxW = vidFormatEx->av1.max_width;
            maxH = vidFormatEx->av1.max_height;
        }*/
        if (maxW < (int)format->coded_width)
            maxW = format->coded_width;
        if (maxH < (int)format->coded_height)
            maxH = format->coded_height;
        newFormat.ulMaxWidth = maxW;
        newFormat.ulMaxHeight = maxH;
        thiz->frameQueue_->init(newFormat.ulNumDecodeSurfaces);

DaneLLL · January 17, 2022, 2:01am

Hi,
Video codec sdk is specific to desktop GPUs. It is not supported on Jetson platforms. Please not to enable the related configs and try again.

May try the scripts:
JEP/install_opencv4.5.0_Jetson.sh at master · AastaNV/JEP · GitHub
GitHub - mdegans/nano_build_opencv: Build OpenCV on Nvidia Jetson Nano

shenshen927 · January 17, 2022, 2:09am

But how do I compile opencv if I want to do hardware-based codec on jetson nano ?

shenshen927 · January 17, 2022, 2:13am

Does the .sh script in the link you sent support hardware codec ?

DaneLLL · January 17, 2022, 5:31am

Hi,
Hardware decoding/encoding is not enabled in OpenCV. For using hardware codec, woudl need to run like the samples:
Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL

To run gstreamer command in cv2.VideoCapture() or cv2.VideoWriter(),

Or may try this solution:
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL
To run a gstreamer command and map the buffer to cv::gpu::gpuMat.

system · February 9, 2022, 2:59am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Install opencv with cuda Jetson Nano opencv , cudnn	4	72	July 24, 2024
How to install opencv-python for python3.6 Jetson Nano opencv	26	19178	October 14, 2021
Build OpenCV 4.6.4 with Cuda on Jetson Nano Jetson Nano opencv , cuda	13	3254	September 18, 2023
Build OpenCV 3.4 with CUDA on NVIDIA Jetson TX2 Jetson TX2 opencv	14	9478	October 18, 2021
OpenCV build script Jetson Nano opencv	59	20231	January 8, 2024
Unable to install opencv with CUDA in Jetson Nano Jetson Nano	30	13554	October 18, 2021
OpenCV 3.1 with USB camera support Jetson TX1 opencv	9	9334	July 16, 2018
gstreamer NVMM <-> opencv gpuMat Jetson TX2 opencv	58	21564	October 18, 2021
opencv and Cuda Jetson TX1 opencv	5	2807	October 18, 2021
Cmake - OpenCV compilation Error ( implemented on Jetson Orin) Jetson Orin NX opencv , cuda	2	1747	October 31, 2022

Error compiling for opencv4.5.5 with cuda acceleration on JetsonNano (cudaVideoCodec_AV1 was not declared)

Related topics