Error compiling for opencv4.5.5 with cuda acceleration on JetsonNano (cudaVideoCodec_AV1 was not declared)

I need to compile opencV with CUDA acceleration, using OpencV version 4.5.5, and compiled according to the following steps.

  1. Install nV-codec-headers SDK-10.0 and make install.
  2. Compile FFMPEG with nvcuvid.
  3. Download and install the Video Codec SDK 10.0.26, and copy libnvcuvid.so and the associated header file to /usr/local/cuda/lib64 (/include).
  4. Compile the opencv.

The opencV cmake compilation options used are as follows:

cmake -D WITH_CUDA=ON  \
      -D WITH_CUDNN=ON  \
      -D WITH_CUBLAS=ON  \
      -D CUDA_ARCH_BIN="5.3"  \
      -D CUDA_ARCH_PTX=""  \
      -D BUILD_CUDA_STUBS=ON  \
      -D ENABLE_FAST_MATH=ON  \
      -D CUDA_FAST_MATH=ON  \
      -D OPENCV_DNN_CUDA=ON  \
      -D OPENCV_ENABLE_NONFREE=ON  \
      -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.5.5/modules  \
      -D WITH_GSTREAMER=ON  \
      -D WITH_LIBV4L=ON  \
      -D WITH_NVCUVID=ON  \
      -D BUILD_opencv_cudacodec=ON  \
      -D WITH_FFMPEG=ON  \
      -D WITH_CUFFT=ON  \
      -D OPENCV_GENERATE_PKGCONFIG=ON  \
      -D BUILD_opencv_python2=OFF  \
      -D BUILD_opencv_python3=ON  \
      -D BUILD_TESTS=OFF  \
      -D BUILD_PERF_TESTS=OFF  \
      -D BUILD_EXAMPLES=OFF  \
      -D CMAKE_BUILD_TYPE=RELEASE  \
      -D CMAKE_INSTALL_PREFIX=/usr/local  \
      -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.2  \
      -D CUDA_nvcuvid_LIBRARY=/usr/local/cuda-10.2/lib64/libnvcuvid.so  \
      ../opencv-4.5.5

The following error occurred while making.

[ 55%] Building NVCC (Device) object modules/cudacodec/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nv12_to_rgb.cu.o
[ 55%] Building CXX object modules/highgui/CMakeFiles/opencv_highgui.dir/src/backend.cpp.o
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/cuvid_video_source.cpp.o
In file included from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core.hpp:53:0,
                 from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/cuda.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/include/opencv2/cudacodec.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/precomp.hpp:54,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/cuvid_video_source.cpp:44:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/cuvid_video_source.cpp: In constructor ‘cv::cudacodec::detail::CuvidVideoSource::CuvidVideoSource(const String&)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/cuvid_video_source.cpp:71:51: warning: comparison between ‘enum cv::cudacodec::Codec’ and ‘enum cudaVideoCodec_enum’ [-Wenum-compare]
     CV_Assert(Codec::NumCodecs == cudaVideoCodec::cudaVideoCodec_NumCodecs);
                                                   ^
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/ffmpeg_video_source.cpp.o
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp: In function ‘int StartCodeLen(unsigned char*, int)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp:116:5: warning: no previous declaration for ‘int StartCodeLen(unsigned char*, int)’ [-Wmissing-declarations]
 int StartCodeLen(unsigned char* data, const int sz) {
     ^~~~~~~~~~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp: In function ‘bool ParamSetsExist(unsigned char*, int, unsigned char*, int)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/ffmpeg_video_source.cpp:125:6: warning: no previous declaration for ‘bool ParamSetsExist(unsigned char*, int, unsigned char*, int)’ [-Wmissing-declarations]
 bool ParamSetsExist(unsigned char* parameterSets, const int szParameterSets, unsigned char* data, const int szData) {
      ^~~~~~~~~~~~~~
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/frame_queue.cpp.o
[ 55%] Building CXX object modules/highgui/CMakeFiles/opencv_highgui.dir/src/window.cpp.o
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/thread.cpp.o
[ 55%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/op_halide.cpp.o
[ 55%] Building CXX object modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/video_decoder.cpp.o
[ 55%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/op_inf_engine.cpp.o
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp: In member function ‘void cv::cudacodec::detail::VideoDecoder::create(const cv::cudacodec::FormatInfo&)’:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:81:29: error: ‘cudaVideoCodec_AV1’ was not declared in this scope
                             cudaVideoCodec_AV1      == _codec ||
                             ^~~~~~~~~~~~~~~~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:81:29: note: suggested alternative: ‘cudaVideoCodec_NV12’
                             cudaVideoCodec_AV1      == _codec ||
                             ^~~~~~~~~~~~~~~~~~
                             cudaVideoCodec_NV12
In file included from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core.hpp:53:0,
                 from /opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/cuda.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/include/opencv2/cudacodec.hpp:51,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/precomp.hpp:54,
                 from /opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:44:
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:106:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         videoFormat.ulWidth <= decodeCaps.nMaxWidth &&
         ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:107:30: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         videoFormat.ulHeight <= decodeCaps.nMaxHeight);
         ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
/opt/opencv/opencv_contrib-4.5.5/modules/cudacodec/src/video_decoder.cpp:109:67: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     CV_Assert((videoFormat.width >> 4)* (videoFormat.height >> 4) <= decodeCaps.nMaxMBCount);
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
/opt/opencv/opencv-4.5.5/modules/core/include/opencv2/core/base.hpp:342:38: note: in definition of macro ‘CV_Assert’
 #define CV_Assert( expr ) do { if(!!(expr)) ; else cv::error( cv::Error::StsAssert, #expr, CV_Func, __FILE__, __LINE__ ); } while(0)
                                      ^~~~
modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/build.make:805: recipe for target 'modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/video_decoder.cpp.o' failed
make[2]: *** [modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/src/video_decoder.cpp.o] Error 1
CMakeFiles/Makefile2:4873: recipe for target 'modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/all' failed
make[1]: *** [modules/cudacodec/CMakeFiles/opencv_cudacodec.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

What is the reason for this? Please help me, thanks.

I compiled opencv4.5.5 on jetson nano by following these steps.

  1. Install nV-codec-headers SDK-10.0 and make install.
  2. Compile FFMPEG with nvcuvid.
  3. Download and install the Video Codec SDK 10.0.26, and copy libnvcuvid.so and the associated header file to /usr/local/cuda/lib64 (/include).
  4. Compile the opencv and opencv_contrib.

The compilation environment and associated dependent libraries are shown below:
Jetpack 4.6 [L4T 32.6.1]
CUDA: 10.2.300
cuDNN: 8.2.1.32
nv-codec-headers-sdk: 10.0.26.2
Video_Codec_SDK: 10.0.26
ffmpeg: 4.4.1

I found this was probably a bug in opencv_contrib or in Video_Codec_SDK. There is no definition for cudaVideoCodec_AV1 in Opencv_contrib, nor in nV-codec-headers - SDK-10.0 or video_codec_sdk_10.0.26. CudaVideoCodec_AV1 is defined in Video_Codec_SDK_11. However, in NVIDIA’s official instructions, Video_Codec_SDK_11 relies on CUDA11, but CUDA10.2 is used In JetPack 4.6. This seems to be an insoluble contradiction at present.

For now, I’ve fixed the problem for the time being by comment out lines 81 in video_decoder.cpp and 132 through 137 in video_parser.cpp. But the solution is not so elegant, so what is the elegant solution?

The commented out code is shown below:

video_decorder.cpp

#if defined (HAVE_CUDA)
#if (CUDART_VERSION >= 6500)
    codecSupported |=       cudaVideoCodec_HEVC     == _codec;
#endif
#if  ((CUDART_VERSION == 7500) || (CUDART_VERSION >= 9000))
    codecSupported |=       cudaVideoCodec_VP8      == _codec ||
                            cudaVideoCodec_VP9      == _codec ||
                            /*cudaVideoCodec_AV1      == _codec ||*/
                            cudaVideoCodec_YUV420   == _codec;
#endif
#endif
vedeo_parser.cpp

    if (format->codec         != thiz->videoDecoder_->codec()       ||
        format->coded_width   != thiz->videoDecoder_->frameWidth()  ||
        format->coded_height  != thiz->videoDecoder_->frameHeight() ||
        format->chroma_format != thiz->videoDecoder_->chromaFormat()||
        format->bit_depth_luma_minus8 != thiz->videoDecoder_->nBitDepthMinus8() ||
        format->min_num_decode_surfaces != thiz->videoDecoder_->maxDecodeSurfaces())
    {
        FormatInfo newFormat;
        newFormat.codec = static_cast<Codec>(format->codec);
        newFormat.chromaFormat = static_cast<ChromaFormat>(format->chroma_format);
        newFormat.nBitDepthMinus8 = format->bit_depth_luma_minus8;
        newFormat.ulWidth = format->coded_width;
        newFormat.ulHeight = format->coded_height;
        newFormat.width = format->coded_width;
        newFormat.height = format->coded_height;
        newFormat.displayArea = Rect(Point(format->display_area.left, format->display_area.top), Point(format->display_area.right, format->display_area.bottom));
        newFormat.fps = format->frame_rate.numerator / static_cast<float>(format->frame_rate.denominator);
        newFormat.ulNumDecodeSurfaces = format->min_num_decode_surfaces;
        if (format->progressive_sequence)
            newFormat.deinterlaceMode = Weave;
        else
            newFormat.deinterlaceMode = Adaptive;
        int maxW = 0, maxH = 0;
        // AV1 has max width/height of sequence in sequence header
        /*if (format->codec == cudaVideoCodec_AV1 && format->seqhdr_data_length > 0)
        {
            CUVIDEOFORMATEX* vidFormatEx = (CUVIDEOFORMATEX*)format;
            maxW = vidFormatEx->av1.max_width;
            maxH = vidFormatEx->av1.max_height;
        }*/
        if (maxW < (int)format->coded_width)
            maxW = format->coded_width;
        if (maxH < (int)format->coded_height)
            maxH = format->coded_height;
        newFormat.ulMaxWidth = maxW;
        newFormat.ulMaxHeight = maxH;
        thiz->frameQueue_->init(newFormat.ulNumDecodeSurfaces);

Hi,
Video codec sdk is specific to desktop GPUs. It is not supported on Jetson platforms. Please not to enable the related configs and try again.

May try the scripts:
JEP/install_opencv4.5.0_Jetson.sh at master · AastaNV/JEP · GitHub
GitHub - mdegans/nano_build_opencv: Build OpenCV on Nvidia Jetson Nano

But how do I compile opencv if I want to do hardware-based codec on jetson nano ?

Does the .sh script in the link you sent support hardware codec ?

Hi,
Hardware decoding/encoding is not enabled in OpenCV. For using hardware codec, woudl need to run like the samples:
Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL

To run gstreamer command in cv2.VideoCapture() or cv2.VideoWriter(),

Or may try this solution:
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL
To run a gstreamer command and map the buffer to cv::gpu::gpuMat.