Impossible to use NvJPEGDecoder and OpenCV in the same program?

[whatnot]

I have an existing web camera app that I’m trying to port over to the Jetson Nano. This is a large app that uses OpenCV (cv::imdecode, cv::imwrite, etc.) extensively.

To display the web camera, the app uses OpenGL. The web camera outputs video in MJPEG. To display in OpenGL, the app first has to decode the MJPEG stream before sending to OpenGL for rendering.

I want to speed up this decoding step by using NvJPEGDecode. However, based on posts such as these:

Segmentation fault with OpenCV and Jetson multimedia API integration

Hi, OpenCV links to libjpeg.so and does not work with libnvjpeg.so. please check explanation in You can try to decode the image through NvJpegDecoder instead of cv::imread().

…it looks like it is not possible to use NvJPEGDecoder and OpenCV in the same app. Is this correct?

To test this, I created a small test program that calls cv::imdecode. It works correctly at first.

But if I add NvJPEGDecoder in the same program, it fails.

If I only link to nvjpeg, then the test program segfaults when it calls cv::imdecode.

If I link to nvjpeg and instantiate NvJPEGDecoder, it doesn’t segfault, but the cv::Mat returned by cv::imdecode doesn’t contain data (cv::Mat::data is null), so it fails in a different way.

So this seems to indicate that it’s not possible to use OpenCV and NvJPEGDecoder in the same program. Is this accurate?

[DaneLLL]

Hi,
We have seen issues when using nvjpegdec with OpenCV functions. A possible solution is to use nvv4l2decoder mjpeg=1. Please try this string:
Set camera decoder in OpenCV on Jetson Nano - #8 by DaneLLL

[whatnot]

Hi DaneLLL,

The post you linked assumes usage of cv::VideoCapture, right? I’m not using cv::VideoCapture at all though.

Here is the test code I’m using:

#include "test_jpeg_data.h"
#if defined(INSTANTIATE_NVJPEG_DECODER)
#   include "NvJpegDecoder.h"
#endif
#include <opencv2/opencv.hpp>
#include <vector>
#include <cstdint>
#include <memory>
#include <iostream>

using std::vector;
using std::unique_ptr;
using std::cerr;

int main()
{
#if defined(INSTANTIATE_NVJPEG_DECODER)
    auto upDecoder = unique_ptr<NvJPEGDecoder>{
        NvJPEGDecoder::createJPEGDecoder("jpegdec")
    };
#endif
    const auto jpeg = vector<uint8_t>{
        test_jpeg_data,
        test_jpeg_data + sizeof(test_jpeg_data)
    };
    cerr << jpeg.size() << "\n";
    const auto mat = cv::imdecode(jpeg, cv::IMREAD_COLOR);
    if (mat.data)
    {
        cerr << "Success.\n";
    }
    else
    {
        cerr << "FAILED.\n";
    }
}

[DaneLLL]

Hi,
It is known that NvJPEGDecoder does not work in OpenCV. Please use jpeg decoder in OpenCV, or run gstreamer string with nvv4l2decoder mjpeg=1 in cv2.VideoCapture().

[whatnot]

Ok, thank you, DaneLLL. That is what I needed to know.