USB Camera resolution 640x480 w/OpenCV4Tegra

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Just a note since this took me a while to track down. The OpenCV version on the TX2 after installing L4T is OpenCV 2.4.13.

OpenCV 2.4.13 appears to have an issue where it limits USB webcam frame captures to 640x480 even if you set the width / height to something else. When I do a set frame width or height with a webcam that supports higher resolutions, I get a HIGHGUI error and it ignores the dimensions set and leaves them at 640x480. This appears to be a known problem in some versions of OpenCV 2.4.12 and it appears to have carried over to 2.4.13.

One possible solution is to rebuild OpenCV 2.4.13 and fix the problem by hardcoding the resolution to the higher value I need. I believe the file to change is cap_v4l.cpp and the values are DEFAULT_V4L_WIDTH and DEFAULT_V4L_HEIGHT.

Alternatively, I could build OpenCV 3.1 or 3.2 and install it as it doesn’t have the problem.

However, either way I build myself, I end up with this:

“Note: Compiling OpenCV from source will not give you NVIDIA’s CPU optimizations that are only available in the closed-source prebuilt OpenCV4Tegra packages.”

You can read that here:

http://elinux.org/Jetson/Installing_OpenCV

For reference, this is another page showing how to uninstall OpenCV4Tegra and install OpenCV 3 on a TX1 since the process should be similar for the TX2

http://dev.t7.ai/jetson/opencv/

However, looking at others performance results after rebuilding either OpenVC 2 or 3 themselves, it looks like it is best to stay with the default OpenCV with the L4T distribution if possible.

So, one further approach I might attempt here is to edit the shared library binary files directly and change the 640 and 480 values where they are used (they appear to be used in 2 places) right in the libraries.

In the future, it would be nice to see OpenCV 3.2 supported with OpenCV4Tegra since it appears to have fixed this issue.

2

Sure it would be nice to get a nvidia-optimized opencv-3.2.0 for tegraX.

As far as I understand, many optimizations for tegra were acheived through gcc4 compiler options, and a better compiler such as linaro gcc5 could acheive some optimizations by itself. Opencv-3.2.0 also uses carotene from nvidia and can use low-level optimizations for tegra multi-cpus.

My own experience on TK1/TX1 is that opencv4tegra is faster for some parts, but for some others it may be better with opencv3, so just benchmark both solutions for your application (averaging multiple runs).

However, I have to say that on TX2 I’ve only tried using opencv4tegra and went onto errors, felt it was mainly built for TX1 R24 (I’m using L4T R27.0.1), thus so far I’ve only been using opencv-3.2.0, that further allows to use gstreamer support with opencv.

3

I just whipped up a quick patch to move ahead on this. I patched the libopencv_highgui.so.2.4.13 shared library directly to change the 640x480 default to 1280x720. I have minimally tested it but it gets me what I need right now, 1280x720 opencv USB webcam RGB frames without having to install a new OpenCV.

I was working with a blindfold without source so its purely experimental. If you do experiment with this, make sure you make a backup copy of the library first. It is specific to the OpenCV4Tegra from Jetpack 3.0 on the Nvidia TX2. The patch changes constants in parts of the open() function, changing the 640 and 480 assembly movz instruction constants to 1280 and 720 in the 8 places I found them.

How to patch:

  1. make a copy of /usr/lib/libopencv_highgui.so.2.4.13 so you have the original
  2. read through, build (cc, no options) and run this C program

There is a sanity check in there to look for the expected “before” values.

#include <stdio.h>
#include <inttypes.h>

int main(int argc, char **argv)
{
    typedef struct {
        uint32_t offset, beforeValue, afterValue;
    } Oba;
    Oba v[] = {
        {0x28314, 0x52803c11, 0x52805A11},      // 480 to 720 lines
        {0x285d4, 0x52803c03, 0x52805A03},
        {0x286d4, 0x52803c00, 0x52805A00},
        {0x28764, 0x52803c02, 0x52805A02},
        {0x288f4, 0x52803c0b, 0x52805A0B},
        {0x289f0, 0x52803c0f, 0x52805A0F},
        {0x28a74, 0x52803c1e, 0x52805A1E},
        {0x28b10, 0x52803c00, 0x52805A00},
        {0x28310, 0x52805010, 0x5280A010},      // 640 to 1280 lines
        {0x285cc, 0x52805005, 0x5280A005},
        {0x286f4, 0x52805006, 0x5280A006},
        {0x28768, 0x52805001, 0x5280A001},
        {0x288ec, 0x5280500a, 0x5280A00A},
        {0x289e8, 0x5280500e, 0x5280A00E},
        {0x28a6c, 0x52805012, 0x5280A012},
        {0x28b30, 0x52805006, 0x5280A006},
    };
    static int numV = sizeof(v) / sizeof(Oba);

    printf ("DO NOT RUN WITH THIS WITHOUT BACKING UP THE LIBRARY FIRST\n");
    return(0);

    FILE *f = fopen("libopencv_highgui.so.2.4.13", "r+");
    for (int i = 0; i < numV; i++) {
        uint32_t x;
        fseek(f, v[i].offset, SEEK_SET);
        fread(&x, 4, 1, f);
        if (x != v[i].beforeValue)
            printf("Location 0x%" PRIx32 " doesn't match unpatched expected value\n", v[i].offset);
        else {
            fseek(f, v[i].offset, SEEK_SET);
            fwrite(&v[i].afterValue, 4, 1, f);
            printf("Wrote 0x%" PRIx32 " to 0x%" PRIx32 "\n", v[i].afterValue, v[i].offset);
        }
    }
    fclose(f);
}
4

.

5

Hello ESJS

I compiled the program but it gives me Location error, i.e. line 38 happens

My file is exactly the “libopencv_highgui.so.2.4.13”

Could you please help me?

6

As you saw, line 38 is a sanity check to make sure the value in the library is the expected value before it modifies it. If it shows that error, it isn’t going to work as it would be modifying the wrong values.

To fix this, you would need to go through by hand and figure out the locations you need to modify. This involves disassembling the object code in the library, looking at the resulting assembly code and determining where the relevant 640, 480 width and height constants are. Once identified you would modify the appropriate location before values and after values in the static structure of the C program. It requires a basic understanding of assembly and is really a horrible hack that I came up with as a last resort to get things working quickly.

Building the current opencv library from source and using it is a much better way of solving this problem if it still exists in the lastest Jetson release.

The current version of OpenVC is 3.3 (August 2017) and it looks like it has a number of optimizations:

I’d download the source for it and build it instead of disassembling and modifying the C patching program. The current opencv release should have the full ability to use a custom width/height.

7

Thanks for explanations

Just a question

How much difference is there regarding the performance of opencv4tegra and conventional opencv?

I know that building with NEON and CUDA flags give both CPU and GPU optimization but I’m not sure how much it is weeker than tegra version? Is the difference really meaningfull?

8

I’m not sure. I still haven’t gotten around to reinstalling and rebuilding the OS/libraries on my system to bring it up so everything is the current version.

I’m going to have to do that in the next couple months and when I do, I’m just going to rebuild opencv with the latest version regardless of the performance impact. The Jetpack SDK will catch up at some point.

9

( accidental post )

10

Most of opencv4tegra optimizations have been included in opencv since v3.1.
Last JetPack 3.2 includes an opencv-3.3.1 that I have no tested yet, but it should be the best solution.

[EDIT: I may have been a bit optimistic…Check https://devtalk.nvidia.com/default/topic/1027301/jetpack-3-2-mdash-l4t-r28-2-developer-preview-for-jetson-tx2/#5225602 You will probably end building your own opencv-3.3.1 with proper CUDA and maybe gstreamer support.]