Xavier NX OpenCV fullscreen window performance

[jan.juran]

Hi, I am running JetPack 4.5.1 and do a camera capture and then some image processing and finally display the result in a window. Works OK in windowed mode, but if I switch the window to fullscreen like this:

    cv::namedWindow("currentImage", cv::WindowFlags::WINDOW_NORMAL);
    cv::setWindowProperty("currentImage", cv::WindowPropertyFlags::WND_PROP_FULLSCREEN, cv::WindowFlags::WINDOW_FULLSCREEN);

…then the performance goes to hell.
In my case, I am trying to show a 808x620 image and display it on FullHD screen.

My understanding is that the problem is that it’s trying to upsample the image on CPU. Is that right?
Now since the OpenCV included in JetPack is built without OpenGL support, seems to me I have little options and that it will always happen on the CPU side.

Is there anything I can do to improve the performance? (upsampling quality settings, anything…)

I may want to consider building OpenCV with OpenGL support, but it looks kind of complicated. I found this thread:

opencv with GL support at Xavier

Note I’ve not been trying any OpenGL callback… Was just a basic thing that may succeed a cast in one case and not with the other one. You may share your openGL findings.

But, I kind of get lost in all the instructions. Plus it’s for AGX, so I am not sure if that also applies for NX.

[DaneLLL]

Hi,
Please check if you can apply your use-case to this sample:
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL

And use nvoverlaysink in cv2.VideoWriter() like:

    gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvoverlaysink "

[jan.juran]

I don’t really much understand how does this relate to GStreamer. I am reading stuff from cv::VideoCapture, then do some image processing and finally just display it on the screen. And the problematic part is the display part, if it involves upsampling the image (which is the fullscreen case).

However, I was able to do some more progress yesterday after writing the post. I tried to use UMat, so that OpenCL can kick in, and upsample the image manually with cv::resize (with OpenCL acceleration obviously) to the actual screen size. In this case, the image does not have to be upsampled by the CPU.

Unless someone has a better suggestion, I think I can live with that…

[DaneLLL]

Hi,
FYR. If the resolution fits display device, it is rendered in full screen while using nvoverlaysink. May try this command and check the overlay:

$ gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=360 ! nvoverlaysink

Good to know you have a working solution and thanks for the sharing.