Xlib's XGetImage() can only capture every second frame on TX2 (maybe a bug of graphics-driver?)

Hi,

I’m observing a strange behavior regarding calls to XGetImage(...) (Xlib-API) on my TX2. This method should provide a screen dump, but it seems to behave differently than on other platforms. On my TX2 it is only capable of capturing every second frame, which maybe seems to be an issue of the graphics-driver.

Setup:
TX2-Devkit with 32.6.1 and the related sample-rootfs (Ubuntu 18.04). A monitor is connected via HDMI.

What I’m trying to achieve:

  • There are two application (let’s call them A and B).
  • A is rending a simple application using SFML-graphics-library.
  • A notifies B whenever a new frame is available (=frame is rendered by SFML and flushed).
  • B then captures the window of A by using Xlib’s call to XGetImage(...) and further processes this data.

The Problem:
While this works on other platforms (e.g. a Ubuntu-PC, a Raspberry-Pi, …) I’m observing a strange behavior on my TX2. It seems as if only every second frame can be captured by XGetImage(...). I’m getting something like:

Rendered F1 -> Captured F1
Rendered F2 -> Captured F1
Rendered F3 -> Captured F3
Rendered F4 -> Captured F3
Rendered F5 -> Captured F5
...

My first thought was, that this is related to timing and that the images are captured too fast/slow/early/late. To test this I reduced my rendering rate to 1Hz and periodically called XGetImage() (every 200ms). However, this resulted in the the same strange behavior:

Rendered F1 -> 5x Captured F1
Rendered F2 -> 5x Captured F1
Rendered F3 -> 5x Captured F3
Rendered F4 -> 5x Captured F3
Rendered F5 -> 5x Captured F5
...

Possible Issue:
It somehow seems like there is a double buffering and XGetImage() only is capable of reading one of the buffers on my TX2. I think this is related to the graphics driver, because I’m not observing this on any other platform (e.g. a Ubuntu-PC, a Raspberry-Pi, …).

What I already tested:

  • Every frame is visible on the screen (HDMI Display)
  • I exchanged the graphics library (SFML, SDL, …)
  • I reduced the whole code to one minimal application → This application only changes the background color of a window and then captures it → example_app.cpp (1.7 KB)
  • I tested the code on other platforms → I never observed that behavior on them.
  • I tested the code on the TX2 with Xvfb (X Window Virtual Framebuffer) instead of the HDMI output. → Works well.

Due to this tests, I’m quite sure that this must be an graphic-driver related setting or issue and is not related to my application.

Questions:
Is there any way to avoid this behavior on my TX2?
Do you agree, that this is an issue of the graphic-driver and not of my application?
Is this behavior intentional or a bug?

Thanks in advance!

Hi,
For capturing screen, suggest try ximagesrc. Please refer to this post:
Zero-copy screen capture - #3 by DaneLLL

Can try this pipeline and check frame rate:

$ gst-launch-1.0 -v ximagesrc num-buffers=100 use-damage=0 ! video/x-raw ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=NV12’ ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0

Hi,

I haven’t thought about using gstreamer until now. It is not optimal for my specific application, but with a little bit of work I can definitely use that.

I tested your pipeline and can confirm that I get the correct frame rate, but only if I change use-damage to 1. Otherwise I get 25FPS, no matter how fast my application renders the frames.

HOWEVER…
I created the same pipeline with an appsink at the end to test this further. The appsink prints me some pixels of the captured frames. And again, I see the same issue as described in the original post → I get two times the same frame and the frames between them are skipped. (Note: It doesn’t matter if I turn on or off use-damage, because I’m only seeing more ‘wrong’ frames if I turn it off).

I had a short look at the source code of gstreamer’s ximagesrc. It internally uses the same API as I have planed to use → XGetImage() or XShmGetImage()if the shared memory extension is available (Note: I already tested my application with both of them).

To me this is another indicator that there is a problem with X11 and the graphics driver…
Can you please give me further advices on how to proceed?

Thanks in advance!

Hi,

here is maybe a more reproduceable path to see the problem:
I use an simple application which switches between red, green, blue and white at a frame rate of 5Hz.
The display output is as expected (captured with a camera):


I then capture a video by using the gstreamer-pipeline from the post you mentioned: Zero-copy screen capture - #3 by DaneLLL

$ gst-launch-1.0 ximagesrc num-buffers=200 use-damage=0 ! video/x-raw ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=a.mp4

This results in a video showing only the red and blue frames, and missing all the green and white ones:


Greetings!

Hi,

some further update:
I have tried to enabled triple buffering in the xorg.conf-file (see How to enable triple buffering).
This results in the same frame being captured three times and skipping the two frames in between.

For me that is another indicator that there is an issue with the implementation of the graphics driver. Somehow it seems like a fixed buffer is being read and not the currently active one like it should be.

Can you please investigate this from your side?
Thanks in advance!

Hi,
These are high-level software frameworks and would need other users check and provide suggestion.

If you suspect something wrong in graphics driver, please try to reproduce it with the samples in

/usr/src/nvidia/graphics_demos

So that we can replicate the error and check with our teams.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.