16 bit Overlays broken with X11 and 4XX series driver

I just rolled back to 390.132 using the P2000 and the overlays work.
Assuming the 4XX series lineage maybe it broke at the branch to 410.57?

Hi nmcracker,
Please help to provide concrete repro steps so that I can try to recreate issue locally.

I’m not certain how best to answer this. I have a complex custom application that uses ffmpeg to handle my overlays.
Is there a good way to evaluate the system state using randr or xdpyinfo?
other than that a bare install of RHEL7.3 and the NVIDIA driver should be enough I think.

Will it be possible for you to share application or guide me with commands which can recreate issue.

sorry this is kinda difficult. What I understand is that we first search for a visual that has the overlay attribute then allocate the 16bit colormap for the visual. I would expect a small app that allocates the 16bit colors for the X drawable then draws a line should be able to reproduce the issue. I havent been able to get a test app shareable.

Between 390 and 410 the driver layout changed heavily, so seems some parts fell off.
Does glxspheres -o work as a repro scenario?

glxspheres -o fails to find a visual for both good and bad drivers.

please let me know if you have reproduced the issue or if I need to work harder at getting a tool build that won’t upset the powers that be.

Any updates I can pass along to my boss? I haven’t been able to get much traction beyond thinking that I am sending RGB555 to something that may be expecting RGB565.

If you can’t upload complete logs, can you please check xdpyinfo and verify that you have both 24-bit and 16-bit visuals showing up, and glxinfo and verify that there are some 16-bit GLX visuals there?

If you do see 16-bit visuals there, can you please test with an app other than your custom one? For example, atlantis -visual <vid> works pretty well, using the Atlantis app from the xscreensaver package.

The 16 bit visuals are present, xdpyinfo and glxinfo show 16bit 555 visuals.
Atlantis -visual 0x23 seems to work.

I have been trying to get a test application completed and now I am curious how Xv and putImage affect this. My test app as it stands now is doing what I expect but it is missing all the underlay components for video frame rendering which I think we do via Xv shared memory. There is a huge amount of code to skeletonize to try and get to a “failure” state so it may take a day or two. Is there anything special nvidia_drv does to handle Xv?

I finally reproduced the issue with a test application today. I don’t yet know if I can get permission to upload the source, What seems to be breaking the overlays is the pixmap I copy the overlay from.
something like this should do the trick.

XSetForeground(display, overlayGC, transparentColor);
XFillRectangle(display, overlayPixmap, overlayGC, 0,0, 100, 100);

XSetForeground(display, overlayGC, 0x7ffff); // white 555
XFillRectangle(display, overlayPixmap, overlayGC, 0,0, 74, 74);
XCopyArea(display, overlayPixmap, overlayWindow, overlayGC, 0,0, 100,100, 0,0);

Has there been any progress at reproducing this on your end?

Not yet, we will need application in order to attempt again or please help us to guide to any open source application which recreate same issu.

here is the zip of my test app.

nvidiaZip.c (3.8 KB)

Hate to keep being a bother but did my test app help?

Sorry for the slow reply, I’ve been swamped with other work lately.

I tried out your test app and confirmed that rendering into a pixmap and then copying it into the overlay window does indeed interpret the pixels in 565 format, while rendering directly into the window uses the correct 555 format. The handling of colors with these 16-bit overlays is a little weird due to the way the driver handles transparency of these windows.

I don’t think that code has changed in many years, so I’m surprised to hear that this is a regression. I’ll try to get an old system set up to try the older driver.

I confirmed that this is a regression and filed internal bug number 3087961. Thanks for reporting this and providing the detailed test case!

1 Like

Quadro P4000 and P5000 here… same issue on RHEL8.3 with 457.xx but worked before on RHEL7.8 with older NV drivers. Thanks to keep us updated :)