If I use the gbrp pixel format the colors are correct; however in light of the issue I had here: Av1_nvenc cannot use gbrp pixel format even if it should be able to
I ended up trying to use the bgr0,rgb0,bgr24,rgb24,rgba,x2rgb10le,x2bgr10le pixel formats with nvenc for screen capture.
And after testing on h264_nvenc, hevc_nvenc and av1_nvenc I’ve come to the conclusion that all of them subtly but noticeably output colors wrong with this pixel format, and this issue carries to all other implementations
To demonstrate:
Screenshot:
av1_nvenc using bgr0 (mismatched colors)
If I use bgr0 with libx264rgb encoder I get correct colors, so this must be an nvenc specific issue.
Notably if I look at the files in mpv, the colormatrix for the bgr0 outputs of nvenc is reported as bt601, but for gbrp and for libx264rgb’s bgr0 it is reported as rgb, this is probably the actua cause, or at least directly related to it.