Video decoders do not properly handle dynamic resolution changes

I’m working with video streams that can change resolution dynamically. Specifically, this is H264 video coming from a mobile phone where the resolution flips from 1280/720 to 720/1280 when the user changes from portrait to landscape mode.

When using nvv4l2decoder, the flipped video looks completely garbled. To me, it looks like the video continues to be decoded at the original resolution. I also tried the deprecated omxh264dec - the results were similar.

Additionally, this is continuously output to the log when the resolution is changed:

NVMAP_IOC_READ failed: Invalid argument
NVMAP_IOC_READ: Offset 0 SrcStride 1280 pDst 0x7f34006470 DstStride 720 Count 1280

I created a sample program that reproduces the issue:
https://github.com/Adjacence/nvv4l2decoder-res-change-bug

Hi,
we do not support dynamic resolution change in gstreamer nvv4l2decoder. Would suggest you use tegra_multimedia_api and refer to 00_video_decode sample.

Okay, if that’s the case, then that should be made clear.

Other GStreamer plugins, such as matroskamux, cannot handle CAPS changes. So, when the CAPS change occurs on one of these plugins, an error is issued and the pipeline fails. However, nvv4l2decoder doesn’t communicate any of that information to the programmer, other than printing nonsense to the log and producing a garbled video. Is this limitation even documented anywhere?

Hi,
The implementation is based on gst-v4l2 open code. You may download it from
https://developer.nvidia.com/embedded/dlc/r32-3-1_Release_v1.0/Sources/T186/public_sources.tbz2

And our understanding is that it is not the case supported in default open source code. Probably our team does not realize the code well enough. If you are able to look at the code and share us suggestion/guidance, it would be great. The low-level v4l2 implementation does support it and we have demonstrated it in 00_video_decode.