V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE not working with AV1 encoder

I am setting a long GOP size and IDR interval, and my application requests the encoder send a key frame if required.

This works with the HEVC encoder, but I cannot get it working with the same code on AV1. I get a key frame generated at the start of the stream, but using the above control does not trigger a new key frame any time afterwards, or at least the V4L2_BUF_FLAG_KEYFRAME flag does not get set on the buffer.

I have also tried V4L2_CID_MPEG_VIDEOENC_FORCE_IDR_FRAME but that has no effect on either HEVC of AV1 encodes.

So it appears that this option does generate an I-frame but not an IDR-frame.

I can’t figure out how to make it work the same as the HEVC encoder, where I-frames are IDR-frames, either as a one-off or for the whole stream.

I’ve tried placing calls to V4L2_CID_MPEG_VIDEOENC_FORCE_IDR_FRAME in various stages, as maybe this option is intended to achieve this? The problem is the documentation is so poor, I’m just struggling along trying things using trial and error.
It’s not clear what V4L2_CID_MPEG_VIDEOENC_FORCE_IDR_FRAME is meant to do, I was hoping it could be used to tell the encoder that all forced I-frames should be IDR frames, but it seems not. It has no apparent effect whatsoever.

The samples have a function called “forceIDR” which just uses V4L2_CID_MPEG_MFC41_VIDEO_FORCE_FRAME_TYPE but as I say, this does not work, in my experience with AV1.

Help!

So I can see this was actually reported back in 2022:

That issue was “closed” but it’s not clear the problem was solved. Certainly the “solution” refers to JP 5.1.1, but I am already using 5.1.1.

If it has since been solved in 5.1.2 or 6.0 I can’t find anything in the release notes - can someone confirm if this issue is still active or, if it has been solved, which version the fix appears in?

Also, further investigation shows that the issue is that V4L2_CID_MPEG_MFC41_VIDEO_FORCE_FRAME_TYPE does indeed force an I-frame but it is not an IDR frame/key frame. This is shown by the increased size of the produced packets. I tried treating them as key frames after a previous discontinuity but this just produced corruption on the screen when decoding, so they were obviously not IDR frames.

Thanks!

Hi,
For forcing IDR frames, please try the commands:
AV1 encoder does not work rpc option for changing bitrate and framerate - #12 by DaneLLL

Sorry what commands - I’m not using gst, what are the actual API calls you are suggesting I use in my code?

Hi,
In the post, there are reference commands of running 01_video_encode. Please configure i1 for forcing IDR frames:

Runtime configurable parameter string should be of the form:
"f<frame_num1>,<prop_id1><val>,<prop_id2><val>#f<frame_num1>,<prop_id1><val>,<prop_id2><val>#..."
e.g. "f20,b8000000,i1#f300,b6000000,r40/1"

Yes and that code calls forceIDR which I am telling you is not working when I use the equivalent (V4L2_CID_MPEG_MFC41_VIDEO_FORCE_FRAME_TYPE) in my code, for AV1.

I am asking if the issue I have reported, which was also reported in this post in 2022 AV1 using jetson orin forceIDR() and bitrate change not working? and which you acknowledged at the time hadn’t been implemented yet, has now been implemented in a later release, and if so which one?

Hi,
We run the command on Jetpack 5.1.3 and can see forceIDR take effect:

01_video_encode$ ./video_encode a.yuv 1920 1080 AV1 a.av1 --report-metadata -idri 60 -rpc "f33,i1#f121,i1"

Could you help give it a try on 5.1.1? We would need some time to set up a 5.1.1 device and it would be great if you can help try it.

Sure here you.

test.out.txt (128.2 KB)

Can I see your results, so I know what to expect? I"m guessing my results illustrate the problem, because after forceIDR there aren’t any key frames. The only key frames are presumably the regular IDR interval ones. Am I right?

Ok, I upgraded one of my dev machines to 5.1.3, and it appears that forceIDR for AV1 is working on this version.

I can’t find any reference to this on any release notes anywhere though, it would have saved me a huge amount of ball ache if it had been listed on the 5.1.2 or 5.1.3 notes, depending on when it was fixed :o)

For the record, I was able to re-use the binary built under 5.1.1 so the fix must have been in libnvv4l2.so or something, and not a later version of the multimedia API.

Hi,
Although the binary of 5.1.1 works on 5.1.3, we would suggest rebuild it on 5.1.3. The header files may be different between releases and better to rebuild it with the header files of 5.1.3. To avoid potential issues from mismatch of header files.

forceIDR of AV1 encoding is tracked and fixed internally and we may miss to list it in release notes. Sorry for this.

Yes your are probably right.

I’m actually using the low level v4l2_ioctl calls and buffer surface/transform and not any of the higher level multimedia APi classes.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.