NvV4l Encoder extreme variance in frame sizes (+setQpRange not working)

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hi,

TL;DR
The main question is:
Is there a way to limit the variance of frame sizes produced by the NvV4l Encoder?

I am using a Jetson AGX Xavier, with 4.9.140-tegra kernel release.

I am developing an application which captures video frames from a USB Basler camera, encodes them using the NvV4l Encoder interface and sends the frames to a remote server (along with FEC protection).
A basic flow chart could look like this:
[capture frame] --> [Encode frame] --> [Generate FEC packets] --> [TCP to server]

The Encoder’s rate control mode is configured to CBR (V4L2_MPEG_VIDEO_BITRATE_MODE_CBR).
I am working at 25 FPS, and the bitrate is configured to 10Mbps.
So the average encoded frame size should be 50B (10,000,000 / 25 / 8).

Here is an example of the results for a session with 27762 frames (approximately 18.5 minutes):

  • Average frame size: 49,045B (indeed very close to the requested bitrate)
  • Median frame size: 36,789B
  • Max frame size: 444,107B
  • Min frame size: 122B
  • Standard deviation: 33724
  • 8% (or 2208 frames) were larger that 100KB
  • 12 frames were >200KB

The extremely large frames are causing problems in our system.
Specifically, the FEC generation for such large frames seems to be too heavy for the Jetson’s CPUs, resulting in very large generation times.

I have tried to limit the QP range, in an attempt to also limit the range of frame sizes (which are correlated to QP), but the configuration did not seem to affect the actual measured QP.
For example, when limiting the QP to 25-47:

ctx.nMinQpI =  ctx.nMinQpP = ctx.nMinQpB = 25;
ctx.nMaxQpI =  ctx.nMaxQpP = ctx.nMaxQpB = 47;
enc->setQpRange(ctx.nMinQpI, ctx.nMaxQpI, ctx.nMinQpP, ctx.nMaxQpP, ctx.nMinQpB, ctx.nMaxQpB);

The resulting average QP was 19.

It would be great to know the following:
1. Is the variance we have observed in frame sizes consistent with the behavior you would expect from the Jetson’s Encoder?
2. Is there a way to limit the variance in frame sizes?
3. Why the setQpRange not working? Are we using it wrong? Should we set additional configurations in order to limit the QP?

Thanks in advance,
Tuval

3

Hi,
setQpRangeworks in VBR mode only. Please set VBR mode and try again.

4

Thanks for the answer DaneLLL, that’s good to know.
But, our desired behaviour is for the bitrate to be as stable as possible.
I assume VBR with qpRange is not the best way to get a constant bitrate, right?

Could you please address the following questions?
1. Is there a way you recommend to make the bitrate more stable, with less fluctuations?
2. Is the deviation in frame sizes, which I described above, the standard Encoder behavior you would expect to see?

5

Hi,
You may set CBR mode + virtual buffer size. Please refer to