How to set v4l2_enc_hw_tuning_info_type in the xavier encoder?

I want to configure xavier h265 encoder with v4l2_enc_hw_tuning_info_type=V4L2_ENC_TUNING_INFO_ULTRA_LOW_LATENCY.

But not able to find an API to do this.
Any idea how to do this?


We don’t support the control in v4l2 stack. For all options, please check 01_video_encode sample.

For encoding with low latency, you may enable slice level encoding and try. There are examples:
Encode H264 by slice - #6 by DaneLLL
How does slice mode video encoding work for Jetson? - #14 by DaneLLL
Encoder output metadata report - #3 by DaneLLL


 I thought ultra  low latency mode is supported with HEVC ME mode (not sure what it means) encoder.

I read slice mode link you provided and a statement below is confused:

We don’t support the mode encoder_capture_plane_dp_callback is called per slice. It is called per a >>complete frame.
How do I know when an encoded slice is ready to read?


The constraint is for old release. In later releases, we have –sle to enable slice output. Please give it a try.

I build 01_video_encode app and use slice encoding like the following command:

./video_encode test.yuv 1920 1080 H265 out.h265 -br 4000000 -nbf 0 --sle -slen 2040 -slt 1

There is no slice capture plan API ready in 01_video_encode app code so I can extract compress bitstream earlier than without slice

My question is which API provides availability of slice capture plane



We are uncertain what this function is. Please share more information/explanation about what you would like to do.


The purpose of using Slice encoding is to reduce the latency so the compressed video data can be read when each slice encoding is completed。 Is there an API to notify each slide encoding completed and compressed video of that slice us ready for my application to read?


The callback function will be called when encoded slice is ready. Please try the commands to show the cases with/without –sle:

01_video_encode$ gst-launch-1.0 nvarguscamerasrc num-buffers=10 ! nvvidconv ! video/x-raw,width=640,height=480,format=I420 ! filesink location= a.yuv
01_video_encode$ ./video_encode a.yuv 640 480 H264 a.h264 -slt 1 -slen 300 --report-metadata
01_video_encode$ ./video_encode a.yuv 640 480 H264 a.h264 -slt 1 -slen 300 --report-metadata --sle

I can see difference from --report-metadata with sle and without sle. on H264.
However, it seems not working with H265.
Can you confirm H265 also supports this feature?


For H265, the calculation of macroblocks is:
Multiple slices on Jetson NX Jetpack 4.6 - #11 by adi4

Please try

01_video_encode$ ./video_encode a.yuv 640 480 H265 a.h265  -slt 1 -slen 100 --report-metadata --sle

When I use the following command:
./video_encode src.yuv 640 480 H265 out.h265 -br 4000000 -nbf 0 -slen 100 -slt 1 --report-metadata --sle

I can see sle working OK up to Frame 31 (report-metadat reports 3 times each frame). Then report-metadata respots 2 times/frame up to Frame 63. After that, report-metadata reports once/frame.
You should be able to see this pattern from the log below.
There are some error messages (NVMAP_IOC_WRITE Failed) at the beginning. What does this error message mean?

Creating Encoder in blocking mode
NvMMLiteOpen : Block : BlockType = 8
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
NVMEDIA: H265 : Profile : 1
NVMEDIA: Need to set EMC bandwidth : 126000
NVMEDIA: Need to set EMC bandwidth : 126000
NVMEDIA_ENC: bBlitMode is set to TRUE
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 40960 SrcStride 8192 pDst 0xffffa4f7bfa8 DstStride 8192 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 49152 SrcStride 8192 pDst 0xffffa4f7bfa8 DstStride 8192 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 57344 SrcStride 4206 pDst 0xffffa4f7bfa8 DstStride 4206 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 40960 SrcStride 3070 pDst 0xffffa4f7bfa8 DstStride 3070 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
Frame 0: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=61550 EncodedBits=0
Frame 1: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=44030 EncodedBits=0
Frame 2: isKeyFrame=1 AvgQP=23 MinQP=15 MaxQP=31 BUfferBits=29173 EncodedBits=1077432
Frame 3: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19496 EncodedBits=0
Frame 4: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19288 EncodedBits=0
Frame 5: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=18171 EncodedBits=455640
Frame 6: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=9844 EncodedBits=0
Frame 7: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=8424 EncodedBits=0
Frame 8: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=4326 EncodedBits=180752
Frame 9: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1011 EncodedBits=0
Frame 10: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=882 EncodedBits=0
Frame 11: isKeyFrame=0 AvgQP=42 MinQP=40 MaxQP=43 BUfferBits=793 EncodedBits=21488
Frame 12: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1504 EncodedBits=0
Frame 13: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=2162 EncodedBits=0
Frame 14: isKeyFrame=0 AvgQP=40 MinQP=39 MaxQP=41 BUfferBits=3034 EncodedBits=53600
Frame 15: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1598 EncodedBits=0
Frame 16: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=836 EncodedBits=0
Frame 17: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=1060 EncodedBits=27952
Frame 18: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=3804 EncodedBits=0
Frame 19: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=3517 EncodedBits=0
Frame 20: isKeyFrame=0 AvgQP=37 MinQP=37 MaxQP=38 BUfferBits=5442 EncodedBits=102104
Frame 21: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=14976 EncodedBits=0
Frame 22: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=15236 EncodedBits=0
Frame 23: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=14431 EncodedBits=357144
Frame 24: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19378 EncodedBits=0
Frame 25: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19374 EncodedBits=0
Frame 26: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=38 BUfferBits=19114 EncodedBits=462928
Frame 27: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=7127 EncodedBits=0
Frame 28: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=3504 EncodedBits=0
Frame 29: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=2638 EncodedBits=106152
Frame 30: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1019 EncodedBits=0
Frame 31: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=793 EncodedBits=0
Frame 32: isKeyFrame=0 AvgQP=41 MinQP=40 MaxQP=42 BUfferBits=841 EncodedBits=21224
Frame 33: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=2624 EncodedBits=0
Frame 34: isKeyFrame=1 AvgQP=23 MinQP=15 MaxQP=31 BUfferBits=2565 EncodedBits=1077432
Frame 35: isKeyFrame=0 AvgQP=39 MinQP=38 MaxQP=40 BUfferBits=2281 EncodedBits=59760
Frame 36: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=899 EncodedBits=0
Frame 37: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1392 EncodedBits=455640
Frame 38: isKeyFrame=0 AvgQP=38 MinQP=38 MaxQP=39 BUfferBits=1538 EncodedBits=30632
Frame 39: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=5599 EncodedBits=0
Frame 40: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=7145 EncodedBits=180752
Frame 41: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=10482 EncodedBits=185808
Frame 42: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=15512 EncodedBits=0
Frame 43: isKeyFrame=0 AvgQP=42 MinQP=40 MaxQP=43 BUfferBits=16334 EncodedBits=21488
Frame 44: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=17807 EncodedBits=397224
Frame 45: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=13599 EncodedBits=0
Frame 46: isKeyFrame=0 AvgQP=40 MinQP=39 MaxQP=41 BUfferBits=14848 EncodedBits=53600
Frame 47: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=14399 EncodedBits=342768
Frame 48: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=2211 EncodedBits=0
Frame 49: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=1663 EncodedBits=27952
Frame 50: isKeyFrame=0 AvgQP=43 MinQP=43 MaxQP=44 BUfferBits=1163 EncodedBits=40296
Frame 51: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=869 EncodedBits=0
Frame 52: isKeyFrame=0 AvgQP=37 MinQP=37 MaxQP=38 BUfferBits=1356 EncodedBits=102104
Frame 53: isKeyFrame=0 AvgQP=42 MinQP=41 MaxQP=43 BUfferBits=1939 EncodedBits=33312
Frame 54: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1451 EncodedBits=0
Frame 55: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1200 EncodedBits=357144
Frame 56: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=1158 EncodedBits=30472
Frame 57: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1049 EncodedBits=0
Frame 58: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=38 BUfferBits=1109 EncodedBits=462928
Frame 59: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=746 EncodedBits=23232
Frame 60: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=6307 EncodedBits=0
Frame 61: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=10851 EncodedBits=106152
Frame 62: isKeyFrame=0 AvgQP=37 MinQP=37 MaxQP=38 BUfferBits=13417 EncodedBits=244600
Frame 63: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=13014 EncodedBits=0
Frame 64: isKeyFrame=0 AvgQP=41 MinQP=40 MaxQP=42 BUfferBits=15909 EncodedBits=21224
Frame 65: isKeyFrame=0 AvgQP=38 MinQP=38 MaxQP=39 BUfferBits=14915 EncodedBits=350704
Frame 66: isKeyFrame=1 AvgQP=23 MinQP=15 MaxQP=31 BUfferBits=9525 EncodedBits=1077432
Frame 67: isKeyFrame=0 AvgQP=39 MinQP=38 MaxQP=40 BUfferBits=10160 EncodedBits=59760
Frame 68: isKeyFrame=0 AvgQP=41 MinQP=41 MaxQP=42 BUfferBits=8608 EncodedBits=226344
Frame 69: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1160 EncodedBits=455640
Frame 70: isKeyFrame=0 AvgQP=38 MinQP=38 MaxQP=39 BUfferBits=857 EncodedBits=30632
Frame 71: isKeyFrame=0 AvgQP=44 MinQP=44 MaxQP=44 BUfferBits=701 EncodedBits=21744
Frame 72: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=1908 EncodedBits=180752
Frame 73: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1679 EncodedBits=185808
Frame 74: isKeyFrame=0 AvgQP=42 MinQP=42 MaxQP=43 BUfferBits=1612 EncodedBits=41592
Frame 75: isKeyFrame=0 AvgQP=42 MinQP=40 MaxQP=43 BUfferBits=1261 EncodedBits=21488
Frame 76: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1244 EncodedBits=397224
Frame 77: isKeyFrame=0 AvgQP=41 MinQP=41 MaxQP=42 BUfferBits=596 EncodedBits=24808
Frame 78: isKeyFrame=0 AvgQP=40 MinQP=39 MaxQP=41 BUfferBits=750 EncodedBits=53600

You are correct there are issues in slice level encoding. We see two issues:

  1. - -report-metadata does not report correct information in slice level encoding
  2. -slt 1 does not work properly

We will investigate further.

On 5.1.2 or 5.1.3, please run in Slice length type = Bytes:

$ ./video_encode a.yuv 640 480 H265 a.h265 -slt 2 -slen 10000 --sle
$ ./video_encode a.yuv 640 480 H264 a.h264 -slt 2 -slen 10000 --sle

And please manually add print of buffer size in encoder_capture_plane_dq_callback() to check.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.