How to set v4l2_enc_hw_tuning_info_type in the xavier encoder?

fcj · March 15, 2024, 12:45am

I want to configure xavier h265 encoder with v4l2_enc_hw_tuning_info_type=V4L2_ENC_TUNING_INFO_ULTRA_LOW_LATENCY.

But not able to find an API to do this.
Any idea how to do this?

Thanks,

DaneLLL · March 15, 2024, 2:32am

Hi,
We don’t support the control in v4l2 stack. For all options, please check 01_video_encode sample.

For encoding with low latency, you may enable slice level encoding and try. There are examples:
Encode H264 by slice - #6 by DaneLLL
How does slice mode video encoding work for Jetson? - #14 by DaneLLL
Encoder output metadata report - #3 by DaneLLL

fcj · March 15, 2024, 2:54am

DaneLLL,

 I thought ultra  low latency mode is supported with HEVC ME mode (not sure what it means) encoder.

I read slice mode link you provided and a statement below is confused:

We don’t support the mode encoder_capture_plane_dp_callback is called per slice. It is called per a >>complete frame.
How do I know when an encoded slice is ready to read?

FC

DaneLLL · March 15, 2024, 4:01am

Hi,
The constraint is for old release. In later releases, we have –sle to enable slice output. Please give it a try.

fcj · March 15, 2024, 6:16pm

I build 01_video_encode app and use slice encoding like the following command:

./video_encode test.yuv 1920 1080 H265 out.h265 -br 4000000 -nbf 0 --sle -slen 2040 -slt 1

There is no slice capture plan API ready in 01_video_encode app code so I can extract compress bitstream earlier than without slice

My question is which API provides availability of slice capture plane

Thanks,

DaneLLL · March 18, 2024, 3:05am

Hi,

We are uncertain what this function is. Please share more information/explanation about what you would like to do.

fcj · March 18, 2024, 3:29am

DaneLLL

The purpose of using Slice encoding is to reduce the latency so the compressed video data can be read when each slice encoding is completed。 Is there an API to notify each slide encoding completed and compressed video of that slice us ready for my application to read？

Thanks,

DaneLLL · March 18, 2024, 6:54am

Hi,
The callback function will be called when encoded slice is ready. Please try the commands to show the cases with/without –sle:

01_video_encode$ gst-launch-1.0 nvarguscamerasrc num-buffers=10 ! nvvidconv ! video/x-raw,width=640,height=480,format=I420 ! filesink location= a.yuv
01_video_encode$ ./video_encode a.yuv 640 480 H264 a.h264 -slt 1 -slen 300 --report-metadata
01_video_encode$ ./video_encode a.yuv 640 480 H264 a.h264 -slt 1 -slen 300 --report-metadata --sle

fcj · March 18, 2024, 7:40am

I can see difference from --report-metadata with sle and without sle. on H264.
However, it seems not working with H265.
Can you confirm H265 also supports this feature?

FC

DaneLLL · March 18, 2024, 8:18am

Hi,
For H265, the calculation of macroblocks is:
Multiple slices on Jetson NX Jetpack 4.6 - #11 by adi4

Please try

01_video_encode$ ./video_encode a.yuv 640 480 H265 a.h265  -slt 1 -slen 100 --report-metadata --sle

fcj · March 18, 2024, 4:50pm

When I use the following command:
./video_encode src.yuv 640 480 H265 out.h265 -br 4000000 -nbf 0 -slen 100 -slt 1 --report-metadata --sle

I can see sle working OK up to Frame 31 (report-metadat reports 3 times each frame). Then report-metadata respots 2 times/frame up to Frame 63. After that, report-metadata reports once/frame.
You should be able to see this pattern from the log below.
There are some error messages (NVMAP_IOC_WRITE Failed) at the beginning. What does this error message mean?

Creating Encoder in blocking mode
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 8
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
892744264
842091865
NVMEDIA: H265 : Profile : 1
NVMEDIA: Need to set EMC bandwidth : 126000
NVMEDIA: Need to set EMC bandwidth : 126000
NVMEDIA_ENC: bBlitMode is set to TRUE
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 40960 SrcStride 8192 pDst 0xffffa4f7bfa8 DstStride 8192 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 49152 SrcStride 8192 pDst 0xffffa4f7bfa8 DstStride 8192 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 57344 SrcStride 4206 pDst 0xffffa4f7bfa8 DstStride 4206 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
NVMAP_IOC_READ:1077431815 failed [22]
[1077431815]: Offset 40960 SrcStride 3070 pDst 0xffffa4f7bfa8 DstStride 3070 Count 1
NVMAP_IOC_WRITE:1077431814 failed [22]
Frame 0: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=61550 EncodedBits=0
Frame 1: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=44030 EncodedBits=0
Frame 2: isKeyFrame=1 AvgQP=23 MinQP=15 MaxQP=31 BUfferBits=29173 EncodedBits=1077432
Frame 3: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19496 EncodedBits=0
Frame 4: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19288 EncodedBits=0
Frame 5: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=18171 EncodedBits=455640
Frame 6: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=9844 EncodedBits=0
Frame 7: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=8424 EncodedBits=0
Frame 8: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=4326 EncodedBits=180752
Frame 9: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1011 EncodedBits=0
Frame 10: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=882 EncodedBits=0
Frame 11: isKeyFrame=0 AvgQP=42 MinQP=40 MaxQP=43 BUfferBits=793 EncodedBits=21488
Frame 12: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1504 EncodedBits=0
Frame 13: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=2162 EncodedBits=0
Frame 14: isKeyFrame=0 AvgQP=40 MinQP=39 MaxQP=41 BUfferBits=3034 EncodedBits=53600
Frame 15: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1598 EncodedBits=0
Frame 16: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=836 EncodedBits=0
Frame 17: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=1060 EncodedBits=27952
Frame 18: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=3804 EncodedBits=0
Frame 19: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=3517 EncodedBits=0
Frame 20: isKeyFrame=0 AvgQP=37 MinQP=37 MaxQP=38 BUfferBits=5442 EncodedBits=102104
Frame 21: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=14976 EncodedBits=0
Frame 22: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=15236 EncodedBits=0
Frame 23: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=14431 EncodedBits=357144
Frame 24: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19378 EncodedBits=0
Frame 25: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=19374 EncodedBits=0
Frame 26: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=38 BUfferBits=19114 EncodedBits=462928
Frame 27: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=7127 EncodedBits=0
Frame 28: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=3504 EncodedBits=0
Frame 29: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=2638 EncodedBits=106152
Frame 30: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1019 EncodedBits=0
Frame 31: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=793 EncodedBits=0
Frame 32: isKeyFrame=0 AvgQP=41 MinQP=40 MaxQP=42 BUfferBits=841 EncodedBits=21224
Frame 33: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=2624 EncodedBits=0
Frame 34: isKeyFrame=1 AvgQP=23 MinQP=15 MaxQP=31 BUfferBits=2565 EncodedBits=1077432
Frame 35: isKeyFrame=0 AvgQP=39 MinQP=38 MaxQP=40 BUfferBits=2281 EncodedBits=59760
Frame 36: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=899 EncodedBits=0
Frame 37: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1392 EncodedBits=455640
Frame 38: isKeyFrame=0 AvgQP=38 MinQP=38 MaxQP=39 BUfferBits=1538 EncodedBits=30632
Frame 39: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=5599 EncodedBits=0
Frame 40: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=7145 EncodedBits=180752
Frame 41: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=10482 EncodedBits=185808
Frame 42: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=15512 EncodedBits=0
Frame 43: isKeyFrame=0 AvgQP=42 MinQP=40 MaxQP=43 BUfferBits=16334 EncodedBits=21488
Frame 44: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=17807 EncodedBits=397224
Frame 45: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=13599 EncodedBits=0
Frame 46: isKeyFrame=0 AvgQP=40 MinQP=39 MaxQP=41 BUfferBits=14848 EncodedBits=53600
Frame 47: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=14399 EncodedBits=342768
Frame 48: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=2211 EncodedBits=0
Frame 49: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=1663 EncodedBits=27952
Frame 50: isKeyFrame=0 AvgQP=43 MinQP=43 MaxQP=44 BUfferBits=1163 EncodedBits=40296
Frame 51: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=869 EncodedBits=0
Frame 52: isKeyFrame=0 AvgQP=37 MinQP=37 MaxQP=38 BUfferBits=1356 EncodedBits=102104
Frame 53: isKeyFrame=0 AvgQP=42 MinQP=41 MaxQP=43 BUfferBits=1939 EncodedBits=33312
Frame 54: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1451 EncodedBits=0
Frame 55: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1200 EncodedBits=357144
Frame 56: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=1158 EncodedBits=30472
Frame 57: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=1049 EncodedBits=0
Frame 58: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=38 BUfferBits=1109 EncodedBits=462928
Frame 59: isKeyFrame=0 AvgQP=39 MinQP=39 MaxQP=40 BUfferBits=746 EncodedBits=23232
Frame 60: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=6307 EncodedBits=0
Frame 61: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=10851 EncodedBits=106152
Frame 62: isKeyFrame=0 AvgQP=37 MinQP=37 MaxQP=38 BUfferBits=13417 EncodedBits=244600
Frame 63: isKeyFrame=0 AvgQP=0 MinQP=0 MaxQP=0 BUfferBits=13014 EncodedBits=0
Frame 64: isKeyFrame=0 AvgQP=41 MinQP=40 MaxQP=42 BUfferBits=15909 EncodedBits=21224
Frame 65: isKeyFrame=0 AvgQP=38 MinQP=38 MaxQP=39 BUfferBits=14915 EncodedBits=350704
Frame 66: isKeyFrame=1 AvgQP=23 MinQP=15 MaxQP=31 BUfferBits=9525 EncodedBits=1077432
Frame 67: isKeyFrame=0 AvgQP=39 MinQP=38 MaxQP=40 BUfferBits=10160 EncodedBits=59760
Frame 68: isKeyFrame=0 AvgQP=41 MinQP=41 MaxQP=42 BUfferBits=8608 EncodedBits=226344
Frame 69: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1160 EncodedBits=455640
Frame 70: isKeyFrame=0 AvgQP=38 MinQP=38 MaxQP=39 BUfferBits=857 EncodedBits=30632
Frame 71: isKeyFrame=0 AvgQP=44 MinQP=44 MaxQP=44 BUfferBits=701 EncodedBits=21744
Frame 72: isKeyFrame=0 AvgQP=40 MinQP=40 MaxQP=41 BUfferBits=1908 EncodedBits=180752
Frame 73: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1679 EncodedBits=185808
Frame 74: isKeyFrame=0 AvgQP=42 MinQP=42 MaxQP=43 BUfferBits=1612 EncodedBits=41592
Frame 75: isKeyFrame=0 AvgQP=42 MinQP=40 MaxQP=43 BUfferBits=1261 EncodedBits=21488
Frame 76: isKeyFrame=0 AvgQP=36 MinQP=36 MaxQP=37 BUfferBits=1244 EncodedBits=397224
Frame 77: isKeyFrame=0 AvgQP=41 MinQP=41 MaxQP=42 BUfferBits=596 EncodedBits=24808
Frame 78: isKeyFrame=0 AvgQP=40 MinQP=39 MaxQP=41 BUfferBits=750 EncodedBits=53600

DaneLLL · March 19, 2024, 2:33am

Hi,
You are correct there are issues in slice level encoding. We see two issues:

- -report-metadata does not report correct information in slice level encoding
-slt 1 does not work properly

We will investigate further.

On 5.1.2 or 5.1.3, please run in Slice length type = Bytes:

$ ./video_encode a.yuv 640 480 H265 a.h265 -slt 2 -slen 10000 --sle
$ ./video_encode a.yuv 640 480 H264 a.h264 -slt 2 -slen 10000 --sle

And please manually add print of buffer size in encoder_capture_plane_dq_callback() to check.

system · April 24, 2024, 3:43am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Jetson Video Encoder: unreliable metadata output with Slice Level Encode Jetson AGX Xavier mmapi , nvbugs	6	941	November 23, 2022
Xavier AGX : Video encoding crash Jetson AGX Xavier nvbugs , encoder	13	3000	October 18, 2021
Guide and Instructions on Encoder SliceIntraRefreshEnable and SliceIntraRefreshInterval? Jetson TX2	7	1766	October 18, 2021
TX2 Camera convert/encode using Multimedia API issue Jetson TX2 camera , encoder	16	2014	October 18, 2021
Encode H264 by slice Jetson TX2	7	2201	October 18, 2021
Question about V4L2 API for encode of TX1 Jetson TX1	12	2443	October 18, 2021
xavier encode and decode do not match official description Jetson AGX Xavier	3	1122	October 18, 2021
H265 Encoding works only with I-frames Jetson Xavier NX encoder	7	814	September 20, 2023
Nvv4l2h265enc Issues on Jetson Xavier AGX Developer Kit 32GB Jetson AGX Xavier gstreamer	2	2002	January 26, 2022
Problem adjusting bitrate during H265 encoding Jetson AGX Xavier mmapi	7	976	October 18, 2021

How to set v4l2_enc_hw_tuning_info_type in the xavier encoder?

Related topics