ROI in Orin encoder

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

@DaneLLL

My input video is 5760x1080 and encoder is HEVC with 1 ROI region which is

    "x": "1792",
    "y": "320",
    "width": "1792",
    "height": "248"

When I extract QP values of a frame, I saw ROI region (deep blue color)in the QP heatmap has some extra area (3 on left side and 1 on the right side) which is in the specified ROI region. Why is that?

image
image1424×298 21.5 KB

Thanks

2

*** Please note that this reply is generated by LLM automatically ***

Based on the provided information, it appears that you are experiencing an issue with the QP heatmap of your encoded video, where the ROI region is not accurately represented.

The ROI (Region of Interest) is a feature in HEVC (High Efficiency Video Coding) that allows for different quality settings to be applied to different regions of the frame. In your case, you have specified a single ROI region, but the QP heatmap is showing an extra area of 3 pixels on the left side and 1 pixel on the right side that is not part of the specified ROI region.

There are a few possible explanations for this issue:

  1. Alignment: HEVC has a requirement that the ROI region must be aligned to a specific grid, which is typically 16x16 or 32x32 pixels. If your ROI region is not aligned to this grid, the encoder may be extending the region to the nearest grid boundary, resulting in the extra pixels being included.
  2. Padding: Some encoders may add padding to the ROI region to ensure that it is properly aligned or to simplify the encoding process. This padding can result in extra pixels being included in the ROI region.
  3. Quantization: The QP (Quantization Parameter) values are used to control the quality of the encoded video. However, the QP values may not always accurately reflect the actual quality of the encoded video, especially in regions with complex content.

To resolve this issue, you can try the following:

  1. Verify ROI region alignment: Check that your ROI region is aligned to the required grid boundary. If it is not, adjust the region to align with the grid.
  2. Check encoder settings: Review your encoder settings to see if padding is being added to the ROI region. If padding is enabled, try disabling it to see if it resolves the issue.
  3. Adjust QP values: Experiment with different QP values to see if it affects the accuracy of the ROI region in the QP heatmap.

It’s also worth noting that the Jetson AGX Orin module has a number of features and settings that can affect the encoding process. You may want to consult the documentation for your specific module to see if there are any settings or features that can help resolve this issue.

In comparison to other Jetson modules, the AGX Orin module has a number of unique features and settings that can affect the encoding process. For example, the Jetson TX2 NX module has a different set of encoder settings and features that may not be directly applicable to the AGX Orin module.

In terms of thermal zones, the AGX Orin module has a number of thermal zones that can affect the encoding process. However, it’s unlikely that thermal zones are directly related to the issue you are experiencing with the QP heatmap.

Overall, the issue you are experiencing with the QP heatmap is likely related to the alignment, padding, or quantization of the ROI region, rather than any specific feature or setting of the Jetson AGX Orin module.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

3

Alignment: HEVC has a requirement that the ROI region must be aligned to a specific grid, >>which is typically 16x16 or 32x32 pixels. If your ROI region is not aligned to this grid, the >>encoder may be extending the region to the nearest grid boundary, resulting in the extra pixels >>being included.

I believe my ROI region is aligned with 64x64 which should be OK with 16x16 or 32x32

4

The picture in the first email is frame 1. This is QP heatmap of frame 50 and extra portion is different from frame 1.

image
image1427×697 35.3 KB

6

Hi,
Do you use Jetpack 6.2.2 r36.5? Would like to confirm you use latest Jetpack release for Orin.

7

@DaneLLL @carolyuu

My system is
Package: nvidia-jetpack
Version: 5.1.2-b104
Priority: standard
Section: metapackages
Maintainer: NVIDIA Corporation
Installed-Size: 199 kB
Depends: nvidia-jetpack-runtime (= 5.1.2-b104), nvidia-jetpack-dev (= 5.1.2-b104)
Homepage: Jetson - Embedded AI Computing Platform | NVIDIA Developer
Download-Size: 29.3 kB
APT-Sources: https://repo.download.nvidia.com/jetson/common r35.4/main arm64 Packages
Description: NVIDIA Jetpack Meta Package

Is this a known issue in 5.1.2-b104?

8

This issue can also be reproduced with this sample application too:
/usr/src/jetson_multimedia_api/samples/01_video_encode

FC

9

@DaneLLL @carolyuu

Can you try to reproduce this issue on my version of jetson and the latest one?

Thanks,
FC

10

Hi,
If you would like to stay on 5.1.2, this looks to be fine since the ROI is encoded correctly, although some nearby macroblocks are also encoded with the same q value.

Besides, the latest release for Orin is Jetpack 6.2.2 r36.5. Would suggest upgrade and test on AGX Orin developer kit. Certain issues are fixed in later releases.

11

This is not OK for my application.
I may consider to upgrade to 6,2,2r36,5 (is high effort on my side) if you can make sure there are no neighboring blocks issues.

Thanks,

FC